Parent Log: http://ci.aztec-labs.com/68098685a1fbea35 Command: b3bf99625f6f0a17 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/d71b7a63300e898a3d40f7da703f7cde545afb41 Env: REF_NAME=gh-readonly-queue/next/pr-15138-d5a262aabc20f5384a720a0d831e2ec28badc7f9 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Wed Jun 18 18:05:40 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15138_amd64_x1-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 18:05:45 [18:05:45.005] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.012] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2af61d02fa109ade epochNumber=42 {"provingJobId":"1:BASE_PARITY:2af61d02fa109ade"} 18:05:45 [18:05:45.015] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:bef229ea8d5dd0b7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bef229ea8d5dd0b7"} 18:05:45 [18:05:45.042] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.043] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:90146c09bd5867d9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:90146c09bd5867d9"} 18:05:45 [18:05:45.045] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5b034657d061b35c epochNumber=1 {"provingJobId":"1:BASE_PARITY:5b034657d061b35c"} 18:05:45 [18:05:45.047] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:204d8cf6b42cae48 epochNumber=1 {"provingJobId":"1:BASE_PARITY:204d8cf6b42cae48"} 18:05:45 [18:05:45.049] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:204d8cf6b42cae48 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:204d8cf6b42cae48"} 18:05:45 [18:05:45.052] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.053] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2dd1f6b5f6398151 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2dd1f6b5f6398151"} 18:05:45 [18:05:45.054] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2dd1f6b5f6398151 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2dd1f6b5f6398151"} 18:05:45 [18:05:45.054] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2dd1f6b5f6398151 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2dd1f6b5f6398151"} 18:05:45 [18:05:45.055] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2dd1f6b5f6398151 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2dd1f6b5f6398151"} 18:05:45 [18:05:45.056] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.056] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:70692024049e378b epochNumber=1 {"provingJobId":"1:BASE_PARITY:70692024049e378b"} 18:05:45 [18:05:45.079] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.081] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.081] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0f51d2a2d55ad786 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0f51d2a2d55ad786"} 18:05:45 [18:05:45.082] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0f51d2a2d55ad786 {"provingJobId":"1:BASE_PARITY:0f51d2a2d55ad786"} 18:05:45 [18:05:45.083] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0f51d2a2d55ad786 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0f51d2a2d55ad786"} 18:05:45 [18:05:45.083] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0f51d2a2d55ad786 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0f51d2a2d55ad786"} 18:05:45 [18:05:45.085] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0db1c665885dc091 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0db1c665885dc091"} 18:05:45 [18:05:45.086] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0db1c665885dc091 {"provingJobId":"1:BASE_PARITY:0db1c665885dc091"} 18:05:45 [18:05:45.086] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0db1c665885dc091 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0db1c665885dc091"} 18:05:45 [18:05:45.088] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.089] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:94d329c0ba487918 epochNumber=1 {"provingJobId":"1:BASE_PARITY:94d329c0ba487918"} 18:05:45 [18:05:45.089] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:94d329c0ba487918 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:94d329c0ba487918"} 18:05:45 [18:05:45.090] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:94d329c0ba487918 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:94d329c0ba487918"} 18:05:45 [18:05:45.091] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.092] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:928613908da95ff4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:928613908da95ff4"} 18:05:45 [18:05:45.092] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:928613908da95ff4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:928613908da95ff4"} 18:05:45 [18:05:45.093] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:928613908da95ff4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:928613908da95ff4"} 18:05:45 [18:05:45.094] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.094] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a3f2decf89289ce epochNumber=0 {"provingJobId":"1:BASE_PARITY:6a3f2decf89289ce"} 18:05:45 [18:05:45.095] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6a3f2decf89289ce epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:6a3f2decf89289ce"} 18:05:45 [18:05:45.095] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b79bea1136c66b62 epochNumber=0 {"provingJobId":"1:BASE_PARITY:b79bea1136c66b62"} 18:05:45 [18:05:45.095] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6a3f2decf89289ce epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:6a3f2decf89289ce"} 18:05:45 [18:05:45.096] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5611074b19d77271 epochNumber=0 {"provingJobId":"1:BASE_PARITY:5611074b19d77271"} 18:05:45 [18:05:45.105] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.107] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.108] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fa8a3e20eab91d40 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fa8a3e20eab91d40"} 18:05:45 [18:05:45.108] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:50d176713ff105de epochNumber=1 {"provingJobId":"1:BASE_PARITY:50d176713ff105de"} 18:05:45 [18:05:45.110] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:608ae6340091cf63 epochNumber=1 {"provingJobId":"1:BASE_PARITY:608ae6340091cf63"} 18:05:45 [18:05:45.111] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.111] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2d8ba60eb9805915 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2d8ba60eb9805915"} 18:05:45 [18:05:45.112] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:be4daeb27e4d40a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:be4daeb27e4d40a4"} 18:05:45 [18:05:45.112] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ccac12f09182619 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4ccac12f09182619"} 18:05:45 [18:05:45.113] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f19f646b22b9d6bb epochNumber=1 {"provingJobId":"1:BASE_PARITY:f19f646b22b9d6bb"} 18:05:45 [18:05:45.114] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.114] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed4f58baf271457b epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed4f58baf271457b"} 18:05:45 [18:05:45.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7924d933e59e474a epochNumber=1 {"provingJobId":"1:BASE_PARITY:7924d933e59e474a"} 18:05:45 [18:05:45.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2fff5b61e595e9f epochNumber=2 {"provingJobId":"1:BASE_PARITY:e2fff5b61e595e9f"} 18:05:45 [18:05:45.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5f3843507ec4c600 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5f3843507ec4c600"} 18:05:45 [18:05:45.116] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.116] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4bf6c279a87c6da0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4bf6c279a87c6da0"} 18:05:45 [18:05:45.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f9ae63f66cac2df3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f9ae63f66cac2df3"} 18:05:45 [18:05:45.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:036f928e0ce06f0f epochNumber=2 {"provingJobId":"1:BASE_PARITY:036f928e0ce06f0f"} 18:05:45 [18:05:45.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:277cb21e62da7f9b epochNumber=1 {"provingJobId":"1:BASE_PARITY:277cb21e62da7f9b"} 18:05:45 [18:05:45.119] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:347666ba6e42ec4d epochNumber=1 {"provingJobId":"1:BASE_PARITY:347666ba6e42ec4d"} 18:05:45 [18:05:45.120] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a302b3c15d9ef400 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a302b3c15d9ef400"} 18:05:45 [18:05:45.120] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:347666ba6e42ec4d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:347666ba6e42ec4d"} 18:05:45 [18:05:45.122] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.122] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9599e652a0ebf51b epochNumber=1 {"provingJobId":"1:BASE_PARITY:9599e652a0ebf51b"} 18:05:45 [18:05:45.122] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:129ffcef82e85e88 epochNumber=1 {"provingJobId":"1:BASE_PARITY:129ffcef82e85e88"} 18:05:45 [18:05:45.123] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9599e652a0ebf51b type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:9599e652a0ebf51b"} 18:05:45 [18:05:45.123] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b50a84476a23bb95 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b50a84476a23bb95"} 18:05:45 [18:05:45.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:43da2c3b201eae0f epochNumber=1 {"provingJobId":"1:BASE_PARITY:43da2c3b201eae0f"} 18:05:45 [18:05:45.124] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b50a84476a23bb95 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:b50a84476a23bb95"} 18:05:45 [18:05:45.125] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.126] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1ac075a498dade26 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1ac075a498dade26"} 18:05:45 [18:05:45.126] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:1ac075a498dade26 {"provingJobId":"1:BASE_PARITY:1ac075a498dade26"} 18:05:45 [18:05:45.127] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1ac075a498dade26 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:1ac075a498dade26"} 18:05:45 [18:05:45.127] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d6109b9e10a71bd3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d6109b9e10a71bd3"} 18:05:45 [18:05:45.127] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1ac075a498dade26 has already been completed {"provingJobId":"1:BASE_PARITY:1ac075a498dade26"} 18:05:45 [18:05:45.128] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ffa49815f256cabd epochNumber=1 {"provingJobId":"1:BASE_PARITY:ffa49815f256cabd"} 18:05:45 [18:05:45.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3b079011044ae6b4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3b079011044ae6b4"} 18:05:46 [18:05:46.181] INFO: prover-client:proving-broker Proving Broker started 18:05:46 [18:05:46.182] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ffa49815f256cabd settled=false {"provingJobId":"1:BASE_PARITY:ffa49815f256cabd","status":"pending"} 18:05:46 [18:05:46.183] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3b079011044ae6b4 settled=false {"provingJobId":"1:BASE_PARITY:3b079011044ae6b4","status":"pending"} 18:05:46 [18:05:46.183] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:ffa49815f256cabd type=BASE_PARITY startedAt=1750269945129 older agent has taken job {"provingJobId":"1:BASE_PARITY:ffa49815f256cabd"} 18:05:46 [18:05:46.184] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ffa49815f256cabd type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:ffa49815f256cabd"} 18:05:46 [18:05:46.185] INFO: prover-client:proving-broker Proving Broker started 18:05:46 [18:05:46.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:756960fb245ac64d epochNumber=1 {"provingJobId":"1:BASE_PARITY:756960fb245ac64d"} 18:05:46 [18:05:46.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b1305985559bb8cc epochNumber=2 {"provingJobId":"1:BASE_PARITY:b1305985559bb8cc"} 18:05:47 [18:05:47.186] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.186] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:756960fb245ac64d settled=false {"provingJobId":"1:BASE_PARITY:756960fb245ac64d","status":"pending"} 18:05:47 [18:05:47.187] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b1305985559bb8cc settled=false {"provingJobId":"1:BASE_PARITY:b1305985559bb8cc","status":"pending"} 18:05:47 [18:05:47.187] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:756960fb245ac64d type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:756960fb245ac64d"} 18:05:47 [18:05:47.189] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.190] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:905d928d6728a264 epochNumber=1 {"provingJobId":"1:BASE_PARITY:905d928d6728a264"} 18:05:47 [18:05:47.190] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:de025205c74ece18 epochNumber=2 {"provingJobId":"1:BASE_PARITY:de025205c74ece18"} 18:05:48 [18:05:48.191] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.191] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:905d928d6728a264 settled=false {"provingJobId":"1:BASE_PARITY:905d928d6728a264","status":"pending"} 18:05:48 [18:05:48.192] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:de025205c74ece18 settled=false {"provingJobId":"1:BASE_PARITY:de025205c74ece18","status":"pending"} 18:05:48 [18:05:48.193] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:905d928d6728a264 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:905d928d6728a264"} 18:05:48 [18:05:48.193] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:905d928d6728a264 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:905d928d6728a264"} 18:05:48 [18:05:48.194] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.194] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:627f6db0672d3678 epochNumber=1 {"provingJobId":"1:BASE_PARITY:627f6db0672d3678"} 18:05:48 [18:05:48.194] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5456b3b902224dd epochNumber=2 {"provingJobId":"1:BASE_PARITY:f5456b3b902224dd"} 18:05:48 [18:05:48.195] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:627f6db0672d3678 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:627f6db0672d3678"} 18:05:48 [18:05:48.195] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f5456b3b902224dd type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f5456b3b902224dd"} 18:05:48 [18:05:48.196] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e02f695421692075 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e02f695421692075"} 18:05:48 [18:05:48.197] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:444ebd5c4d95e970 epochNumber=2 {"provingJobId":"1:BASE_PARITY:444ebd5c4d95e970"} 18:05:48 [18:05:48.197] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e02f695421692075 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e02f695421692075"} 18:05:48 [18:05:48.197] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e02f695421692075 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e02f695421692075"} 18:05:48 [18:05:48.197] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:444ebd5c4d95e970 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:444ebd5c4d95e970"} 18:05:48 [18:05:48.198] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.198] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:9852fee7347420ae err=test error {"provingJoId":"1:BASE_PARITY:9852fee7347420ae"} 18:05:48 [18:05:48.199] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.199] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:359600d6b71e41b8 not found {"provingJobId":"1:BASE_PARITY:359600d6b71e41b8"} 18:05:48 [18:05:48.200] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.201] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8816fda561b9422c epochNumber=1 {"provingJobId":"1:BASE_PARITY:8816fda561b9422c"} 18:05:48 [18:05:48.201] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.202] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0b2786eafc857222 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0b2786eafc857222"} 18:05:48 [18:05:48.304] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0b2786eafc857222 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:0b2786eafc857222"} 18:05:48 [18:05:48.312] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.313] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:69101e6d7cf50cca epochNumber=1 {"provingJobId":"1:BASE_PARITY:69101e6d7cf50cca"} 18:05:48 [18:05:48.417] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:69101e6d7cf50cca timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:69101e6d7cf50cca"} 18:05:48 [18:05:48.422] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:42d3439a697b47bc epochNumber=10 {"provingJobId":"1:BASE_PARITY:42d3439a697b47bc"} 18:05:48 [18:05:48.443] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:48 [18:05:48.549] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.550] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e553a561a03baba0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e553a561a03baba0"} 18:05:48 [18:05:48.702] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e553a561a03baba0 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:e553a561a03baba0"} 18:05:48 [18:05:48.754] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.755] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0ee4468f7d826f5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0ee4468f7d826f5"} 18:05:48 [18:05:48.756] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e0ee4468f7d826f5 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:e0ee4468f7d826f5"} 18:05:48 [18:05:48.760] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.761] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b8e3a9afca5bc8f0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b8e3a9afca5bc8f0"} 18:05:48 [18:05:48.763] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b8e3a9afca5bc8f0 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:b8e3a9afca5bc8f0"} 18:05:48 [18:05:48.764] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b8e3a9afca5bc8f0 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:b8e3a9afca5bc8f0"} 18:05:48 [18:05:48.767] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.768] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5d079b84cb2b6ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:b5d079b84cb2b6ea"} 18:05:48 [18:05:48.769] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b5d079b84cb2b6ea type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b5d079b84cb2b6ea"} 18:05:48 [18:05:48.772] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.773] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:99c1022bd0d0ebf2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:99c1022bd0d0ebf2"} 18:05:48 [18:05:48.773] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:99c1022bd0d0ebf2 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:99c1022bd0d0ebf2"} 18:05:48 [18:05:48.774] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:315e87c5c4604cf2 epochNumber=3 {"provingJobId":"1:BASE_PARITY:315e87c5c4604cf2"} 18:05:48 [18:05:48.798] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:48 [18:05:48.800] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:99c1022bd0d0ebf2 err=test error {"provingJoId":"1:BASE_PARITY:99c1022bd0d0ebf2"} 18:05:48 [18:05:48.802] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.802] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e2698d0010777408 settled=false {"provingJobId":"1:BASE_PARITY:e2698d0010777408","status":"pending"} 18:05:48 [18:05:48.803] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:69f823ab67e576ac settled=false {"provingJobId":"1:BASE_PARITY:69f823ab67e576ac","status":"pending"} 18:05:48 [18:05:48.805] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.806] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:88b25c5da5cbb2b4 settled=true {"provingJobId":"1:BASE_PARITY:88b25c5da5cbb2b4","status":"fulfilled"} 18:05:48 [18:05:48.806] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:0898cb0e8498fc35 settled=true {"provingJobId":"2:BASE_PARITY:0898cb0e8498fc35","status":"fulfilled"} 18:05:48 [18:05:48.807] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.808] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:aa1c64f8fbb79ffa settled=true {"provingJobId":"1:BASE_PARITY:aa1c64f8fbb79ffa","status":"fulfilled"} 18:05:48 [18:05:48.808] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1f558ae5cfa3db3e settled=false {"provingJobId":"1:BASE_PARITY:1f558ae5cfa3db3e","status":"pending"} 18:05:48 [18:05:48.809] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.811] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d6fdb5d5c6bf394 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d6fdb5d5c6bf394"} 18:05:48 [18:05:48.813] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.813] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3e177c951a5450fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:3e177c951a5450fb"} 18:05:48 [18:05:48.814] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:3e177c951a5450fb: Error: db error: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:3e177c951a5450fb"} 18:05:48 [18:05:48.815] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.815] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b446625206d75ad2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b446625206d75ad2"} 18:05:48 [18:05:48.816] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b446625206d75ad2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b446625206d75ad2"} 18:05:48 [18:05:48.816] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b446625206d75ad2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b446625206d75ad2"} 18:05:48 [18:05:48.817] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.817] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed451a1a50c3cb30 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed451a1a50c3cb30"} 18:05:48 [18:05:48.817] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ed451a1a50c3cb30 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ed451a1a50c3cb30"} 18:05:48 [18:05:48.818] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ed451a1a50c3cb30 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ed451a1a50c3cb30"} 18:05:48 [18:05:48.818] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:ed451a1a50c3cb30: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:ed451a1a50c3cb30"} 18:05:48 [18:05:48.819] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.820] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:295be3e0833df5e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:295be3e0833df5e7"} 18:05:48 [18:05:48.820] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:295be3e0833df5e7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:295be3e0833df5e7"} 18:05:48 [18:05:48.821] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:295be3e0833df5e7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:295be3e0833df5e7"} 18:05:48 [18:05:48.822] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.822] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:41740888ccdce24b epochNumber=1 {"provingJobId":"1:BASE_PARITY:41740888ccdce24b"} 18:05:48 [18:05:48.823] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:41740888ccdce24b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:41740888ccdce24b"} 18:05:48 [18:05:48.823] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:41740888ccdce24b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:41740888ccdce24b"} 18:05:48 [18:05:48.823] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:41740888ccdce24b jobErr=test error: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:41740888ccdce24b"} 18:05:48 [18:05:48.824] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.825] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9acbd48da13796ea not found {"provingJobId":"1:BASE_PARITY:9acbd48da13796ea"} 18:05:48 [18:05:48.826] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.826] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b9fe5e592c62c98c err=test error {"provingJoId":"1:BASE_PARITY:b9fe5e592c62c98c"} 18:05:48 [18:05:48.828] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.838] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:82d6b67140d22e8f epochNumber=1 {"provingJobId":"1:BASE_PARITY:82d6b67140d22e8f"} 18:05:48 [18:05:48.839] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:82d6b67140d22e8f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:82d6b67140d22e8f"} 18:05:48 [18:05:48.839] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:82d6b67140d22e8f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:82d6b67140d22e8f"} 18:05:48 [18:05:48.852] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:8088287d29963455 epochNumber=2 {"provingJobId":"2:BASE_PARITY:8088287d29963455"} 18:05:48 [18:05:48.853] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:8088287d29963455 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:8088287d29963455"} 18:05:48 [18:05:48.853] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:8088287d29963455 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:8088287d29963455"} 18:05:48 [18:05:48.915] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:89b84a2ef226df34 epochNumber=3 {"provingJobId":"3:BASE_PARITY:89b84a2ef226df34"} 18:05:48 [18:05:48.931] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:48 [18:05:48.976] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:9d7ed3a172660084 epochNumber=4 {"provingJobId":"4:BASE_PARITY:9d7ed3a172660084"} 18:05:48 [18:05:48.982] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:49 [18:05:49.037] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:1ee338c8fe3c7f86 epochNumber=5 {"provingJobId":"5:BASE_PARITY:1ee338c8fe3c7f86"} 18:05:49 [18:05:49.058] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:49 [18:05:49.092] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.092] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0fbfc599d81f20a3 epochNumber=42 {"provingJobId":"1:BASE_PARITY:0fbfc599d81f20a3"} 18:05:49 [18:05:49.093] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testoxMla1/42 with map size 1073741824 18:05:49 [18:05:49.099] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.128] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:5e320ce4c18a2f9f epochNumber=1 {"provingJobId":"1:BASE_PARITY:5e320ce4c18a2f9f"} 18:05:49 [18:05:49.132] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.132] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:26f3f258c300af8c epochNumber=1 {"provingJobId":"1:BASE_PARITY:26f3f258c300af8c"} 18:05:49 [18:05:49.133] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test91c1nL/1 with map size 1073741824 18:05:49 [18:05:49.136] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.160] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3b52fde62352a911 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3b52fde62352a911"} 18:05:49 [18:05:49.172] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.173] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:93b631445feffa33 epochNumber=1 {"provingJobId":"1:BASE_PARITY:93b631445feffa33"} 18:05:49 [18:05:49.174] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testENNoiQ/1 with map size 1073741824 18:05:49 [18:05:49.175] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.194] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:93b631445feffa33 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:93b631445feffa33"} 18:05:49 [18:05:49.198] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d9e33162f2f3fb3a epochNumber=1 {"provingJobId":"1:BASE_PARITY:d9e33162f2f3fb3a"} 18:05:49 [18:05:49.199] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYrAxR7/1 with map size 1073741824 18:05:49 [18:05:49.201] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.227] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d9e33162f2f3fb3a epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d9e33162f2f3fb3a"} 18:05:49 [18:05:49.228] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d9e33162f2f3fb3a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d9e33162f2f3fb3a"} 18:05:49 [18:05:49.235] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d9e33162f2f3fb3a epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d9e33162f2f3fb3a"} 18:05:49 [18:05:49.239] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.239] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e199f427888e5633 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e199f427888e5633"} 18:05:49 [18:05:49.240] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlA1U45/1 with map size 1073741824 18:05:49 [18:05:49.242] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.276] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.280] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.281] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9073ea011d5a23ef epochNumber=1 {"provingJobId":"1:BASE_PARITY:9073ea011d5a23ef"} 18:05:49 [18:05:49.282] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiW3K3U/1 with map size 1073741824 18:05:49 [18:05:49.285] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.304] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:9073ea011d5a23ef {"provingJobId":"1:BASE_PARITY:9073ea011d5a23ef"} 18:05:49 [18:05:49.304] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9073ea011d5a23ef type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9073ea011d5a23ef"} 18:05:49 [18:05:49.305] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9073ea011d5a23ef type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:9073ea011d5a23ef"} 18:05:49 [18:05:49.314] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.315] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ee87099da16ae404 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ee87099da16ae404"} 18:05:49 [18:05:49.316] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testO81Tzi/1 with map size 1073741824 18:05:49 [18:05:49.318] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.335] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ee87099da16ae404 {"provingJobId":"1:BASE_PARITY:ee87099da16ae404"} 18:05:49 [18:05:49.336] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ee87099da16ae404 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:ee87099da16ae404"} 18:05:49 [18:05:49.344] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.345] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c844f55fd6d32101 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c844f55fd6d32101"} 18:05:49 [18:05:49.348] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOPyRYR/1 with map size 1073741824 18:05:49 [18:05:49.357] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.377] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c844f55fd6d32101 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c844f55fd6d32101"} 18:05:49 [18:05:49.377] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c844f55fd6d32101 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c844f55fd6d32101"} 18:05:49 [18:05:49.388] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.389] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5287e90068551079 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5287e90068551079"} 18:05:49 [18:05:49.389] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testa2RHP8/1 with map size 1073741824 18:05:49 [18:05:49.391] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.410] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5287e90068551079 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5287e90068551079"} 18:05:49 [18:05:49.411] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5287e90068551079 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:5287e90068551079"} 18:05:49 [18:05:49.419] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:32e15964fb7adf76 epochNumber=0 {"provingJobId":"1:BASE_PARITY:32e15964fb7adf76"} 18:05:49 [18:05:49.420] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testcmTUzz/0 with map size 1073741824 18:05:49 [18:05:49.422] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.439] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:32e15964fb7adf76 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:32e15964fb7adf76"} 18:05:49 [18:05:49.440] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd8d36eb0343d2d7 epochNumber=0 {"provingJobId":"1:BASE_PARITY:bd8d36eb0343d2d7"} 18:05:49 [18:05:49.441] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:32e15964fb7adf76 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:32e15964fb7adf76"} 18:05:49 [18:05:49.441] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34dd8d9439c86c28 epochNumber=0 {"provingJobId":"1:BASE_PARITY:34dd8d9439c86c28"} 18:05:49 [18:05:49.457] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.460] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.461] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1af8fe2358dafd76 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1af8fe2358dafd76"} 18:05:49 [18:05:49.462] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test28KcjP/2 with map size 1073741824 18:05:49 [18:05:49.464] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.480] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5e3f3887d630d74f epochNumber=1 {"provingJobId":"1:BASE_PARITY:5e3f3887d630d74f"} 18:05:49 [18:05:49.481] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test28KcjP/1 with map size 1073741824 18:05:49 [18:05:49.483] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.506] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.507] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:46e459329ef96386 epochNumber=1 {"provingJobId":"1:BASE_PARITY:46e459329ef96386"} 18:05:49 [18:05:49.508] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7cR435/1 with map size 1073741824 18:05:49 [18:05:49.509] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.528] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.529] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:303f78ccb69b986e epochNumber=1 {"provingJobId":"1:BASE_PARITY:303f78ccb69b986e"} 18:05:49 [18:05:49.529] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrp0Q2G/1 with map size 1073741824 18:05:49 [18:05:49.533] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.548] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dc9cbdebae818a42 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dc9cbdebae818a42"} 18:05:49 [18:05:49.554] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fa89c94e766b3be9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fa89c94e766b3be9"} 18:05:49 [18:05:49.554] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrp0Q2G/2 with map size 1073741824 18:05:49 [18:05:49.556] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.572] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9ecc35859a643e3f epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ecc35859a643e3f"} 18:05:49 [18:05:49.581] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.582] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2cc9652b1b76fcc epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2cc9652b1b76fcc"} 18:05:49 [18:05:49.583] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkNKiug/1 with map size 1073741824 18:05:49 [18:05:49.585] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.602] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4e63f3a5680080f2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4e63f3a5680080f2"} 18:05:49 [18:05:49.608] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f2ca3c49e017a7d6 epochNumber=2 {"provingJobId":"1:BASE_PARITY:f2ca3c49e017a7d6"} 18:05:49 [18:05:49.609] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkNKiug/2 with map size 1073741824 18:05:49 [18:05:49.612] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.628] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:edff9bf7ef01722b epochNumber=1 {"provingJobId":"1:BASE_PARITY:edff9bf7ef01722b"} 18:05:49 [18:05:49.637] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.638] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e1748c3d6325ef3d epochNumber=1 {"provingJobId":"1:BASE_PARITY:e1748c3d6325ef3d"} 18:05:49 [18:05:49.638] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrQP6n1/1 with map size 1073741824 18:05:49 [18:05:49.640] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.658] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:10a9671101a71086 epochNumber=1 {"provingJobId":"1:BASE_PARITY:10a9671101a71086"} 18:05:49 [18:05:49.664] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:36880d0289a4fdd9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:36880d0289a4fdd9"} 18:05:49 [18:05:49.665] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrQP6n1/2 with map size 1073741824 18:05:49 [18:05:49.667] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.682] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a7f6644b07639ffa epochNumber=1 {"provingJobId":"1:BASE_PARITY:a7f6644b07639ffa"} 18:05:49 [18:05:49.693] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.694] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a6aaac7e484face8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a6aaac7e484face8"} 18:05:49 [18:05:49.694] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg6pJu9/1 with map size 1073741824 18:05:49 [18:05:49.696] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.712] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc167c56aedd55f5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc167c56aedd55f5"} 18:05:49 [18:05:49.718] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a6aaac7e484face8 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a6aaac7e484face8"} 18:05:49 [18:05:49.730] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.732] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3164bf719baac4bc epochNumber=1 {"provingJobId":"1:BASE_PARITY:3164bf719baac4bc"} 18:05:49 [18:05:49.732] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0p9dQs/1 with map size 1073741824 18:05:49 [18:05:49.735] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.754] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f5a57d6d4af24b7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f5a57d6d4af24b7"} 18:05:49 [18:05:49.761] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3164bf719baac4bc type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:3164bf719baac4bc"} 18:05:49 [18:05:49.771] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.773] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:21542660c2792b4e epochNumber=1 {"provingJobId":"1:BASE_PARITY:21542660c2792b4e"} 18:05:49 [18:05:49.774] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjOqTk2/1 with map size 1073741824 18:05:49 [18:05:49.776] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.794] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:877a3def6d57ed33 epochNumber=1 {"provingJobId":"1:BASE_PARITY:877a3def6d57ed33"} 18:05:49 [18:05:49.801] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:21542660c2792b4e type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:21542660c2792b4e"} 18:05:49 [18:05:49.806] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.807] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0d6515473f64c1b9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0d6515473f64c1b9"} 18:05:49 [18:05:49.808] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testA8jsGH/1 with map size 1073741824 18:05:49 [18:05:49.809] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.828] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0d6515473f64c1b9 {"provingJobId":"1:BASE_PARITY:0d6515473f64c1b9"} 18:05:49 [18:05:49.828] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0d6515473f64c1b9 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0d6515473f64c1b9"} 18:05:49 [18:05:49.834] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:527cbd9b20d8bc83 epochNumber=1 {"provingJobId":"1:BASE_PARITY:527cbd9b20d8bc83"} 18:05:49 [18:05:49.840] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0d6515473f64c1b9 has already been completed {"provingJobId":"1:BASE_PARITY:0d6515473f64c1b9"} 18:05:49 [18:05:49.844] INFO: prover-client:proving-broker Proving Broker started 18:05:49 [18:05:49.844] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4d2d813f404cd6b epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4d2d813f404cd6b"} 18:05:49 [18:05:49.845] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testa3vzOC/1 with map size 1073741824 18:05:49 [18:05:49.848] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.863] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b9b1a46fb5d3f716 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b9b1a46fb5d3f716"} 18:05:49 [18:05:49.864] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testa3vzOC/2 with map size 1073741824 18:05:49 [18:05:49.867] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:50 [18:05:50.938] INFO: prover-client:proving-broker Proving Broker started 18:05:50 [18:05:50.944] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d4d2d813f404cd6b settled=false {"provingJobId":"1:BASE_PARITY:d4d2d813f404cd6b","status":"pending"} 18:05:50 [18:05:50.948] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b9b1a46fb5d3f716 settled=false {"provingJobId":"1:BASE_PARITY:b9b1a46fb5d3f716","status":"pending"} 18:05:50 [18:05:50.954] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:d4d2d813f404cd6b type=BASE_PARITY startedAt=1750269949886 older agent has taken job {"provingJobId":"1:BASE_PARITY:d4d2d813f404cd6b"} 18:05:50 [18:05:50.956] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d4d2d813f404cd6b type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:d4d2d813f404cd6b"} 18:05:50 [18:05:50.976] INFO: prover-client:proving-broker Proving Broker started 18:05:50 [18:05:50.979] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da24596f320a14e2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:da24596f320a14e2"} 18:05:50 [18:05:50.980] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzjqaoT/1 with map size 1073741824 18:05:50 [18:05:50.988] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:51 [18:05:51.010] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:868e040cff402431 epochNumber=2 {"provingJobId":"1:BASE_PARITY:868e040cff402431"} 18:05:51 [18:05:51.011] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzjqaoT/2 with map size 1073741824 18:05:51 [18:05:51.013] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:52 [18:05:52.035] INFO: prover-client:proving-broker Proving Broker started 18:05:52 [18:05:52.039] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:da24596f320a14e2 settled=false {"provingJobId":"1:BASE_PARITY:da24596f320a14e2","status":"pending"} 18:05:52 [18:05:52.041] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:868e040cff402431 settled=false {"provingJobId":"1:BASE_PARITY:868e040cff402431","status":"pending"} 18:05:52 [18:05:52.043] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:da24596f320a14e2 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:da24596f320a14e2"} 18:05:52 [18:05:52.051] INFO: prover-client:proving-broker Proving Broker started 18:05:52 [18:05:52.053] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac056c68685c4a5a epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac056c68685c4a5a"} 18:05:52 [18:05:52.054] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUQlUuG/1 with map size 1073741824 18:05:52 [18:05:52.060] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:52 [18:05:52.083] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f87f69403023077 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4f87f69403023077"} 18:05:52 [18:05:52.084] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUQlUuG/2 with map size 1073741824 18:05:52 [18:05:52.087] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.109] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.111] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ac056c68685c4a5a settled=false {"provingJobId":"1:BASE_PARITY:ac056c68685c4a5a","status":"pending"} 18:05:53 [18:05:53.113] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4f87f69403023077 settled=false {"provingJobId":"1:BASE_PARITY:4f87f69403023077","status":"pending"} 18:05:53 [18:05:53.115] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ac056c68685c4a5a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ac056c68685c4a5a"} 18:05:53 [18:05:53.115] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ac056c68685c4a5a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ac056c68685c4a5a"} 18:05:53 [18:05:53.127] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.128] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9bec7f0d529305a6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9bec7f0d529305a6"} 18:05:53 [18:05:53.129] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqHa6ud/1 with map size 1073741824 18:05:53 [18:05:53.132] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.150] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c64a1de8d47002cf epochNumber=2 {"provingJobId":"1:BASE_PARITY:c64a1de8d47002cf"} 18:05:53 [18:05:53.151] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqHa6ud/2 with map size 1073741824 18:05:53 [18:05:53.156] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.176] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9bec7f0d529305a6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9bec7f0d529305a6"} 18:05:53 [18:05:53.183] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c64a1de8d47002cf type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c64a1de8d47002cf"} 18:05:53 [18:05:53.194] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1015341ac301210a epochNumber=1 {"provingJobId":"1:BASE_PARITY:1015341ac301210a"} 18:05:53 [18:05:53.198] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUc8YJK/1 with map size 1073741824 18:05:53 [18:05:53.201] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e90387968aa47b23 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e90387968aa47b23"} 18:05:53 [18:05:53.227] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUc8YJK/2 with map size 1073741824 18:05:53 [18:05:53.229] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.251] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1015341ac301210a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1015341ac301210a"} 18:05:53 [18:05:53.252] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1015341ac301210a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1015341ac301210a"} 18:05:53 [18:05:53.257] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e90387968aa47b23 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:e90387968aa47b23"} 18:05:53 [18:05:53.266] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.267] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:dd602d764475f076 err=test error {"provingJoId":"1:BASE_PARITY:dd602d764475f076"} 18:05:53 [18:05:53.271] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.271] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:57b0eb8bb904c625 not found {"provingJobId":"1:BASE_PARITY:57b0eb8bb904c625"} 18:05:53 [18:05:53.275] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.275] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34a6f557cee5af0e epochNumber=1 {"provingJobId":"1:BASE_PARITY:34a6f557cee5af0e"} 18:05:53 [18:05:53.276] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUBwkei/1 with map size 1073741824 18:05:53 [18:05:53.280] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.299] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.299] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e87db33b414fb3a6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e87db33b414fb3a6"} 18:05:53 [18:05:53.300] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testF4xo4J/1 with map size 1073741824 18:05:53 [18:05:53.301] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.426] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e87db33b414fb3a6 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:e87db33b414fb3a6"} 18:05:53 [18:05:53.432] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.433] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:101225e54791c717 epochNumber=1 {"provingJobId":"1:BASE_PARITY:101225e54791c717"} 18:05:53 [18:05:53.436] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4tmUpV/1 with map size 1073741824 18:05:53 [18:05:53.438] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.561] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:101225e54791c717 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:101225e54791c717"} 18:05:53 [18:05:53.564] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e5e8c4643c0daa54 epochNumber=10 {"provingJobId":"1:BASE_PARITY:e5e8c4643c0daa54"} 18:05:53 [18:05:53.565] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test4tmUpV/10 with map size 1073741824 18:05:53 [18:05:53.568] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.587] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:53 [18:05:53.587] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:53 [18:05:53.714] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.714] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:af3802462b2642dd epochNumber=1 {"provingJobId":"1:BASE_PARITY:af3802462b2642dd"} 18:05:53 [18:05:53.716] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test26ZYCi/1 with map size 1073741824 18:05:53 [18:05:53.719] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.893] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:af3802462b2642dd timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:af3802462b2642dd"} 18:05:53 [18:05:53.943] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.944] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:714d4c9a793746f8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:714d4c9a793746f8"} 18:05:53 [18:05:53.946] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testthTfll/1 with map size 1073741824 18:05:53 [18:05:53.949] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:53 [18:05:53.968] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:714d4c9a793746f8 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:714d4c9a793746f8"} 18:05:53 [18:05:53.973] INFO: prover-client:proving-broker Proving Broker started 18:05:53 [18:05:53.974] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67d01a54dc3222bc epochNumber=1 {"provingJobId":"1:BASE_PARITY:67d01a54dc3222bc"} 18:05:53 [18:05:53.975] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxKdqpc/1 with map size 1073741824 18:05:53 [18:05:53.976] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.003] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:67d01a54dc3222bc type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:67d01a54dc3222bc"} 18:05:54 [18:05:54.004] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:67d01a54dc3222bc type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:67d01a54dc3222bc"} 18:05:54 [18:05:54.015] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.016] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c99cc33dc51f8e7b epochNumber=1 {"provingJobId":"1:BASE_PARITY:c99cc33dc51f8e7b"} 18:05:54 [18:05:54.016] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbTdwGm/1 with map size 1073741824 18:05:54 [18:05:54.018] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.037] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c99cc33dc51f8e7b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c99cc33dc51f8e7b"} 18:05:54 [18:05:54.048] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7fea2b45e422234a epochNumber=1 {"provingJobId":"1:BASE_PARITY:7fea2b45e422234a"} 18:05:54 [18:05:54.049] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJJ48t5/1 with map size 1073741824 18:05:54 [18:05:54.052] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.075] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7fea2b45e422234a type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:7fea2b45e422234a"} 18:05:54 [18:05:54.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f86c45a9d1f4fd59 epochNumber=3 {"provingJobId":"1:BASE_PARITY:f86c45a9d1f4fd59"} 18:05:54 [18:05:54.077] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testJJ48t5/3 with map size 1073741824 18:05:54 [18:05:54.091] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.123] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:54 [18:05:54.123] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:54 [18:05:54.137] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7fea2b45e422234a err=test error {"provingJoId":"1:BASE_PARITY:7fea2b45e422234a"} 18:05:54 [18:05:54.145] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuPqSp7/1 with map size 1073741824 18:05:54 [18:05:54.148] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.178] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testuPqSp7/2 with map size 1073741824 18:05:54 [18:05:54.181] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.201] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.203] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a2edb2755254cece settled=false {"provingJobId":"1:BASE_PARITY:a2edb2755254cece","status":"pending"} 18:05:54 [18:05:54.205] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:541b567e51cc913d settled=false {"provingJobId":"1:BASE_PARITY:541b567e51cc913d","status":"pending"} 18:05:54 [18:05:54.213] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPutI2n/1 with map size 1073741824 18:05:54 [18:05:54.216] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.244] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPutI2n/2 with map size 1073741824 18:05:54 [18:05:54.253] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.289] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.292] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3c7c8f9a15c22aec settled=true {"provingJobId":"1:BASE_PARITY:3c7c8f9a15c22aec","status":"fulfilled"} 18:05:54 [18:05:54.294] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:82e37f42c731bae5 settled=true {"provingJobId":"2:BASE_PARITY:82e37f42c731bae5","status":"fulfilled"} 18:05:54 [18:05:54.300] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testif4m7S/1 with map size 1073741824 18:05:54 [18:05:54.302] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.336] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testif4m7S/2 with map size 1073741824 18:05:54 [18:05:54.338] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.369] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.373] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:54d6a613a501ca94 settled=true {"provingJobId":"1:BASE_PARITY:54d6a613a501ca94","status":"fulfilled"} 18:05:54 [18:05:54.376] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:86ab0cdf8d6eace4 settled=false {"provingJobId":"1:BASE_PARITY:86ab0cdf8d6eace4","status":"pending"} 18:05:54 [18:05:54.385] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.386] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3f0eb98c3bb2452 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e3f0eb98c3bb2452"} 18:05:54 [18:05:54.387] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmvMQkS/1 with map size 1073741824 18:05:54 [18:05:54.390] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.418] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.419] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9c0ac3e19cf9fa01 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9c0ac3e19cf9fa01"} 18:05:54 [18:05:54.420] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:9c0ac3e19cf9fa01: Error: db error: Error: db error 18:05:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:9c0ac3e19cf9fa01"} 18:05:54 [18:05:54.425] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.426] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:94a6bb3086025a31 epochNumber=1 {"provingJobId":"1:BASE_PARITY:94a6bb3086025a31"} 18:05:54 [18:05:54.429] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzhwnSa/1 with map size 1073741824 18:05:54 [18:05:54.432] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.466] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:94a6bb3086025a31 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:94a6bb3086025a31"} 18:05:54 [18:05:54.466] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:94a6bb3086025a31 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:94a6bb3086025a31"} 18:05:54 [18:05:54.484] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.484] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59d3a6ea9b7f7e70 epochNumber=1 {"provingJobId":"1:BASE_PARITY:59d3a6ea9b7f7e70"} 18:05:54 [18:05:54.486] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDCyOX5/1 with map size 1073741824 18:05:54 [18:05:54.488] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.510] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:59d3a6ea9b7f7e70 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:59d3a6ea9b7f7e70"} 18:05:54 [18:05:54.511] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:59d3a6ea9b7f7e70 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:59d3a6ea9b7f7e70"} 18:05:54 [18:05:54.512] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:59d3a6ea9b7f7e70: Error: db error 18:05:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:59d3a6ea9b7f7e70"} 18:05:54 [18:05:54.516] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.517] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd5c433dbfe334b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bd5c433dbfe334b0"} 18:05:54 [18:05:54.519] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testarIkDs/1 with map size 1073741824 18:05:54 [18:05:54.521] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.547] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bd5c433dbfe334b0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bd5c433dbfe334b0"} 18:05:54 [18:05:54.547] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bd5c433dbfe334b0 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:bd5c433dbfe334b0"} 18:05:54 [18:05:54.566] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.567] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:89fef25b308034fc epochNumber=1 {"provingJobId":"1:BASE_PARITY:89fef25b308034fc"} 18:05:54 [18:05:54.569] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testC7cgke/1 with map size 1073741824 18:05:54 [18:05:54.576] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.605] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:89fef25b308034fc type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:89fef25b308034fc"} 18:05:54 [18:05:54.606] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:89fef25b308034fc type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:89fef25b308034fc"} 18:05:54 [18:05:54.606] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:89fef25b308034fc jobErr=test error: Error: db error 18:05:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:89fef25b308034fc"} 18:05:54 [18:05:54.610] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.611] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d2a2f51cf1bff7a4 not found {"provingJobId":"1:BASE_PARITY:d2a2f51cf1bff7a4"} 18:05:54 [18:05:54.615] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.616] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0c5e54e96135b753 err=test error {"provingJoId":"1:BASE_PARITY:0c5e54e96135b753"} 18:05:54 [18:05:54.624] INFO: prover-client:proving-broker Proving Broker started 18:05:54 [18:05:54.634] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:be03fbf722f27d63 epochNumber=1 {"provingJobId":"1:BASE_PARITY:be03fbf722f27d63"} 18:05:54 [18:05:54.637] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNbaD94/1 with map size 1073741824 18:05:54 [18:05:54.642] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.677] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:be03fbf722f27d63 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:be03fbf722f27d63"} 18:05:54 [18:05:54.677] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:be03fbf722f27d63 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:be03fbf722f27d63"} 18:05:54 [18:05:54.698] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:7847af58be873dc9 epochNumber=2 {"provingJobId":"2:BASE_PARITY:7847af58be873dc9"} 18:05:54 [18:05:54.700] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNbaD94/2 with map size 1073741824 18:05:54 [18:05:54.702] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.727] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:7847af58be873dc9 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:7847af58be873dc9"} 18:05:54 [18:05:54.728] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:7847af58be873dc9 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:7847af58be873dc9"} 18:05:54 [18:05:54.797] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:2cab43702066dc00 epochNumber=3 {"provingJobId":"3:BASE_PARITY:2cab43702066dc00"} 18:05:54 [18:05:54.800] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testNbaD94/3 with map size 1073741824 18:05:54 [18:05:54.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.831] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:54 [18:05:54.832] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:54 [18:05:54.886] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:d8b96180378970b0 epochNumber=4 {"provingJobId":"4:BASE_PARITY:d8b96180378970b0"} 18:05:54 [18:05:54.887] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testNbaD94/4 with map size 1073741824 18:05:54 [18:05:54.891] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:54 [18:05:54.936] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:54 [18:05:54.937] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 18:05:54 [18:05:54.982] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:6f37d393fbca3a8d epochNumber=5 {"provingJobId":"5:BASE_PARITY:6f37d393fbca3a8d"} 18:05:54 [18:05:54.983] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testNbaD94/5 with map size 1073741824 18:05:54 [18:05:54.988] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:55 [18:05:55.017] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:55 [18:05:55.018] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 18:05:55 PASS src/proving_broker/proving_broker.test.ts (13.433 s) 18:05:55 ProvingBroker 18:05:55 Producer API 18:05:55 ✓ refuses stale jobs (40 ms) 18:05:55 ✓ enqueues jobs (4 ms) 18:05:55 ✓ ignores duplicate jobs (5 ms) 18:05:55 ✓ reports correct status when enqueuing repeat jobs (4 ms) 18:05:55 ✓ throws an error in case of duplicate job IDs (22 ms) 18:05:55 ✓ returns not-found status for non-existing jobs (2 ms) 18:05:55 ✓ cancels jobs in queue (4 ms) 18:05:55 ✓ cancels jobs in-progress (3 ms) 18:05:55 ✓ returns job result if successful (3 ms) 18:05:55 ✓ returns job error if failed (2 ms) 18:05:55 ✓ correctly returns job status for concurrent writes (10 ms) 18:05:55 ✓ refuses stale jobs (41 ms) 18:05:55 ✓ enqueues jobs (40 ms) 18:05:55 ✓ ignores duplicate jobs (26 ms) 18:05:55 ✓ reports correct status when enqueuing repeat jobs (40 ms) 18:05:55 ✓ throws an error in case of duplicate job IDs (34 ms) 18:05:55 ✓ returns not-found status for non-existing jobs (5 ms) 18:05:55 ✓ cancels jobs in queue (36 ms) 18:05:55 ✓ cancels jobs in-progress (29 ms) 18:05:55 ✓ returns job result if successful (44 ms) 18:05:55 ✓ returns job error if failed (31 ms) 18:05:55 ✓ correctly returns job status for concurrent writes (36 ms) 18:05:55 Consumer API 18:05:55 ✓ returns undefined if no jobs are available (2 ms) 18:05:55 ✓ returns jobs in priority order (2 ms) 18:05:55 ✓ returns undefined if no jobs are available for the given allowList (2 ms) 18:05:55 ✓ returns a job if it is in the allowList (2 ms) 18:05:55 ✓ returns the most important job if it is in the allowList (2 ms) 18:05:55 ✓ returns any job if filter is empty (2 ms) 18:05:55 ✓ returns a new job when reporting job success (3 ms) 18:05:55 ✓ returns a new job when reporting permanent error (2 ms) 18:05:55 ✓ returns a new job when reporting retry-able error (2 ms) 18:05:55 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 18:05:55 ✓ returns a new job if job is already in progress elsewhere (1056 ms) 18:05:55 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 18:05:55 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 18:05:55 ✓ tracks job result if in progress (2 ms) 18:05:55 ✓ tracks job result even if job is in queue (2 ms) 18:05:55 ✓ ignores reported job error if unknown job (1 ms) 18:05:55 ✓ ignores job result if unknown job (1 ms) 18:05:55 ✓ returns undefined if no jobs are available (4 ms) 18:05:55 ✓ returns jobs in priority order (46 ms) 18:05:55 ✓ returns undefined if no jobs are available for the given allowList (21 ms) 18:05:55 ✓ returns a job if it is in the allowList (53 ms) 18:05:55 ✓ returns the most important job if it is in the allowList (55 ms) 18:05:55 ✓ returns any job if filter is empty (56 ms) 18:05:55 ✓ returns a new job when reporting job success (34 ms) 18:05:55 ✓ returns a new job when reporting permanent error (43 ms) 18:05:55 ✓ returns a new job when reporting retry-able error (35 ms) 18:05:55 ✓ returns a new job when reporting progress if current one is cancelled (37 ms) 18:05:55 ✓ returns a new job if job is already in progress elsewhere (1117 ms) 18:05:55 ✓ avoids sending the same job to a new agent after a restart (1086 ms) 18:05:55 ✓ avoids sending a completed job to a new agent after a restart (1078 ms) 18:05:55 ✓ tracks job result if in progress (67 ms) 18:05:55 ✓ tracks job result even if job is in queue (72 ms) 18:05:55 ✓ ignores reported job error if unknown job (3 ms) 18:05:55 ✓ ignores job result if unknown job (3 ms) 18:05:55 Timeouts 18:05:55 ✓ tracks in progress jobs (1 ms) 18:05:55 ✓ re-enqueues jobs that time out (110 ms) 18:05:55 ✓ cancel stale jobs that time out (236 ms) 18:05:55 ✓ keeps the jobs in progress while it is alive (204 ms) 18:05:55 ✓ tracks in progress jobs (24 ms) 18:05:55 ✓ re-enqueues jobs that time out (132 ms) 18:05:55 ✓ cancel stale jobs that time out (281 ms) 18:05:55 ✓ keeps the jobs in progress while it is alive (229 ms) 18:05:55 Retries 18:05:55 ✓ retries jobs (5 ms) 18:05:55 ✓ retries up to a maximum number of times (7 ms) 18:05:55 ✓ passing retry=false does not retry (5 ms) 18:05:55 ✓ does not retry if job is stale (30 ms) 18:05:55 ✓ retries jobs (29 ms) 18:05:55 ✓ retries up to a maximum number of times (42 ms) 18:05:55 ✓ passing retry=false does not retry (32 ms) 18:05:55 ✓ does not retry if job is stale (93 ms) 18:05:55 Database management 18:05:55 ✓ re-enqueues proof requests on start (3 ms) 18:05:55 ✓ restores proof results on start (2 ms) 18:05:55 ✓ only re-enqueues unfinished jobs (2 ms) 18:05:55 ✓ saves job when enqueued (3 ms) 18:05:55 ✓ does not retain job if database fails to save (3 ms) 18:05:55 ✓ saves job result (1 ms) 18:05:55 ✓ saves result even if database fails to save (2 ms) 18:05:55 ✓ saves job error (2 ms) 18:05:55 ✓ saves job error even if database fails to save (2 ms) 18:05:55 ✓ does not save job result if job is unknown (1 ms) 18:05:55 ✓ does not save job error if job is unknown (1 ms) 18:05:55 ✓ cleans up old jobs periodically (262 ms) 18:05:55 ✓ re-enqueues proof requests on start (69 ms) 18:05:55 ✓ restores proof results on start (88 ms) 18:05:55 ✓ only re-enqueues unfinished jobs (84 ms) 18:05:55 ✓ saves job when enqueued (33 ms) 18:05:55 ✓ does not retain job if database fails to save (7 ms) 18:05:55 ✓ saves job result (57 ms) 18:05:55 ✓ saves result even if database fails to save (35 ms) 18:05:55 ✓ saves job error (43 ms) 18:05:55 ✓ saves job error even if database fails to save (49 ms) 18:05:55 ✓ does not save job result if job is unknown (3 ms) 18:05:55 ✓ does not save job error if job is unknown (4 ms) 18:05:55 ✓ cleans up old jobs periodically (446 ms) 18:05:55 18:05:55 Test Suites: 1 passed, 1 total 18:05:55 Tests: 96 passed, 96 total 18:05:55 Snapshots: 0 total 18:05:55 Time: 13.546 s 18:05:55 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 18:05:55 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?