Parent Log: http://ci.aztec-labs.com/bbd24c41c3ef4011 Command: 5e456509c4335308 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/1b10c7d500621ca2e4c0db764df67f3085beb282 Env: REF_NAME=gh-readonly-queue/next/pr-15229-99e234fe4473771b32124e0402885610a6a3f98a CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Tue Jun 24 10:07:04 UTC 2025 System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15229_arm64_a1-fast Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 10:07:07 [10:07:07.130] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.135] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6963699e67090b91 epochNumber=42 {"provingJobId":"1:BASE_PARITY:6963699e67090b91"} 10:07:07 [10:07:07.138] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:0e065e97c44d87d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0e065e97c44d87d7"} 10:07:07 [10:07:07.164] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b82cc5e839563893 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b82cc5e839563893"} 10:07:07 [10:07:07.166] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b7546ccc805e8212 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b7546ccc805e8212"} 10:07:07 [10:07:07.168] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.169] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9569d34fe09a4f59 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9569d34fe09a4f59"} 10:07:07 [10:07:07.170] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9569d34fe09a4f59 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:9569d34fe09a4f59"} 10:07:07 [10:07:07.172] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.173] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c49d29f918d43231 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c49d29f918d43231"} 10:07:07 [10:07:07.174] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c49d29f918d43231 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c49d29f918d43231"} 10:07:07 [10:07:07.175] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c49d29f918d43231 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c49d29f918d43231"} 10:07:07 [10:07:07.175] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c49d29f918d43231 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c49d29f918d43231"} 10:07:07 [10:07:07.177] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.177] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:55d438ca3227d3fe epochNumber=1 {"provingJobId":"1:BASE_PARITY:55d438ca3227d3fe"} 10:07:07 [10:07:07.196] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.198] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:75ed26faf9931a59 epochNumber=1 {"provingJobId":"1:BASE_PARITY:75ed26faf9931a59"} 10:07:07 [10:07:07.199] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:75ed26faf9931a59 {"provingJobId":"1:BASE_PARITY:75ed26faf9931a59"} 10:07:07 [10:07:07.199] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:75ed26faf9931a59 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:75ed26faf9931a59"} 10:07:07 [10:07:07.200] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:75ed26faf9931a59 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:75ed26faf9931a59"} 10:07:07 [10:07:07.201] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.202] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3388c6fc2bc89838 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3388c6fc2bc89838"} 10:07:07 [10:07:07.202] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3388c6fc2bc89838 {"provingJobId":"1:BASE_PARITY:3388c6fc2bc89838"} 10:07:07 [10:07:07.203] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3388c6fc2bc89838 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:3388c6fc2bc89838"} 10:07:07 [10:07:07.204] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8679113f12f61ad5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8679113f12f61ad5"} 10:07:07 [10:07:07.206] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8679113f12f61ad5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8679113f12f61ad5"} 10:07:07 [10:07:07.206] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8679113f12f61ad5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:8679113f12f61ad5"} 10:07:07 [10:07:07.207] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.208] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9acd4d6aadee0980 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9acd4d6aadee0980"} 10:07:07 [10:07:07.209] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9acd4d6aadee0980 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9acd4d6aadee0980"} 10:07:07 [10:07:07.209] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9acd4d6aadee0980 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:9acd4d6aadee0980"} 10:07:07 [10:07:07.210] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0307e6fd8ce4c44d epochNumber=0 {"provingJobId":"1:BASE_PARITY:0307e6fd8ce4c44d"} 10:07:07 [10:07:07.211] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0307e6fd8ce4c44d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:0307e6fd8ce4c44d"} 10:07:07 [10:07:07.211] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6e0a1a7115736a77 epochNumber=0 {"provingJobId":"1:BASE_PARITY:6e0a1a7115736a77"} 10:07:07 [10:07:07.211] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0307e6fd8ce4c44d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:0307e6fd8ce4c44d"} 10:07:07 [10:07:07.211] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0200388702b6c233 epochNumber=0 {"provingJobId":"1:BASE_PARITY:0200388702b6c233"} 10:07:07 [10:07:07.213] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.214] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:68cd974684edda76 epochNumber=2 {"provingJobId":"1:BASE_PARITY:68cd974684edda76"} 10:07:07 [10:07:07.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dedd2cc24e37452b epochNumber=1 {"provingJobId":"1:BASE_PARITY:dedd2cc24e37452b"} 10:07:07 [10:07:07.220] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.220] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f819935cbfb71c25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f819935cbfb71c25"} 10:07:07 [10:07:07.221] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.222] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b98d2114cf35f034 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b98d2114cf35f034"} 10:07:07 [10:07:07.222] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34be5d1eab55230d epochNumber=1 {"provingJobId":"1:BASE_PARITY:34be5d1eab55230d"} 10:07:07 [10:07:07.222] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e1d53d891bc4a3ee epochNumber=2 {"provingJobId":"1:BASE_PARITY:e1d53d891bc4a3ee"} 10:07:07 [10:07:07.223] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d38bd75e172cb572 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d38bd75e172cb572"} 10:07:07 [10:07:07.224] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.224] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c67912e4366ee6d1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c67912e4366ee6d1"} 10:07:07 [10:07:07.224] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c040bc3ad44fdf1b epochNumber=1 {"provingJobId":"1:BASE_PARITY:c040bc3ad44fdf1b"} 10:07:07 [10:07:07.225] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3fff8b241e948b85 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3fff8b241e948b85"} 10:07:07 [10:07:07.225] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:92856c79a2dcd408 epochNumber=1 {"provingJobId":"1:BASE_PARITY:92856c79a2dcd408"} 10:07:07 [10:07:07.226] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.226] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:76ef6facc6072a79 epochNumber=1 {"provingJobId":"1:BASE_PARITY:76ef6facc6072a79"} 10:07:07 [10:07:07.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e1a14154610ffd4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e1a14154610ffd4"} 10:07:07 [10:07:07.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f94417c02e0c942 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4f94417c02e0c942"} 10:07:07 [10:07:07.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5b63a97fae5a240c epochNumber=1 {"provingJobId":"1:BASE_PARITY:5b63a97fae5a240c"} 10:07:07 [10:07:07.228] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b1604443fdef49c epochNumber=1 {"provingJobId":"1:BASE_PARITY:6b1604443fdef49c"} 10:07:07 [10:07:07.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0aa942829bb491ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:0aa942829bb491ee"} 10:07:07 [10:07:07.230] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6b1604443fdef49c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6b1604443fdef49c"} 10:07:07 [10:07:07.231] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.231] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9f600d534905c22e epochNumber=1 {"provingJobId":"1:BASE_PARITY:9f600d534905c22e"} 10:07:07 [10:07:07.231] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d04875a5fd73b7c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d04875a5fd73b7c8"} 10:07:07 [10:07:07.232] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9f600d534905c22e type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:9f600d534905c22e"} 10:07:07 [10:07:07.233] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.233] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d91c1c6b8c78ed19 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d91c1c6b8c78ed19"} 10:07:07 [10:07:07.233] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:adda2a8e820b0c80 epochNumber=1 {"provingJobId":"1:BASE_PARITY:adda2a8e820b0c80"} 10:07:07 [10:07:07.234] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d91c1c6b8c78ed19 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:d91c1c6b8c78ed19"} 10:07:07 [10:07:07.235] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.235] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2fa0c74e2916364b epochNumber=1 {"provingJobId":"1:BASE_PARITY:2fa0c74e2916364b"} 10:07:07 [10:07:07.236] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2fa0c74e2916364b {"provingJobId":"1:BASE_PARITY:2fa0c74e2916364b"} 10:07:07 [10:07:07.236] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2fa0c74e2916364b type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:2fa0c74e2916364b"} 10:07:07 [10:07:07.236] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9f509d38cf9341ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:9f509d38cf9341ea"} 10:07:07 [10:07:07.237] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2fa0c74e2916364b has already been completed {"provingJobId":"1:BASE_PARITY:2fa0c74e2916364b"} 10:07:07 [10:07:07.238] INFO: prover-client:proving-broker Proving Broker started 10:07:07 [10:07:07.238] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8b536dea0c715b38 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8b536dea0c715b38"} 10:07:07 [10:07:07.238] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c030adc690b0447c epochNumber=2 {"provingJobId":"1:BASE_PARITY:c030adc690b0447c"} 10:07:08 [10:07:08.289] INFO: prover-client:proving-broker Proving Broker started 10:07:08 [10:07:08.289] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8b536dea0c715b38 settled=false {"provingJobId":"1:BASE_PARITY:8b536dea0c715b38","status":"pending"} 10:07:08 [10:07:08.290] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c030adc690b0447c settled=false {"provingJobId":"1:BASE_PARITY:c030adc690b0447c","status":"pending"} 10:07:08 [10:07:08.290] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:8b536dea0c715b38 type=BASE_PARITY startedAt=1750759627239 older agent has taken job {"provingJobId":"1:BASE_PARITY:8b536dea0c715b38"} 10:07:08 [10:07:08.291] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8b536dea0c715b38 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:8b536dea0c715b38"} 10:07:08 [10:07:08.292] INFO: prover-client:proving-broker Proving Broker started 10:07:08 [10:07:08.292] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11d51b609c8e2931 epochNumber=1 {"provingJobId":"1:BASE_PARITY:11d51b609c8e2931"} 10:07:08 [10:07:08.293] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5b59ae25777b9061 epochNumber=2 {"provingJobId":"1:BASE_PARITY:5b59ae25777b9061"} 10:07:09 [10:07:09.293] INFO: prover-client:proving-broker Proving Broker started 10:07:09 [10:07:09.294] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:11d51b609c8e2931 settled=false {"provingJobId":"1:BASE_PARITY:11d51b609c8e2931","status":"pending"} 10:07:09 [10:07:09.294] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5b59ae25777b9061 settled=false {"provingJobId":"1:BASE_PARITY:5b59ae25777b9061","status":"pending"} 10:07:09 [10:07:09.295] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:11d51b609c8e2931 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:11d51b609c8e2931"} 10:07:09 [10:07:09.296] INFO: prover-client:proving-broker Proving Broker started 10:07:09 [10:07:09.296] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:641ad081523976a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:641ad081523976a4"} 10:07:09 [10:07:09.296] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9eb6d8b19646b74a epochNumber=2 {"provingJobId":"1:BASE_PARITY:9eb6d8b19646b74a"} 10:07:10 [10:07:10.297] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.298] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:641ad081523976a4 settled=false {"provingJobId":"1:BASE_PARITY:641ad081523976a4","status":"pending"} 10:07:10 [10:07:10.298] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9eb6d8b19646b74a settled=false {"provingJobId":"1:BASE_PARITY:9eb6d8b19646b74a","status":"pending"} 10:07:10 [10:07:10.299] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:641ad081523976a4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:641ad081523976a4"} 10:07:10 [10:07:10.299] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:641ad081523976a4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:641ad081523976a4"} 10:07:10 [10:07:10.300] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.301] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c9d019ad22e5b39e epochNumber=1 {"provingJobId":"1:BASE_PARITY:c9d019ad22e5b39e"} 10:07:10 [10:07:10.301] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a04a40f2721b7d0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3a04a40f2721b7d0"} 10:07:10 [10:07:10.301] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c9d019ad22e5b39e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c9d019ad22e5b39e"} 10:07:10 [10:07:10.302] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3a04a40f2721b7d0 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3a04a40f2721b7d0"} 10:07:10 [10:07:10.303] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.303] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b1edb5c14a888f9b epochNumber=1 {"provingJobId":"1:BASE_PARITY:b1edb5c14a888f9b"} 10:07:10 [10:07:10.304] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c35679f202d2e7c4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:c35679f202d2e7c4"} 10:07:10 [10:07:10.304] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b1edb5c14a888f9b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b1edb5c14a888f9b"} 10:07:10 [10:07:10.304] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b1edb5c14a888f9b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b1edb5c14a888f9b"} 10:07:10 [10:07:10.305] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c35679f202d2e7c4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c35679f202d2e7c4"} 10:07:10 [10:07:10.305] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.306] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:db69e58885999536 err=test error {"provingJoId":"1:BASE_PARITY:db69e58885999536"} 10:07:10 [10:07:10.307] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.307] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:17ee04760817ca1c not found {"provingJobId":"1:BASE_PARITY:17ee04760817ca1c"} 10:07:10 [10:07:10.309] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.309] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ee338e538148fed epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ee338e538148fed"} 10:07:10 [10:07:10.310] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.310] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:087a242c06b54cb6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:087a242c06b54cb6"} 10:07:10 [10:07:10.412] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:087a242c06b54cb6 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:087a242c06b54cb6"} 10:07:10 [10:07:10.419] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9eb13f04a5a4bbe5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9eb13f04a5a4bbe5"} 10:07:10 [10:07:10.521] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9eb13f04a5a4bbe5 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:9eb13f04a5a4bbe5"} 10:07:10 [10:07:10.528] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:109db2dda846a6a6 epochNumber=10 {"provingJobId":"1:BASE_PARITY:109db2dda846a6a6"} 10:07:10 [10:07:10.546] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:10 [10:07:10.654] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.654] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd19ffd13a6a98ab epochNumber=1 {"provingJobId":"1:BASE_PARITY:bd19ffd13a6a98ab"} 10:07:10 [10:07:10.806] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bd19ffd13a6a98ab timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:bd19ffd13a6a98ab"} 10:07:10 [10:07:10.857] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.857] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8311a9da958ace6f epochNumber=1 {"provingJobId":"1:BASE_PARITY:8311a9da958ace6f"} 10:07:10 [10:07:10.858] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8311a9da958ace6f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:8311a9da958ace6f"} 10:07:10 [10:07:10.859] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.860] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5c365b7469e01d1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b5c365b7469e01d1"} 10:07:10 [10:07:10.860] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b5c365b7469e01d1 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:b5c365b7469e01d1"} 10:07:10 [10:07:10.861] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b5c365b7469e01d1 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:b5c365b7469e01d1"} 10:07:10 [10:07:10.862] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.862] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a1cd94ac56732fa epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a1cd94ac56732fa"} 10:07:10 [10:07:10.862] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1a1cd94ac56732fa type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:1a1cd94ac56732fa"} 10:07:10 [10:07:10.863] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.864] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9e04c5b9898f43bc epochNumber=1 {"provingJobId":"1:BASE_PARITY:9e04c5b9898f43bc"} 10:07:10 [10:07:10.864] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9e04c5b9898f43bc type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:9e04c5b9898f43bc"} 10:07:10 [10:07:10.865] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6ca498d6c66df36a epochNumber=3 {"provingJobId":"1:BASE_PARITY:6ca498d6c66df36a"} 10:07:10 [10:07:10.888] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:10 [10:07:10.890] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:9e04c5b9898f43bc err=test error {"provingJoId":"1:BASE_PARITY:9e04c5b9898f43bc"} 10:07:10 [10:07:10.891] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.891] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a0e0f433647f6897 settled=false {"provingJobId":"1:BASE_PARITY:a0e0f433647f6897","status":"pending"} 10:07:10 [10:07:10.892] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4716a36e47e1b03c settled=false {"provingJobId":"1:BASE_PARITY:4716a36e47e1b03c","status":"pending"} 10:07:10 [10:07:10.894] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.894] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6c5a5efe9fd8ccd9 settled=true {"provingJobId":"1:BASE_PARITY:6c5a5efe9fd8ccd9","status":"fulfilled"} 10:07:10 [10:07:10.894] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:f7fe7a284972bacd settled=true {"provingJobId":"2:BASE_PARITY:f7fe7a284972bacd","status":"fulfilled"} 10:07:10 [10:07:10.895] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.895] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:316a5e7b59bc937b settled=true {"provingJobId":"1:BASE_PARITY:316a5e7b59bc937b","status":"fulfilled"} 10:07:10 [10:07:10.896] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:faf19c5bd8dd52f3 settled=false {"provingJobId":"1:BASE_PARITY:faf19c5bd8dd52f3","status":"pending"} 10:07:10 [10:07:10.897] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.898] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d593babc8308d0e0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d593babc8308d0e0"} 10:07:10 [10:07:10.899] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.899] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e4500debdc00a78d epochNumber=1 {"provingJobId":"1:BASE_PARITY:e4500debdc00a78d"} 10:07:10 [10:07:10.900] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:e4500debdc00a78d: Error: db error: Error: db error 10:07:10 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:e4500debdc00a78d"} 10:07:10 [10:07:10.901] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.901] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2bba9735819186b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2bba9735819186b0"} 10:07:10 [10:07:10.902] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2bba9735819186b0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2bba9735819186b0"} 10:07:10 [10:07:10.902] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2bba9735819186b0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2bba9735819186b0"} 10:07:10 [10:07:10.903] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.903] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9f248017a55a3e57 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9f248017a55a3e57"} 10:07:10 [10:07:10.904] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9f248017a55a3e57 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9f248017a55a3e57"} 10:07:10 [10:07:10.904] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9f248017a55a3e57 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9f248017a55a3e57"} 10:07:10 [10:07:10.904] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:9f248017a55a3e57: Error: db error 10:07:10 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:9f248017a55a3e57"} 10:07:10 [10:07:10.905] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.906] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11d6383d83ab801f epochNumber=1 {"provingJobId":"1:BASE_PARITY:11d6383d83ab801f"} 10:07:10 [10:07:10.906] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:11d6383d83ab801f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:11d6383d83ab801f"} 10:07:10 [10:07:10.906] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:11d6383d83ab801f type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:11d6383d83ab801f"} 10:07:10 [10:07:10.907] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.908] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:787bdd27a5ad2e4e epochNumber=1 {"provingJobId":"1:BASE_PARITY:787bdd27a5ad2e4e"} 10:07:10 [10:07:10.908] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:787bdd27a5ad2e4e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:787bdd27a5ad2e4e"} 10:07:10 [10:07:10.908] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:787bdd27a5ad2e4e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:787bdd27a5ad2e4e"} 10:07:10 [10:07:10.909] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:787bdd27a5ad2e4e jobErr=test error: Error: db error 10:07:10 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:787bdd27a5ad2e4e"} 10:07:10 [10:07:10.911] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.912] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f5d8d8ec1f9ef201 not found {"provingJobId":"1:BASE_PARITY:f5d8d8ec1f9ef201"} 10:07:10 [10:07:10.913] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.913] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b2cfe4823c897610 err=test error {"provingJoId":"1:BASE_PARITY:b2cfe4823c897610"} 10:07:10 [10:07:10.914] INFO: prover-client:proving-broker Proving Broker started 10:07:10 [10:07:10.925] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e5696be81e34fe1b epochNumber=1 {"provingJobId":"1:BASE_PARITY:e5696be81e34fe1b"} 10:07:10 [10:07:10.926] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e5696be81e34fe1b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e5696be81e34fe1b"} 10:07:10 [10:07:10.926] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e5696be81e34fe1b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e5696be81e34fe1b"} 10:07:10 [10:07:10.936] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:ef5a9f824803fc58 epochNumber=2 {"provingJobId":"2:BASE_PARITY:ef5a9f824803fc58"} 10:07:10 [10:07:10.936] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:ef5a9f824803fc58 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:ef5a9f824803fc58"} 10:07:10 [10:07:10.936] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:ef5a9f824803fc58 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:ef5a9f824803fc58"} 10:07:10 [10:07:10.998] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:521643fcd020c35a epochNumber=3 {"provingJobId":"3:BASE_PARITY:521643fcd020c35a"} 10:07:11 [10:07:11.015] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:11 [10:07:11.059] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:105860fc62c80f3b epochNumber=4 {"provingJobId":"4:BASE_PARITY:105860fc62c80f3b"} 10:07:11 [10:07:11.066] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:11 [10:07:11.119] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:fc90b8be8f09ad09 epochNumber=5 {"provingJobId":"5:BASE_PARITY:fc90b8be8f09ad09"} 10:07:11 [10:07:11.142] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:11 [10:07:11.172] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.173] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:781b3c1e67fce0f3 epochNumber=42 {"provingJobId":"1:BASE_PARITY:781b3c1e67fce0f3"} 10:07:11 [10:07:11.174] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testCMK0QT/42 with map size 1073741824 10:07:11 [10:07:11.176] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.196] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:aff54b0682d6575a epochNumber=1 {"provingJobId":"1:BASE_PARITY:aff54b0682d6575a"} 10:07:11 [10:07:11.199] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e702f789ca09e4ac epochNumber=1 {"provingJobId":"1:BASE_PARITY:e702f789ca09e4ac"} 10:07:11 [10:07:11.200] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJUorcZ/1 with map size 1073741824 10:07:11 [10:07:11.202] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.220] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4630a359db21a545 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4630a359db21a545"} 10:07:11 [10:07:11.227] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.228] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8f077ff95db0c720 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8f077ff95db0c720"} 10:07:11 [10:07:11.229] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnaUwNf/1 with map size 1073741824 10:07:11 [10:07:11.230] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.245] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:8f077ff95db0c720 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:8f077ff95db0c720"} 10:07:11 [10:07:11.248] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.248] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b4f0054f819c610 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9b4f0054f819c610"} 10:07:11 [10:07:11.249] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzhoFkf/1 with map size 1073741824 10:07:11 [10:07:11.250] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.264] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9b4f0054f819c610 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:9b4f0054f819c610"} 10:07:11 [10:07:11.264] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9b4f0054f819c610 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9b4f0054f819c610"} 10:07:11 [10:07:11.269] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9b4f0054f819c610 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:9b4f0054f819c610"} 10:07:11 [10:07:11.271] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.272] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33895bbb0bea94bf epochNumber=1 {"provingJobId":"1:BASE_PARITY:33895bbb0bea94bf"} 10:07:11 [10:07:11.272] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwNMV00/1 with map size 1073741824 10:07:11 [10:07:11.274] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.291] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.293] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.294] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a0d8fae1814dbf8e epochNumber=1 {"provingJobId":"1:BASE_PARITY:a0d8fae1814dbf8e"} 10:07:11 [10:07:11.294] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1D5mXD/1 with map size 1073741824 10:07:11 [10:07:11.295] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.309] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:a0d8fae1814dbf8e {"provingJobId":"1:BASE_PARITY:a0d8fae1814dbf8e"} 10:07:11 [10:07:11.309] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a0d8fae1814dbf8e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a0d8fae1814dbf8e"} 10:07:11 [10:07:11.310] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a0d8fae1814dbf8e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:a0d8fae1814dbf8e"} 10:07:11 [10:07:11.317] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.317] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:051deced6852965a epochNumber=1 {"provingJobId":"1:BASE_PARITY:051deced6852965a"} 10:07:11 [10:07:11.318] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtPulJI/1 with map size 1073741824 10:07:11 [10:07:11.319] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.333] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:051deced6852965a {"provingJobId":"1:BASE_PARITY:051deced6852965a"} 10:07:11 [10:07:11.334] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:051deced6852965a type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:051deced6852965a"} 10:07:11 [10:07:11.342] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.343] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac3836d1b385a15c epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac3836d1b385a15c"} 10:07:11 [10:07:11.344] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgRuMmZ/1 with map size 1073741824 10:07:11 [10:07:11.345] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.364] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ac3836d1b385a15c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ac3836d1b385a15c"} 10:07:11 [10:07:11.365] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ac3836d1b385a15c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ac3836d1b385a15c"} 10:07:11 [10:07:11.372] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.372] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34e0ffd80f0f9b35 epochNumber=1 {"provingJobId":"1:BASE_PARITY:34e0ffd80f0f9b35"} 10:07:11 [10:07:11.373] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI3Dxu1/1 with map size 1073741824 10:07:11 [10:07:11.374] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.389] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34e0ffd80f0f9b35 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:34e0ffd80f0f9b35"} 10:07:11 [10:07:11.390] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:34e0ffd80f0f9b35 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:34e0ffd80f0f9b35"} 10:07:11 [10:07:11.396] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.397] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:639412cec28cd7ce epochNumber=0 {"provingJobId":"1:BASE_PARITY:639412cec28cd7ce"} 10:07:11 [10:07:11.397] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testygddZk/0 with map size 1073741824 10:07:11 [10:07:11.399] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.412] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:639412cec28cd7ce epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:639412cec28cd7ce"} 10:07:11 [10:07:11.413] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3ef280f9005f90e9 epochNumber=0 {"provingJobId":"1:BASE_PARITY:3ef280f9005f90e9"} 10:07:11 [10:07:11.413] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:639412cec28cd7ce epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:639412cec28cd7ce"} 10:07:11 [10:07:11.414] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c50d2432c23b3ed2 epochNumber=0 {"provingJobId":"1:BASE_PARITY:c50d2432c23b3ed2"} 10:07:11 [10:07:11.425] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.426] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.427] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:667645d824549ae9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:667645d824549ae9"} 10:07:11 [10:07:11.427] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testoMpOkO/2 with map size 1073741824 10:07:11 [10:07:11.428] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.442] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8e66061042f7b13e epochNumber=1 {"provingJobId":"1:BASE_PARITY:8e66061042f7b13e"} 10:07:11 [10:07:11.442] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoMpOkO/1 with map size 1073741824 10:07:11 [10:07:11.443] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.461] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.462] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0f48801253bb31d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0f48801253bb31d3"} 10:07:11 [10:07:11.462] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWQAPrM/1 with map size 1073741824 10:07:11 [10:07:11.464] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.480] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.481] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:76339e0c85d9227f epochNumber=1 {"provingJobId":"1:BASE_PARITY:76339e0c85d9227f"} 10:07:11 [10:07:11.482] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdzDOdc/1 with map size 1073741824 10:07:11 [10:07:11.483] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.499] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64efe95cd977703d epochNumber=1 {"provingJobId":"1:BASE_PARITY:64efe95cd977703d"} 10:07:11 [10:07:11.505] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1cd4ef58d7e3eb5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a1cd4ef58d7e3eb5"} 10:07:11 [10:07:11.505] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testdzDOdc/2 with map size 1073741824 10:07:11 [10:07:11.506] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.524] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab695a2a8ff04f71 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ab695a2a8ff04f71"} 10:07:11 [10:07:11.532] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.532] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:267df6f0b37e0a87 epochNumber=1 {"provingJobId":"1:BASE_PARITY:267df6f0b37e0a87"} 10:07:11 [10:07:11.533] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7RBR29/1 with map size 1073741824 10:07:11 [10:07:11.534] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.550] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a3a4da5917db63b epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a3a4da5917db63b"} 10:07:11 [10:07:11.556] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:78dc560713513fe3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:78dc560713513fe3"} 10:07:11 [10:07:11.556] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7RBR29/2 with map size 1073741824 10:07:11 [10:07:11.557] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.572] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ca3c857328b7197 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8ca3c857328b7197"} 10:07:11 [10:07:11.579] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.579] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d98fc622c4d634b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d98fc622c4d634b6"} 10:07:11 [10:07:11.580] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test39sB57/1 with map size 1073741824 10:07:11 [10:07:11.582] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.595] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a1354b3ed5ba555 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a1354b3ed5ba555"} 10:07:11 [10:07:11.600] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:31a8a9daba7120a1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:31a8a9daba7120a1"} 10:07:11 [10:07:11.601] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test39sB57/2 with map size 1073741824 10:07:11 [10:07:11.602] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.615] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3ca0595971226f6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e3ca0595971226f6"} 10:07:11 [10:07:11.622] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.623] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d9ddde75ee9b1c4f epochNumber=1 {"provingJobId":"1:BASE_PARITY:d9ddde75ee9b1c4f"} 10:07:11 [10:07:11.623] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDPidXF/1 with map size 1073741824 10:07:11 [10:07:11.624] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.639] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:479c782b4c74c495 epochNumber=1 {"provingJobId":"1:BASE_PARITY:479c782b4c74c495"} 10:07:11 [10:07:11.644] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d9ddde75ee9b1c4f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d9ddde75ee9b1c4f"} 10:07:11 [10:07:11.651] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.652] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0f0194795bd61b3c epochNumber=1 {"provingJobId":"1:BASE_PARITY:0f0194795bd61b3c"} 10:07:11 [10:07:11.652] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLUKTWu/1 with map size 1073741824 10:07:11 [10:07:11.654] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8717dab0cf893b53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8717dab0cf893b53"} 10:07:11 [10:07:11.672] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0f0194795bd61b3c type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:0f0194795bd61b3c"} 10:07:11 [10:07:11.679] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.680] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8416529e159acd78 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8416529e159acd78"} 10:07:11 [10:07:11.680] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvBYyEU/1 with map size 1073741824 10:07:11 [10:07:11.683] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.696] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f40397c9ea2d9d7a epochNumber=1 {"provingJobId":"1:BASE_PARITY:f40397c9ea2d9d7a"} 10:07:11 [10:07:11.701] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8416529e159acd78 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:8416529e159acd78"} 10:07:11 [10:07:11.704] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.704] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc0a236949ad0887 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc0a236949ad0887"} 10:07:11 [10:07:11.705] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmWZTeL/1 with map size 1073741824 10:07:11 [10:07:11.706] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.719] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cc0a236949ad0887 {"provingJobId":"1:BASE_PARITY:cc0a236949ad0887"} 10:07:11 [10:07:11.719] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cc0a236949ad0887 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:cc0a236949ad0887"} 10:07:11 [10:07:11.724] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ecd80116e4a18f0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8ecd80116e4a18f0"} 10:07:11 [10:07:11.729] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cc0a236949ad0887 has already been completed {"provingJobId":"1:BASE_PARITY:cc0a236949ad0887"} 10:07:11 [10:07:11.732] INFO: prover-client:proving-broker Proving Broker started 10:07:11 [10:07:11.733] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59fbe56bbc097422 epochNumber=1 {"provingJobId":"1:BASE_PARITY:59fbe56bbc097422"} 10:07:11 [10:07:11.734] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD8P7qP/1 with map size 1073741824 10:07:11 [10:07:11.735] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:11 [10:07:11.751] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ffcad3f50d10c58d epochNumber=2 {"provingJobId":"1:BASE_PARITY:ffcad3f50d10c58d"} 10:07:11 [10:07:11.752] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testD8P7qP/2 with map size 1073741824 10:07:11 [10:07:11.754] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:12 [10:07:12.821] INFO: prover-client:proving-broker Proving Broker started 10:07:12 [10:07:12.825] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:59fbe56bbc097422 settled=false {"provingJobId":"1:BASE_PARITY:59fbe56bbc097422","status":"pending"} 10:07:12 [10:07:12.826] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ffcad3f50d10c58d settled=false {"provingJobId":"1:BASE_PARITY:ffcad3f50d10c58d","status":"pending"} 10:07:12 [10:07:12.827] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:59fbe56bbc097422 type=BASE_PARITY startedAt=1750759631770 older agent has taken job {"provingJobId":"1:BASE_PARITY:59fbe56bbc097422"} 10:07:12 [10:07:12.828] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:59fbe56bbc097422 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:59fbe56bbc097422"} 10:07:12 [10:07:12.830] INFO: prover-client:proving-broker Proving Broker started 10:07:12 [10:07:12.831] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a67f5d884378c16 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a67f5d884378c16"} 10:07:12 [10:07:12.831] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkE3GRM/1 with map size 1073741824 10:07:12 [10:07:12.833] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:12 [10:07:12.847] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:86a67383ce64ed89 epochNumber=2 {"provingJobId":"1:BASE_PARITY:86a67383ce64ed89"} 10:07:12 [10:07:12.848] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkE3GRM/2 with map size 1073741824 10:07:12 [10:07:12.849] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:13 [10:07:13.867] INFO: prover-client:proving-broker Proving Broker started 10:07:13 [10:07:13.868] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1a67f5d884378c16 settled=false {"provingJobId":"1:BASE_PARITY:1a67f5d884378c16","status":"pending"} 10:07:13 [10:07:13.870] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:86a67383ce64ed89 settled=false {"provingJobId":"1:BASE_PARITY:86a67383ce64ed89","status":"pending"} 10:07:13 [10:07:13.870] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1a67f5d884378c16 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:1a67f5d884378c16"} 10:07:13 [10:07:13.875] INFO: prover-client:proving-broker Proving Broker started 10:07:13 [10:07:13.875] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e90f30a387cf3f66 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e90f30a387cf3f66"} 10:07:13 [10:07:13.877] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCAutvr/1 with map size 1073741824 10:07:13 [10:07:13.879] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:13 [10:07:13.897] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:22a06b5d585eaad2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:22a06b5d585eaad2"} 10:07:13 [10:07:13.898] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCAutvr/2 with map size 1073741824 10:07:13 [10:07:13.900] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:14 [10:07:14.917] INFO: prover-client:proving-broker Proving Broker started 10:07:14 [10:07:14.918] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e90f30a387cf3f66 settled=false {"provingJobId":"1:BASE_PARITY:e90f30a387cf3f66","status":"pending"} 10:07:14 [10:07:14.920] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:22a06b5d585eaad2 settled=false {"provingJobId":"1:BASE_PARITY:22a06b5d585eaad2","status":"pending"} 10:07:14 [10:07:14.921] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e90f30a387cf3f66 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e90f30a387cf3f66"} 10:07:14 [10:07:14.921] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e90f30a387cf3f66 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e90f30a387cf3f66"} 10:07:14 [10:07:14.931] INFO: prover-client:proving-broker Proving Broker started 10:07:14 [10:07:14.932] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:de1a0dde5573f136 epochNumber=1 {"provingJobId":"1:BASE_PARITY:de1a0dde5573f136"} 10:07:14 [10:07:14.933] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqMqoMl/1 with map size 1073741824 10:07:14 [10:07:14.935] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:14 [10:07:14.951] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8540964179f067d5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:8540964179f067d5"} 10:07:14 [10:07:14.952] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqMqoMl/2 with map size 1073741824 10:07:14 [10:07:14.953] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:14 [10:07:14.968] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:de1a0dde5573f136 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:de1a0dde5573f136"} 10:07:14 [10:07:14.973] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8540964179f067d5 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8540964179f067d5"} 10:07:14 [10:07:14.981] INFO: prover-client:proving-broker Proving Broker started 10:07:14 [10:07:14.982] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8abd3cf806c10450 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8abd3cf806c10450"} 10:07:14 [10:07:14.982] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhYClPe/1 with map size 1073741824 10:07:14 [10:07:14.984] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:14 [10:07:14.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f7c2832992f558d epochNumber=2 {"provingJobId":"1:BASE_PARITY:4f7c2832992f558d"} 10:07:14 [10:07:14.998] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhYClPe/2 with map size 1073741824 10:07:14 [10:07:15.000] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.013] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8abd3cf806c10450 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8abd3cf806c10450"} 10:07:15 [10:07:15.014] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8abd3cf806c10450 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:8abd3cf806c10450"} 10:07:15 [10:07:15.019] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4f7c2832992f558d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4f7c2832992f558d"} 10:07:15 [10:07:15.027] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.028] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b0775d20a6850d19 err=test error {"provingJoId":"1:BASE_PARITY:b0775d20a6850d19"} 10:07:15 [10:07:15.030] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.031] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8dd063ab4d5ea978 not found {"provingJobId":"1:BASE_PARITY:8dd063ab4d5ea978"} 10:07:15 [10:07:15.034] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.034] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:040d387ab320219a epochNumber=1 {"provingJobId":"1:BASE_PARITY:040d387ab320219a"} 10:07:15 [10:07:15.035] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIwbkuK/1 with map size 1073741824 10:07:15 [10:07:15.036] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.053] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.054] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:832d26dcd82d7c3e epochNumber=1 {"provingJobId":"1:BASE_PARITY:832d26dcd82d7c3e"} 10:07:15 [10:07:15.054] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7NdbPQ/1 with map size 1073741824 10:07:15 [10:07:15.056] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.180] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:832d26dcd82d7c3e timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:832d26dcd82d7c3e"} 10:07:15 [10:07:15.185] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9869126e492a5302 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9869126e492a5302"} 10:07:15 [10:07:15.187] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrSEe4Y/1 with map size 1073741824 10:07:15 [10:07:15.188] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.312] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9869126e492a5302 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:9869126e492a5302"} 10:07:15 [10:07:15.316] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4dbddd97e7b0ef52 epochNumber=10 {"provingJobId":"1:BASE_PARITY:4dbddd97e7b0ef52"} 10:07:15 [10:07:15.317] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testrSEe4Y/10 with map size 1073741824 10:07:15 [10:07:15.318] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.338] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:15 [10:07:15.339] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:07:15 [10:07:15.462] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.462] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:401c630f4b942386 epochNumber=1 {"provingJobId":"1:BASE_PARITY:401c630f4b942386"} 10:07:15 [10:07:15.464] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfdCKOk/1 with map size 1073741824 10:07:15 [10:07:15.465] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.638] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:401c630f4b942386 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:401c630f4b942386"} 10:07:15 [10:07:15.686] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.687] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ec5c7da88d7df36 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ec5c7da88d7df36"} 10:07:15 [10:07:15.687] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIRh9Zu/1 with map size 1073741824 10:07:15 [10:07:15.689] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.705] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2ec5c7da88d7df36 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:2ec5c7da88d7df36"} 10:07:15 [10:07:15.707] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.708] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:edad9919a770c878 epochNumber=1 {"provingJobId":"1:BASE_PARITY:edad9919a770c878"} 10:07:15 [10:07:15.709] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testekoqBo/1 with map size 1073741824 10:07:15 [10:07:15.710] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.725] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:edad9919a770c878 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:edad9919a770c878"} 10:07:15 [10:07:15.726] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:edad9919a770c878 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:edad9919a770c878"} 10:07:15 [10:07:15.733] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.734] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f77f92b5c3c2dc38 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f77f92b5c3c2dc38"} 10:07:15 [10:07:15.734] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuk6UcQ/1 with map size 1073741824 10:07:15 [10:07:15.735] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.750] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f77f92b5c3c2dc38 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f77f92b5c3c2dc38"} 10:07:15 [10:07:15.757] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.758] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2c9fbe0a3cb9c7df epochNumber=1 {"provingJobId":"1:BASE_PARITY:2c9fbe0a3cb9c7df"} 10:07:15 [10:07:15.760] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testR9WZSN/1 with map size 1073741824 10:07:15 [10:07:15.761] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.777] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2c9fbe0a3cb9c7df type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:2c9fbe0a3cb9c7df"} 10:07:15 [10:07:15.778] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:82b4790df98efa49 epochNumber=3 {"provingJobId":"1:BASE_PARITY:82b4790df98efa49"} 10:07:15 [10:07:15.778] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testR9WZSN/3 with map size 1073741824 10:07:15 [10:07:15.780] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.807] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:15 [10:07:15.808] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:07:15 [10:07:15.819] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2c9fbe0a3cb9c7df err=test error {"provingJoId":"1:BASE_PARITY:2c9fbe0a3cb9c7df"} 10:07:15 [10:07:15.823] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZNjflW/1 with map size 1073741824 10:07:15 [10:07:15.824] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.839] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testZNjflW/2 with map size 1073741824 10:07:15 [10:07:15.841] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.854] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.855] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0ef20ebbf7008acc settled=false {"provingJobId":"1:BASE_PARITY:0ef20ebbf7008acc","status":"pending"} 10:07:15 [10:07:15.856] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4804645169d137a3 settled=false {"provingJobId":"1:BASE_PARITY:4804645169d137a3","status":"pending"} 10:07:15 [10:07:15.862] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testazgJTN/1 with map size 1073741824 10:07:15 [10:07:15.863] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.877] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testazgJTN/2 with map size 1073741824 10:07:15 [10:07:15.878] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.901] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.903] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1cbe7491b35ac07f settled=true {"provingJobId":"1:BASE_PARITY:1cbe7491b35ac07f","status":"fulfilled"} 10:07:15 [10:07:15.904] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:185a8340e584e9c7 settled=true {"provingJobId":"2:BASE_PARITY:185a8340e584e9c7","status":"fulfilled"} 10:07:15 [10:07:15.908] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testum6KLc/1 with map size 1073741824 10:07:15 [10:07:15.910] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.928] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testum6KLc/2 with map size 1073741824 10:07:15 [10:07:15.929] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.942] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.944] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0e413c4055935e31 settled=true {"provingJobId":"1:BASE_PARITY:0e413c4055935e31","status":"fulfilled"} 10:07:15 [10:07:15.946] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0fcf3a28848e2e01 settled=false {"provingJobId":"1:BASE_PARITY:0fcf3a28848e2e01","status":"pending"} 10:07:15 [10:07:15.950] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.951] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c1c3d878ea0571f1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c1c3d878ea0571f1"} 10:07:15 [10:07:15.951] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmAUVW5/1 with map size 1073741824 10:07:15 [10:07:15.952] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.969] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.970] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:df75049f8f92a489 epochNumber=1 {"provingJobId":"1:BASE_PARITY:df75049f8f92a489"} 10:07:15 [10:07:15.971] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:df75049f8f92a489: Error: db error: Error: db error 10:07:15 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:df75049f8f92a489"} 10:07:15 [10:07:15.974] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.974] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:75ef719155fe7917 epochNumber=1 {"provingJobId":"1:BASE_PARITY:75ef719155fe7917"} 10:07:15 [10:07:15.975] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbb7Osh/1 with map size 1073741824 10:07:15 [10:07:15.976] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:15 [10:07:15.989] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:75ef719155fe7917 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:75ef719155fe7917"} 10:07:15 [10:07:15.990] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:75ef719155fe7917 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:75ef719155fe7917"} 10:07:15 [10:07:15.998] INFO: prover-client:proving-broker Proving Broker started 10:07:15 [10:07:15.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:918d59da32f22ddc epochNumber=1 {"provingJobId":"1:BASE_PARITY:918d59da32f22ddc"} 10:07:15 [10:07:15.999] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0PKuxH/1 with map size 1073741824 10:07:16 [10:07:16.000] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.014] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:918d59da32f22ddc type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:918d59da32f22ddc"} 10:07:16 [10:07:16.015] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:918d59da32f22ddc type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:918d59da32f22ddc"} 10:07:16 [10:07:16.016] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:918d59da32f22ddc: Error: db error 10:07:16 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:918d59da32f22ddc"} 10:07:16 [10:07:16.019] INFO: prover-client:proving-broker Proving Broker started 10:07:16 [10:07:16.020] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f82b1bd70ea080f9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f82b1bd70ea080f9"} 10:07:16 [10:07:16.020] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjDAk5V/1 with map size 1073741824 10:07:16 [10:07:16.022] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.036] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f82b1bd70ea080f9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f82b1bd70ea080f9"} 10:07:16 [10:07:16.036] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f82b1bd70ea080f9 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f82b1bd70ea080f9"} 10:07:16 [10:07:16.045] INFO: prover-client:proving-broker Proving Broker started 10:07:16 [10:07:16.045] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:df43500b9d6a5e9a epochNumber=1 {"provingJobId":"1:BASE_PARITY:df43500b9d6a5e9a"} 10:07:16 [10:07:16.047] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJtIJDp/1 with map size 1073741824 10:07:16 [10:07:16.048] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.062] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:df43500b9d6a5e9a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:df43500b9d6a5e9a"} 10:07:16 [10:07:16.063] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:df43500b9d6a5e9a type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:df43500b9d6a5e9a"} 10:07:16 [10:07:16.063] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:df43500b9d6a5e9a jobErr=test error: Error: db error 10:07:16 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:df43500b9d6a5e9a"} 10:07:16 [10:07:16.067] INFO: prover-client:proving-broker Proving Broker started 10:07:16 [10:07:16.068] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a08a364cb5b3d355 not found {"provingJobId":"1:BASE_PARITY:a08a364cb5b3d355"} 10:07:16 [10:07:16.071] INFO: prover-client:proving-broker Proving Broker started 10:07:16 [10:07:16.071] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:401e47f9a9ae6c54 err=test error {"provingJoId":"1:BASE_PARITY:401e47f9a9ae6c54"} 10:07:16 [10:07:16.074] INFO: prover-client:proving-broker Proving Broker started 10:07:16 [10:07:16.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0929d0982fcbdce4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0929d0982fcbdce4"} 10:07:16 [10:07:16.086] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1vknmR/1 with map size 1073741824 10:07:16 [10:07:16.087] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.101] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0929d0982fcbdce4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0929d0982fcbdce4"} 10:07:16 [10:07:16.102] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0929d0982fcbdce4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:0929d0982fcbdce4"} 10:07:16 [10:07:16.117] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:a6980ce09222400e epochNumber=2 {"provingJobId":"2:BASE_PARITY:a6980ce09222400e"} 10:07:16 [10:07:16.118] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test1vknmR/2 with map size 1073741824 10:07:16 [10:07:16.120] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.134] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:a6980ce09222400e type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:a6980ce09222400e"} 10:07:16 [10:07:16.135] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:a6980ce09222400e type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:a6980ce09222400e"} 10:07:16 [10:07:16.200] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:225aad226bc65cab epochNumber=3 {"provingJobId":"3:BASE_PARITY:225aad226bc65cab"} 10:07:16 [10:07:16.201] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test1vknmR/3 with map size 1073741824 10:07:16 [10:07:16.203] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.227] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:16 [10:07:16.227] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:07:16 [10:07:16.278] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:0bb77e696959d62d epochNumber=4 {"provingJobId":"4:BASE_PARITY:0bb77e696959d62d"} 10:07:16 [10:07:16.278] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test1vknmR/4 with map size 1073741824 10:07:16 [10:07:16.280] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.304] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:16 [10:07:16.304] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 10:07:16 [10:07:16.355] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:7ad726b184ca0275 epochNumber=5 {"provingJobId":"5:BASE_PARITY:7ad726b184ca0275"} 10:07:16 [10:07:16.356] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test1vknmR/5 with map size 1073741824 10:07:16 [10:07:16.357] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:07:16 [10:07:16.381] INFO: prover-client:proving-broker Cleaned up jobs=1 10:07:16 [10:07:16.381] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 10:07:16 PASS src/proving_broker/proving_broker.test.ts (11.195 s) 10:07:16 ProvingBroker 10:07:16 Producer API 10:07:16 ✓ refuses stale jobs (36 ms) 10:07:16 ✓ enqueues jobs (4 ms) 10:07:16 ✓ ignores duplicate jobs (5 ms) 10:07:16 ✓ reports correct status when enqueuing repeat jobs (4 ms) 10:07:16 ✓ throws an error in case of duplicate job IDs (19 ms) 10:07:16 ✓ returns not-found status for non-existing jobs (1 ms) 10:07:16 ✓ cancels jobs in queue (4 ms) 10:07:16 ✓ cancels jobs in-progress (2 ms) 10:07:16 ✓ returns job result if successful (4 ms) 10:07:16 ✓ returns job error if failed (2 ms) 10:07:16 ✓ correctly returns job status for concurrent writes (2 ms) 10:07:16 ✓ refuses stale jobs (27 ms) 10:07:16 ✓ enqueues jobs (28 ms) 10:07:16 ✓ ignores duplicate jobs (20 ms) 10:07:16 ✓ reports correct status when enqueuing repeat jobs (23 ms) 10:07:16 ✓ throws an error in case of duplicate job IDs (20 ms) 10:07:16 ✓ returns not-found status for non-existing jobs (2 ms) 10:07:16 ✓ cancels jobs in queue (24 ms) 10:07:16 ✓ cancels jobs in-progress (24 ms) 10:07:16 ✓ returns job result if successful (29 ms) 10:07:16 ✓ returns job error if failed (24 ms) 10:07:16 ✓ correctly returns job status for concurrent writes (28 ms) 10:07:16 Consumer API 10:07:16 ✓ returns undefined if no jobs are available (1 ms) 10:07:16 ✓ returns jobs in priority order (2 ms) 10:07:16 ✓ returns undefined if no jobs are available for the given allowList (6 ms) 10:07:16 ✓ returns a job if it is in the allowList (2 ms) 10:07:16 ✓ returns the most important job if it is in the allowList (3 ms) 10:07:16 ✓ returns any job if filter is empty (2 ms) 10:07:16 ✓ returns a new job when reporting job success (2 ms) 10:07:16 ✓ returns a new job when reporting permanent error (2 ms) 10:07:16 ✓ returns a new job when reporting retry-able error (2 ms) 10:07:16 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 10:07:16 ✓ returns a new job if job is already in progress elsewhere (1054 ms) 10:07:16 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 10:07:16 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 10:07:16 ✓ tracks job result if in progress (2 ms) 10:07:16 ✓ tracks job result even if job is in queue (2 ms) 10:07:16 ✓ ignores reported job error if unknown job (1 ms) 10:07:16 ✓ ignores job result if unknown job (1 ms) 10:07:16 ✓ returns undefined if no jobs are available (1 ms) 10:07:16 ✓ returns jobs in priority order (34 ms) 10:07:16 ✓ returns undefined if no jobs are available for the given allowList (19 ms) 10:07:16 ✓ returns a job if it is in the allowList (51 ms) 10:07:16 ✓ returns the most important job if it is in the allowList (47 ms) 10:07:16 ✓ returns any job if filter is empty (43 ms) 10:07:16 ✓ returns a new job when reporting job success (29 ms) 10:07:16 ✓ returns a new job when reporting permanent error (28 ms) 10:07:16 ✓ returns a new job when reporting retry-able error (24 ms) 10:07:16 ✓ returns a new job when reporting progress if current one is cancelled (28 ms) 10:07:16 ✓ returns a new job if job is already in progress elsewhere (1098 ms) 10:07:16 ✓ avoids sending the same job to a new agent after a restart (1043 ms) 10:07:16 ✓ avoids sending a completed job to a new agent after a restart (1057 ms) 10:07:16 ✓ tracks job result if in progress (50 ms) 10:07:16 ✓ tracks job result even if job is in queue (45 ms) 10:07:16 ✓ ignores reported job error if unknown job (2 ms) 10:07:16 ✓ ignores job result if unknown job (2 ms) 10:07:16 Timeouts 10:07:16 ✓ tracks in progress jobs (2 ms) 10:07:16 ✓ re-enqueues jobs that time out (109 ms) 10:07:16 ✓ cancel stale jobs that time out (234 ms) 10:07:16 ✓ keeps the jobs in progress while it is alive (202 ms) 10:07:16 ✓ tracks in progress jobs (20 ms) 10:07:16 ✓ re-enqueues jobs that time out (132 ms) 10:07:16 ✓ cancel stale jobs that time out (276 ms) 10:07:16 ✓ keeps the jobs in progress while it is alive (224 ms) 10:07:16 Retries 10:07:16 ✓ retries jobs (2 ms) 10:07:16 ✓ retries up to a maximum number of times (2 ms) 10:07:16 ✓ passing retry=false does not retry (2 ms) 10:07:16 ✓ does not retry if job is stale (27 ms) 10:07:16 ✓ retries jobs (21 ms) 10:07:16 ✓ retries up to a maximum number of times (26 ms) 10:07:16 ✓ passing retry=false does not retry (23 ms) 10:07:16 ✓ does not retry if job is stale (64 ms) 10:07:16 Database management 10:07:16 ✓ re-enqueues proof requests on start (2 ms) 10:07:16 ✓ restores proof results on start (2 ms) 10:07:16 ✓ only re-enqueues unfinished jobs (1 ms) 10:07:16 ✓ saves job when enqueued (2 ms) 10:07:16 ✓ does not retain job if database fails to save (2 ms) 10:07:16 ✓ saves job result (1 ms) 10:07:16 ✓ saves result even if database fails to save (2 ms) 10:07:16 ✓ saves job error (2 ms) 10:07:16 ✓ saves job error even if database fails to save (4 ms) 10:07:16 ✓ does not save job result if job is unknown (1 ms) 10:07:16 ✓ does not save job error if job is unknown (1 ms) 10:07:16 ✓ cleans up old jobs periodically (256 ms) 10:07:16 ✓ re-enqueues proof requests on start (38 ms) 10:07:16 ✓ restores proof results on start (47 ms) 10:07:16 ✓ only re-enqueues unfinished jobs (42 ms) 10:07:16 ✓ saves job when enqueued (18 ms) 10:07:16 ✓ does not retain job if database fails to save (4 ms) 10:07:16 ✓ saves job result (24 ms) 10:07:16 ✓ saves result even if database fails to save (21 ms) 10:07:16 ✓ saves job error (26 ms) 10:07:16 ✓ saves job error even if database fails to save (22 ms) 10:07:16 ✓ does not save job result if job is unknown (3 ms) 10:07:16 ✓ does not save job error if job is unknown (3 ms) 10:07:16 ✓ cleans up old jobs periodically (351 ms) 10:07:16 10:07:16 Test Suites: 1 passed, 1 total 10:07:16 Tests: 96 passed, 96 total 10:07:16 Snapshots: 0 total 10:07:16 Time: 11.269 s 10:07:16 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 10:07:16 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?