Parent Log:
http://ci.aztec-labs.com/fadb814e2da9969e
Command: 22a91bf956a4cb93 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/47bd93499e6606131b6ececa3536632cf5e84409
Env: REF_NAME=gh-readonly-queue/next/pr-15080-215be54ba7d05741f58f591612a9e5454104b24d CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Wed Jun 18 09:36:34 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15080_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
09:36:39 [09:36:39.230]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.235]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4b861086edb9b3d epochNumber=42
{"provingJobId":"1:BASE_PARITY:a4b861086edb9b3d"}
09:36:39 [09:36:39.238]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:55e729edba815b3f epochNumber=1
{"provingJobId":"1:BASE_PARITY:55e729edba815b3f"}
09:36:39 [09:36:39.275]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.276]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:186d34329e2db515 epochNumber=1
{"provingJobId":"1:BASE_PARITY:186d34329e2db515"}
09:36:39 [09:36:39.278]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36c9ea87414c3342 epochNumber=1
{"provingJobId":"1:BASE_PARITY:36c9ea87414c3342"}
09:36:39 [09:36:39.279]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.280]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e4debb326d93b27 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e4debb326d93b27"}
09:36:39 [09:36:39.280]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0e4debb326d93b27 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0e4debb326d93b27"}
09:36:39 [09:36:39.282]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.282]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:21aae7d756fa768d epochNumber=1
{"provingJobId":"1:BASE_PARITY:21aae7d756fa768d"}
09:36:39 [09:36:39.283]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:21aae7d756fa768d epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:21aae7d756fa768d"}
09:36:39 [09:36:39.284]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:21aae7d756fa768d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:21aae7d756fa768d"}
09:36:39 [09:36:39.284]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:21aae7d756fa768d epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:21aae7d756fa768d"}
09:36:39 [09:36:39.285]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.286]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:04459d31d70a50bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:04459d31d70a50bc"}
09:36:39 [09:36:39.307]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.308]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.308]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3421e7b6917c7f29 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3421e7b6917c7f29"}
09:36:39 [09:36:39.309]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3421e7b6917c7f29
{"provingJobId":"1:BASE_PARITY:3421e7b6917c7f29"}
09:36:39 [09:36:39.309]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3421e7b6917c7f29 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3421e7b6917c7f29"}
09:36:39 [09:36:39.309]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3421e7b6917c7f29 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3421e7b6917c7f29"}
09:36:39 [09:36:39.310]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.310]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91d0153b75b57da3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:91d0153b75b57da3"}
09:36:39 [09:36:39.311]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:91d0153b75b57da3
{"provingJobId":"1:BASE_PARITY:91d0153b75b57da3"}
09:36:39 [09:36:39.311]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:91d0153b75b57da3 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:91d0153b75b57da3"}
09:36:39 [09:36:39.312]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.313]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce9d98dc7358b98f epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce9d98dc7358b98f"}
09:36:39 [09:36:39.313]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ce9d98dc7358b98f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ce9d98dc7358b98f"}
09:36:39 [09:36:39.313]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ce9d98dc7358b98f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ce9d98dc7358b98f"}
09:36:39 [09:36:39.314]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.314]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a4d047a52cc6460 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a4d047a52cc6460"}
09:36:39 [09:36:39.315]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7a4d047a52cc6460 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7a4d047a52cc6460"}
09:36:39 [09:36:39.315]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a4d047a52cc6460 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7a4d047a52cc6460"}
09:36:39 [09:36:39.316]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b2e10772e65c7b0e epochNumber=0
{"provingJobId":"1:BASE_PARITY:b2e10772e65c7b0e"}
09:36:39 [09:36:39.317]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b2e10772e65c7b0e epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b2e10772e65c7b0e"}
09:36:39 [09:36:39.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d5fe56ecdba19cc epochNumber=0
{"provingJobId":"1:BASE_PARITY:3d5fe56ecdba19cc"}
09:36:39 [09:36:39.317]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b2e10772e65c7b0e epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b2e10772e65c7b0e"}
09:36:39 [09:36:39.318]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:29a03d8b39b1113e epochNumber=0
{"provingJobId":"1:BASE_PARITY:29a03d8b39b1113e"}
09:36:39 [09:36:39.319]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.320]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.320]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b5845caf4fe3eb95 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b5845caf4fe3eb95"}
09:36:39 [09:36:39.320]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b13f672d93a981f epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b13f672d93a981f"}
09:36:39 [09:36:39.321]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.321]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68c30dcfce91e78d epochNumber=1
{"provingJobId":"1:BASE_PARITY:68c30dcfce91e78d"}
09:36:39 [09:36:39.330]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.331]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:19121a94342e4114 epochNumber=1
{"provingJobId":"1:BASE_PARITY:19121a94342e4114"}
09:36:39 [09:36:39.331]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0061f44515bc7263 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0061f44515bc7263"}
09:36:39 [09:36:39.331]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b058d90d39d8922 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9b058d90d39d8922"}
09:36:39 [09:36:39.331]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a0126c9ea8e87147 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a0126c9ea8e87147"}
09:36:39 [09:36:39.332]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3df3f018831d8ac epochNumber=1
{"provingJobId":"1:BASE_PARITY:b3df3f018831d8ac"}
09:36:39 [09:36:39.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:19e12feef6330bd6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:19e12feef6330bd6"}
09:36:39 [09:36:39.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6633d9e0d90480b5 epochNumber=2
{"provingJobId":"1:BASE_PARITY:6633d9e0d90480b5"}
09:36:39 [09:36:39.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d37efe9b12831923 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d37efe9b12831923"}
09:36:39 [09:36:39.334]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.334]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b3cc83e76cd84e7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0b3cc83e76cd84e7"}
09:36:39 [09:36:39.334]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a883a3ef57c0d7b epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a883a3ef57c0d7b"}
09:36:39 [09:36:39.335]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9062ef429b7d85bf epochNumber=2
{"provingJobId":"1:BASE_PARITY:9062ef429b7d85bf"}
09:36:39 [09:36:39.335]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b586ac29578467ed epochNumber=1
{"provingJobId":"1:BASE_PARITY:b586ac29578467ed"}
09:36:39 [09:36:39.336]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.336]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a361592bd3aef7e5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a361592bd3aef7e5"}
09:36:39 [09:36:39.336]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d1b9e7b48d8b7f95 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d1b9e7b48d8b7f95"}
09:36:39 [09:36:39.337]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a361592bd3aef7e5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a361592bd3aef7e5"}
09:36:39 [09:36:39.338]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.338]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e064c7bfec0cf630 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e064c7bfec0cf630"}
09:36:39 [09:36:39.338]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3035b68cc539c08a epochNumber=1
{"provingJobId":"1:BASE_PARITY:3035b68cc539c08a"}
09:36:39 [09:36:39.338]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e064c7bfec0cf630 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:e064c7bfec0cf630"}
09:36:39 [09:36:39.339]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a7071a660e4f1c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a7071a660e4f1c3"}
09:36:39 [09:36:39.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30087cc41bd777ce epochNumber=1
{"provingJobId":"1:BASE_PARITY:30087cc41bd777ce"}
09:36:39 [09:36:39.341]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3a7071a660e4f1c3 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:3a7071a660e4f1c3"}
09:36:39 [09:36:39.341]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.342]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:def4251c5863cded epochNumber=1
{"provingJobId":"1:BASE_PARITY:def4251c5863cded"}
09:36:39 [09:36:39.342]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:def4251c5863cded
{"provingJobId":"1:BASE_PARITY:def4251c5863cded"}
09:36:39 [09:36:39.342]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:def4251c5863cded type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:def4251c5863cded"}
09:36:39 [09:36:39.342]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b4ee5deaceb108b3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b4ee5deaceb108b3"}
09:36:39 [09:36:39.343]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:def4251c5863cded has already been completed
{"provingJobId":"1:BASE_PARITY:def4251c5863cded"}
09:36:39 [09:36:39.344]
INFO:
prover-client:proving-broker Proving Broker started
09:36:39 [09:36:39.344]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:89503e65b3d5f8ba epochNumber=1
{"provingJobId":"1:BASE_PARITY:89503e65b3d5f8ba"}
09:36:39 [09:36:39.344]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b5acc2add2219e3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:0b5acc2add2219e3"}
09:36:40 [09:36:40.397]
INFO:
prover-client:proving-broker Proving Broker started
09:36:40 [09:36:40.398]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:89503e65b3d5f8ba settled=false
{"provingJobId":"1:BASE_PARITY:89503e65b3d5f8ba","status":"pending"}
09:36:40 [09:36:40.398]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0b5acc2add2219e3 settled=false
{"provingJobId":"1:BASE_PARITY:0b5acc2add2219e3","status":"pending"}
09:36:40 [09:36:40.399]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:89503e65b3d5f8ba type=BASE_PARITY startedAt=1750239399345 older agent has taken job
{"provingJobId":"1:BASE_PARITY:89503e65b3d5f8ba"}
09:36:40 [09:36:40.400]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:89503e65b3d5f8ba type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:89503e65b3d5f8ba"}
09:36:40 [09:36:40.401]
INFO:
prover-client:proving-broker Proving Broker started
09:36:40 [09:36:40.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9c22085df5f6a87f epochNumber=1
{"provingJobId":"1:BASE_PARITY:9c22085df5f6a87f"}
09:36:40 [09:36:40.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:603e5666e3f83ff0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:603e5666e3f83ff0"}
09:36:41 [09:36:41.404]
INFO:
prover-client:proving-broker Proving Broker started
09:36:41 [09:36:41.404]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9c22085df5f6a87f settled=false
{"provingJobId":"1:BASE_PARITY:9c22085df5f6a87f","status":"pending"}
09:36:41 [09:36:41.405]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:603e5666e3f83ff0 settled=false
{"provingJobId":"1:BASE_PARITY:603e5666e3f83ff0","status":"pending"}
09:36:41 [09:36:41.405]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9c22085df5f6a87f type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:9c22085df5f6a87f"}
09:36:41 [09:36:41.407]
INFO:
prover-client:proving-broker Proving Broker started
09:36:41 [09:36:41.408]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ca3c390b7646b2f epochNumber=1
{"provingJobId":"1:BASE_PARITY:7ca3c390b7646b2f"}
09:36:41 [09:36:41.409]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0dceda8c82d1d63 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b0dceda8c82d1d63"}
09:36:42 [09:36:42.411]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.411]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7ca3c390b7646b2f settled=false
{"provingJobId":"1:BASE_PARITY:7ca3c390b7646b2f","status":"pending"}
09:36:42 [09:36:42.412]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b0dceda8c82d1d63 settled=false
{"provingJobId":"1:BASE_PARITY:b0dceda8c82d1d63","status":"pending"}
09:36:42 [09:36:42.412]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7ca3c390b7646b2f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7ca3c390b7646b2f"}
09:36:42 [09:36:42.413]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7ca3c390b7646b2f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7ca3c390b7646b2f"}
09:36:42 [09:36:42.414]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.415]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d9c5c28369d23d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d9c5c28369d23d6"}
09:36:42 [09:36:42.415]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6bea7b42110e94a6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:6bea7b42110e94a6"}
09:36:42 [09:36:42.416]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2d9c5c28369d23d6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2d9c5c28369d23d6"}
09:36:42 [09:36:42.416]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6bea7b42110e94a6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6bea7b42110e94a6"}
09:36:42 [09:36:42.418]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.418]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4b20baa90ab49b73 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4b20baa90ab49b73"}
09:36:42 [09:36:42.418]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f293951ad27ca363 epochNumber=2
{"provingJobId":"1:BASE_PARITY:f293951ad27ca363"}
09:36:42 [09:36:42.418]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4b20baa90ab49b73 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4b20baa90ab49b73"}
09:36:42 [09:36:42.419]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4b20baa90ab49b73 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4b20baa90ab49b73"}
09:36:42 [09:36:42.419]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f293951ad27ca363 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f293951ad27ca363"}
09:36:42 [09:36:42.420]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.420]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a78608a3534942fc err=test error
{"provingJoId":"1:BASE_PARITY:a78608a3534942fc"}
09:36:42 [09:36:42.421]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.421]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f7b4d39aa27078f9 not found
{"provingJobId":"1:BASE_PARITY:f7b4d39aa27078f9"}
09:36:42 [09:36:42.422]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.423]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2dfcd68b301ac9a2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2dfcd68b301ac9a2"}
09:36:42 [09:36:42.424]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.424]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c08f5a78503028fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:c08f5a78503028fc"}
09:36:42 [09:36:42.525]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c08f5a78503028fc timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:c08f5a78503028fc"}
09:36:42 [09:36:42.533]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.534]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c62e0b6fee174b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c62e0b6fee174b4"}
09:36:42 [09:36:42.636]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5c62e0b6fee174b4 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:5c62e0b6fee174b4"}
09:36:42 [09:36:42.646]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c27bc7efa17179ce epochNumber=10
{"provingJobId":"1:BASE_PARITY:c27bc7efa17179ce"}
09:36:42 [09:36:42.662]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:42 [09:36:42.776]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.777]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:74ce48fe4466ca90 epochNumber=1
{"provingJobId":"1:BASE_PARITY:74ce48fe4466ca90"}
09:36:42 [09:36:42.935]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:74ce48fe4466ca90 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:74ce48fe4466ca90"}
09:36:42 [09:36:42.983]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.985]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38d1437e86d50c30 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38d1437e86d50c30"}
09:36:42 [09:36:42.986]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:38d1437e86d50c30 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:38d1437e86d50c30"}
09:36:42 [09:36:42.988]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.989]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:96b6660897d5daf0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:96b6660897d5daf0"}
09:36:42 [09:36:42.990]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:96b6660897d5daf0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:96b6660897d5daf0"}
09:36:42 [09:36:42.990]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:96b6660897d5daf0 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:96b6660897d5daf0"}
09:36:42 [09:36:42.991]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.992]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8031f11b8ade740 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8031f11b8ade740"}
09:36:42 [09:36:42.993]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d8031f11b8ade740 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d8031f11b8ade740"}
09:36:42 [09:36:42.994]
INFO:
prover-client:proving-broker Proving Broker started
09:36:42 [09:36:42.994]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7998f1b5bd3c6fbf epochNumber=1
{"provingJobId":"1:BASE_PARITY:7998f1b5bd3c6fbf"}
09:36:42 [09:36:42.995]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7998f1b5bd3c6fbf type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:7998f1b5bd3c6fbf"}
09:36:42 [09:36:42.996]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8657d8e448e89b6c epochNumber=3
{"provingJobId":"1:BASE_PARITY:8657d8e448e89b6c"}
09:36:43 [09:36:43.020]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:43 [09:36:43.021]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7998f1b5bd3c6fbf err=test error
{"provingJoId":"1:BASE_PARITY:7998f1b5bd3c6fbf"}
09:36:43 [09:36:43.023]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.023]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:973ce27f8f7d648a settled=false
{"provingJobId":"1:BASE_PARITY:973ce27f8f7d648a","status":"pending"}
09:36:43 [09:36:43.024]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:33fe98a90a3e29c4 settled=false
{"provingJobId":"1:BASE_PARITY:33fe98a90a3e29c4","status":"pending"}
09:36:43 [09:36:43.026]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.026]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cd21d88b9bf30dd6 settled=true
{"provingJobId":"1:BASE_PARITY:cd21d88b9bf30dd6","status":"fulfilled"}
09:36:43 [09:36:43.027]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:9471bfe5e2282354 settled=true
{"provingJobId":"2:BASE_PARITY:9471bfe5e2282354","status":"fulfilled"}
09:36:43 [09:36:43.028]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.028]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fe735bc4994b1f43 settled=true
{"provingJobId":"1:BASE_PARITY:fe735bc4994b1f43","status":"fulfilled"}
09:36:43 [09:36:43.028]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d5f852cae697dcb2 settled=false
{"provingJobId":"1:BASE_PARITY:d5f852cae697dcb2","status":"pending"}
09:36:43 [09:36:43.030]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.031]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7faf575e814aa5bf epochNumber=1
{"provingJobId":"1:BASE_PARITY:7faf575e814aa5bf"}
09:36:43 [09:36:43.033]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.034]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f84e95cf17449141 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f84e95cf17449141"}
09:36:43 [09:36:43.035]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:f84e95cf17449141: Error: db error: Error: db error
09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:f84e95cf17449141"}
09:36:43 [09:36:43.036]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.037]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:567a7d3a176ecfab epochNumber=1
{"provingJobId":"1:BASE_PARITY:567a7d3a176ecfab"}
09:36:43 [09:36:43.037]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:567a7d3a176ecfab type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:567a7d3a176ecfab"}
09:36:43 [09:36:43.037]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:567a7d3a176ecfab type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:567a7d3a176ecfab"}
09:36:43 [09:36:43.043]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.044]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f90a5a1308f2151 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f90a5a1308f2151"}
09:36:43 [09:36:43.044]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4f90a5a1308f2151 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4f90a5a1308f2151"}
09:36:43 [09:36:43.045]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4f90a5a1308f2151 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4f90a5a1308f2151"}
09:36:43 [09:36:43.046]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:4f90a5a1308f2151: Error: db error
09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:4f90a5a1308f2151"}
09:36:43 [09:36:43.048]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.048]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4bf9c50c9199b0e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4bf9c50c9199b0e3"}
09:36:43 [09:36:43.049]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4bf9c50c9199b0e3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4bf9c50c9199b0e3"}
09:36:43 [09:36:43.049]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4bf9c50c9199b0e3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4bf9c50c9199b0e3"}
09:36:43 [09:36:43.051]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.052]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ef9d688b370f23bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:ef9d688b370f23bc"}
09:36:43 [09:36:43.052]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ef9d688b370f23bc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ef9d688b370f23bc"}
09:36:43 [09:36:43.052]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ef9d688b370f23bc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ef9d688b370f23bc"}
09:36:43 [09:36:43.053]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:ef9d688b370f23bc jobErr=test error: Error: db error
09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:ef9d688b370f23bc"}
09:36:43 [09:36:43.061]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.061]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:add09b89179b1ac3 not found
{"provingJobId":"1:BASE_PARITY:add09b89179b1ac3"}
09:36:43 [09:36:43.063]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.063]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a179a252f0373ab9 err=test error
{"provingJoId":"1:BASE_PARITY:a179a252f0373ab9"}
09:36:43 [09:36:43.065]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.075]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c8cdbaf368418ed8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c8cdbaf368418ed8"}
09:36:43 [09:36:43.076]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c8cdbaf368418ed8 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c8cdbaf368418ed8"}
09:36:43 [09:36:43.076]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c8cdbaf368418ed8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c8cdbaf368418ed8"}
09:36:43 [09:36:43.087]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:99c339c60e0bfb38 epochNumber=2
{"provingJobId":"2:BASE_PARITY:99c339c60e0bfb38"}
09:36:43 [09:36:43.087]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:99c339c60e0bfb38 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:99c339c60e0bfb38"}
09:36:43 [09:36:43.089]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:99c339c60e0bfb38 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:99c339c60e0bfb38"}
09:36:43 [09:36:43.151]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:37b9ca834c6be8ee epochNumber=3
{"provingJobId":"3:BASE_PARITY:37b9ca834c6be8ee"}
09:36:43 [09:36:43.170]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:43 [09:36:43.213]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:9a0f6f289581609f epochNumber=4
{"provingJobId":"4:BASE_PARITY:9a0f6f289581609f"}
09:36:43 [09:36:43.222]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:43 [09:36:43.277]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:78e258e12d2fa3ab epochNumber=5
{"provingJobId":"5:BASE_PARITY:78e258e12d2fa3ab"}
09:36:43 [09:36:43.299]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:43 [09:36:43.333]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d5c77f318a1dc61f epochNumber=42
{"provingJobId":"1:BASE_PARITY:d5c77f318a1dc61f"}
09:36:43 [09:36:43.334]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-test9wUcZF/42 with map size 1073741824
09:36:43 [09:36:43.343]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.370]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:296e408fe0625272 epochNumber=1
{"provingJobId":"1:BASE_PARITY:296e408fe0625272"}
09:36:43 [09:36:43.376]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.377]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:96a2b22aade114c1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:96a2b22aade114c1"}
09:36:43 [09:36:43.377]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpsbamw/1 with map size 1073741824
09:36:43 [09:36:43.380]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.398]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:173597e35d9df6e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:173597e35d9df6e6"}
09:36:43 [09:36:43.406]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.407]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5086d118176afcb epochNumber=1
{"provingJobId":"1:BASE_PARITY:c5086d118176afcb"}
09:36:43 [09:36:43.408]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVjw0AM/1 with map size 1073741824
09:36:43 [09:36:43.411]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.429]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c5086d118176afcb epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c5086d118176afcb"}
09:36:43 [09:36:43.434]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.434]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:52d60e830dee8ec6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:52d60e830dee8ec6"}
09:36:43 [09:36:43.435]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrd04lS/1 with map size 1073741824
09:36:43 [09:36:43.442]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.465]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:52d60e830dee8ec6 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:52d60e830dee8ec6"}
09:36:43 [09:36:43.466]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:52d60e830dee8ec6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:52d60e830dee8ec6"}
09:36:43 [09:36:43.472]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:52d60e830dee8ec6 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:52d60e830dee8ec6"}
09:36:43 [09:36:43.477]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.478]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e030e23f83ccaad0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e030e23f83ccaad0"}
09:36:43 [09:36:43.479]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpqjY5s/1 with map size 1073741824
09:36:43 [09:36:43.481]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.502]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.505]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.505]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12da21b1cc235944 epochNumber=1
{"provingJobId":"1:BASE_PARITY:12da21b1cc235944"}
09:36:43 [09:36:43.506]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPdib94/1 with map size 1073741824
09:36:43 [09:36:43.508]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.526]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:12da21b1cc235944
{"provingJobId":"1:BASE_PARITY:12da21b1cc235944"}
09:36:43 [09:36:43.526]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:12da21b1cc235944 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:12da21b1cc235944"}
09:36:43 [09:36:43.526]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:12da21b1cc235944 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:12da21b1cc235944"}
09:36:43 [09:36:43.536]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.537]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae662f23ec426d34 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae662f23ec426d34"}
09:36:43 [09:36:43.541]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFd8K5l/1 with map size 1073741824
09:36:43 [09:36:43.547]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.572]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ae662f23ec426d34
{"provingJobId":"1:BASE_PARITY:ae662f23ec426d34"}
09:36:43 [09:36:43.573]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ae662f23ec426d34 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:ae662f23ec426d34"}
09:36:43 [09:36:43.582]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.583]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a06bfb18796c595f epochNumber=1
{"provingJobId":"1:BASE_PARITY:a06bfb18796c595f"}
09:36:43 [09:36:43.583]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBUiRPs/1 with map size 1073741824
09:36:43 [09:36:43.586]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.604]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a06bfb18796c595f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a06bfb18796c595f"}
09:36:43 [09:36:43.605]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a06bfb18796c595f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a06bfb18796c595f"}
09:36:43 [09:36:43.615]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8c1faed3975397c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8c1faed3975397c2"}
09:36:43 [09:36:43.617]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhuNIn7/1 with map size 1073741824
09:36:43 [09:36:43.620]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.639]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8c1faed3975397c2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8c1faed3975397c2"}
09:36:43 [09:36:43.641]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8c1faed3975397c2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8c1faed3975397c2"}
09:36:43 [09:36:43.664]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.666]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f26f1cd1c67683b4 epochNumber=0
{"provingJobId":"1:BASE_PARITY:f26f1cd1c67683b4"}
09:36:43 [09:36:43.668]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testvDxjLd/0 with map size 1073741824
09:36:43 [09:36:43.672]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.690]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f26f1cd1c67683b4 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f26f1cd1c67683b4"}
09:36:43 [09:36:43.691]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e2e4453627caf918 epochNumber=0
{"provingJobId":"1:BASE_PARITY:e2e4453627caf918"}
09:36:43 [09:36:43.692]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f26f1cd1c67683b4 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f26f1cd1c67683b4"}
09:36:43 [09:36:43.692]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f19c9a7f87c84f1e epochNumber=0
{"provingJobId":"1:BASE_PARITY:f19c9a7f87c84f1e"}
09:36:43 [09:36:43.705]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.708]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.709]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:04b8281e1569047b epochNumber=2
{"provingJobId":"1:BASE_PARITY:04b8281e1569047b"}
09:36:43 [09:36:43.709]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXRyhj6/2 with map size 1073741824
09:36:43 [09:36:43.710]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.726]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dede0806f7c7c6a8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dede0806f7c7c6a8"}
09:36:43 [09:36:43.726]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXRyhj6/1 with map size 1073741824
09:36:43 [09:36:43.728]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.750]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.751]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:07ecc0a8ec352c59 epochNumber=1
{"provingJobId":"1:BASE_PARITY:07ecc0a8ec352c59"}
09:36:43 [09:36:43.751]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWWh0FK/1 with map size 1073741824
09:36:43 [09:36:43.754]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.775]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.776]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f832d2841e96acc8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f832d2841e96acc8"}
09:36:43 [09:36:43.776]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg2sBCn/1 with map size 1073741824
09:36:43 [09:36:43.780]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.795]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3292b11c9093c9d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3292b11c9093c9d7"}
09:36:43 [09:36:43.800]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4432c7c32997d004 epochNumber=2
{"provingJobId":"1:BASE_PARITY:4432c7c32997d004"}
09:36:43 [09:36:43.801]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testg2sBCn/2 with map size 1073741824
09:36:43 [09:36:43.803]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.821]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1aa57baecf0084df epochNumber=1
{"provingJobId":"1:BASE_PARITY:1aa57baecf0084df"}
09:36:43 [09:36:43.831]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.831]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a47e117fe8433e25 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a47e117fe8433e25"}
09:36:43 [09:36:43.832]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testphuIbm/1 with map size 1073741824
09:36:43 [09:36:43.833]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.848]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:750b151ca2ac1ea1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:750b151ca2ac1ea1"}
09:36:43 [09:36:43.854]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2dd2a53c288bcffa epochNumber=2
{"provingJobId":"1:BASE_PARITY:2dd2a53c288bcffa"}
09:36:43 [09:36:43.855]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testphuIbm/2 with map size 1073741824
09:36:43 [09:36:43.856]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.873]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db5734b82bba90d4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:db5734b82bba90d4"}
09:36:43 [09:36:43.880]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.881]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1158420e57b0de9d epochNumber=1
{"provingJobId":"1:BASE_PARITY:1158420e57b0de9d"}
09:36:43 [09:36:43.881]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjvuoEz/1 with map size 1073741824
09:36:43 [09:36:43.883]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.897]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cc937758c4aadeb0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cc937758c4aadeb0"}
09:36:43 [09:36:43.903]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebdb3af9c18dd350 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ebdb3af9c18dd350"}
09:36:43 [09:36:43.904]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testjvuoEz/2 with map size 1073741824
09:36:43 [09:36:43.906]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.921]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:829454c14e7cd853 epochNumber=1
{"provingJobId":"1:BASE_PARITY:829454c14e7cd853"}
09:36:43 [09:36:43.930]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.931]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:64010b1b8c80815f epochNumber=1
{"provingJobId":"1:BASE_PARITY:64010b1b8c80815f"}
09:36:43 [09:36:43.932]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCRbbcr/1 with map size 1073741824
09:36:43 [09:36:43.937]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:43 [09:36:43.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62f7201cc0056b76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:62f7201cc0056b76"}
09:36:43 [09:36:43.968]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:64010b1b8c80815f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:64010b1b8c80815f"}
09:36:43 [09:36:43.977]
INFO:
prover-client:proving-broker Proving Broker started
09:36:43 [09:36:43.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4811148b2b1112c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4811148b2b1112c2"}
09:36:43 [09:36:43.979]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb8vxzg/1 with map size 1073741824
09:36:43 [09:36:43.980]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:44 [09:36:44.000]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d8be7f1a088c703 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d8be7f1a088c703"}
09:36:44 [09:36:44.006]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4811148b2b1112c2 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:4811148b2b1112c2"}
09:36:44 [09:36:44.015]
INFO:
prover-client:proving-broker Proving Broker started
09:36:44 [09:36:44.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cde55afbdf924bcf epochNumber=1
{"provingJobId":"1:BASE_PARITY:cde55afbdf924bcf"}
09:36:44 [09:36:44.018]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKbvfcB/1 with map size 1073741824
09:36:44 [09:36:44.020]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:44 [09:36:44.038]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a54fe166838d841e epochNumber=1
{"provingJobId":"1:BASE_PARITY:a54fe166838d841e"}
09:36:44 [09:36:44.046]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:cde55afbdf924bcf type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:cde55afbdf924bcf"}
09:36:44 [09:36:44.050]
INFO:
prover-client:proving-broker Proving Broker started
09:36:44 [09:36:44.051]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3241aec39fdbe26a epochNumber=1
{"provingJobId":"1:BASE_PARITY:3241aec39fdbe26a"}
09:36:44 [09:36:44.052]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuvVvxV/1 with map size 1073741824
09:36:44 [09:36:44.056]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:44 [09:36:44.075]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3241aec39fdbe26a
{"provingJobId":"1:BASE_PARITY:3241aec39fdbe26a"}
09:36:44 [09:36:44.076]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3241aec39fdbe26a type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3241aec39fdbe26a"}
09:36:44 [09:36:44.082]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:73213c444baa6cf5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:73213c444baa6cf5"}
09:36:44 [09:36:44.089]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3241aec39fdbe26a has already been completed
{"provingJobId":"1:BASE_PARITY:3241aec39fdbe26a"}
09:36:44 [09:36:44.095]
INFO:
prover-client:proving-broker Proving Broker started
09:36:44 [09:36:44.096]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0928cf110b6b3d2e epochNumber=1
{"provingJobId":"1:BASE_PARITY:0928cf110b6b3d2e"}
09:36:44 [09:36:44.098]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testU48CHq/1 with map size 1073741824
09:36:44 [09:36:44.102]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:44 [09:36:44.122]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c3e69f6445360c6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3c3e69f6445360c6"}
09:36:44 [09:36:44.123]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testU48CHq/2 with map size 1073741824
09:36:44 [09:36:44.125]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:45 [09:36:45.197]
INFO:
prover-client:proving-broker Proving Broker started
09:36:45 [09:36:45.212]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0928cf110b6b3d2e settled=false
{"provingJobId":"1:BASE_PARITY:0928cf110b6b3d2e","status":"pending"}
09:36:45 [09:36:45.215]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3c3e69f6445360c6 settled=false
{"provingJobId":"1:BASE_PARITY:3c3e69f6445360c6","status":"pending"}
09:36:45 [09:36:45.217]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0928cf110b6b3d2e type=BASE_PARITY startedAt=1750239404145 older agent has taken job
{"provingJobId":"1:BASE_PARITY:0928cf110b6b3d2e"}
09:36:45 [09:36:45.221]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0928cf110b6b3d2e type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:0928cf110b6b3d2e"}
09:36:45 [09:36:45.230]
INFO:
prover-client:proving-broker Proving Broker started
09:36:45 [09:36:45.231]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:11680910155081cc epochNumber=1
{"provingJobId":"1:BASE_PARITY:11680910155081cc"}
09:36:45 [09:36:45.231]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBqeIc5/1 with map size 1073741824
09:36:45 [09:36:45.233]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:45 [09:36:45.261]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c3398315cdc104d epochNumber=2
{"provingJobId":"1:BASE_PARITY:6c3398315cdc104d"}
09:36:45 [09:36:45.262]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBqeIc5/2 with map size 1073741824
09:36:45 [09:36:45.265]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:46 [09:36:46.282]
INFO:
prover-client:proving-broker Proving Broker started
09:36:46 [09:36:46.288]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:11680910155081cc settled=false
{"provingJobId":"1:BASE_PARITY:11680910155081cc","status":"pending"}
09:36:46 [09:36:46.290]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6c3398315cdc104d settled=false
{"provingJobId":"1:BASE_PARITY:6c3398315cdc104d","status":"pending"}
09:36:46 [09:36:46.291]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:11680910155081cc type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:11680910155081cc"}
09:36:46 [09:36:46.301]
INFO:
prover-client:proving-broker Proving Broker started
09:36:46 [09:36:46.302]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c717a5905f4f0e20 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c717a5905f4f0e20"}
09:36:46 [09:36:46.303]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testipGZ6l/1 with map size 1073741824
09:36:46 [09:36:46.306]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:46 [09:36:46.322]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:74ee36abcdc54d9f epochNumber=2
{"provingJobId":"1:BASE_PARITY:74ee36abcdc54d9f"}
09:36:46 [09:36:46.323]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testipGZ6l/2 with map size 1073741824
09:36:46 [09:36:46.327]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.350]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.355]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c717a5905f4f0e20 settled=false
{"provingJobId":"1:BASE_PARITY:c717a5905f4f0e20","status":"pending"}
09:36:47 [09:36:47.358]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:74ee36abcdc54d9f settled=false
{"provingJobId":"1:BASE_PARITY:74ee36abcdc54d9f","status":"pending"}
09:36:47 [09:36:47.360]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c717a5905f4f0e20 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c717a5905f4f0e20"}
09:36:47 [09:36:47.360]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c717a5905f4f0e20 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c717a5905f4f0e20"}
09:36:47 [09:36:47.377]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.379]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b14b707f42537a90 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b14b707f42537a90"}
09:36:47 [09:36:47.382]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkg53FB/1 with map size 1073741824
09:36:47 [09:36:47.388]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.416]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5c1a6becc4ddd64 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c5c1a6becc4ddd64"}
09:36:47 [09:36:47.419]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkg53FB/2 with map size 1073741824
09:36:47 [09:36:47.423]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.451]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b14b707f42537a90 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b14b707f42537a90"}
09:36:47 [09:36:47.460]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c5c1a6becc4ddd64 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c5c1a6becc4ddd64"}
09:36:47 [09:36:47.470]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.471]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc3221d11cda36df epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc3221d11cda36df"}
09:36:47 [09:36:47.473]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKEJ3lk/1 with map size 1073741824
09:36:47 [09:36:47.477]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.499]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e538602d5df0b9db epochNumber=2
{"provingJobId":"1:BASE_PARITY:e538602d5df0b9db"}
09:36:47 [09:36:47.500]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testKEJ3lk/2 with map size 1073741824
09:36:47 [09:36:47.507]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.541]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fc3221d11cda36df type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fc3221d11cda36df"}
09:36:47 [09:36:47.542]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fc3221d11cda36df type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fc3221d11cda36df"}
09:36:47 [09:36:47.551]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e538602d5df0b9db type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e538602d5df0b9db"}
09:36:47 [09:36:47.567]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.568]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:74cd81d08cc48da9 err=test error
{"provingJoId":"1:BASE_PARITY:74cd81d08cc48da9"}
09:36:47 [09:36:47.573]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.574]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9ec51361889b6419 not found
{"provingJobId":"1:BASE_PARITY:9ec51361889b6419"}
09:36:47 [09:36:47.579]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.580]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8a6e276bef21a7a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8a6e276bef21a7a0"}
09:36:47 [09:36:47.582]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyz8yX4/1 with map size 1073741824
09:36:47 [09:36:47.588]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.616]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.617]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60b9c4a803309d9c epochNumber=1
{"provingJobId":"1:BASE_PARITY:60b9c4a803309d9c"}
09:36:47 [09:36:47.618]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhG7Nyl/1 with map size 1073741824
09:36:47 [09:36:47.622]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.768]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:60b9c4a803309d9c timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:60b9c4a803309d9c"}
09:36:47 [09:36:47.778]
INFO:
prover-client:proving-broker Proving Broker started
09:36:47 [09:36:47.779]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38b79202c9864c83 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38b79202c9864c83"}
09:36:47 [09:36:47.779]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVH2oPm/1 with map size 1073741824
09:36:47 [09:36:47.781]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.906]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:38b79202c9864c83 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:38b79202c9864c83"}
09:36:47 [09:36:47.913]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aa7b947897462a5c epochNumber=10
{"provingJobId":"1:BASE_PARITY:aa7b947897462a5c"}
09:36:47 [09:36:47.914]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testVH2oPm/10 with map size 1073741824
09:36:47 [09:36:47.916]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:47 [09:36:47.956]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:47 [09:36:47.957]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:36:48 [09:36:48.067]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.068]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7de53f260588c61c epochNumber=1
{"provingJobId":"1:BASE_PARITY:7de53f260588c61c"}
09:36:48 [09:36:48.068]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5PuhJl/1 with map size 1073741824
09:36:48 [09:36:48.070]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.246]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7de53f260588c61c timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:7de53f260588c61c"}
09:36:48 [09:36:48.294]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.295]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12d1b80caa78da81 epochNumber=1
{"provingJobId":"1:BASE_PARITY:12d1b80caa78da81"}
09:36:48 [09:36:48.296]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoJQkL3/1 with map size 1073741824
09:36:48 [09:36:48.298]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.319]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:12d1b80caa78da81 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:12d1b80caa78da81"}
09:36:48 [09:36:48.326]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.328]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4358408394c9be83 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4358408394c9be83"}
09:36:48 [09:36:48.330]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2AaGST/1 with map size 1073741824
09:36:48 [09:36:48.335]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.355]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:4358408394c9be83 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:4358408394c9be83"}
09:36:48 [09:36:48.356]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4358408394c9be83 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:4358408394c9be83"}
09:36:48 [09:36:48.364]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.365]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:baa444a7d7cbef1f epochNumber=1
{"provingJobId":"1:BASE_PARITY:baa444a7d7cbef1f"}
09:36:48 [09:36:48.365]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCphzA9/1 with map size 1073741824
09:36:48 [09:36:48.367]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.388]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:baa444a7d7cbef1f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:baa444a7d7cbef1f"}
09:36:48 [09:36:48.402]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.403]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2cc8f2c0ddcee3e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2cc8f2c0ddcee3e3"}
09:36:48 [09:36:48.404]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWtiyh6/1 with map size 1073741824
09:36:48 [09:36:48.408]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.439]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2cc8f2c0ddcee3e3 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:2cc8f2c0ddcee3e3"}
09:36:48 [09:36:48.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f19dc1a7ce73919 epochNumber=3
{"provingJobId":"1:BASE_PARITY:2f19dc1a7ce73919"}
09:36:48 [09:36:48.444]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testWtiyh6/3 with map size 1073741824
09:36:48 [09:36:48.448]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.477]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:48 [09:36:48.478]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:36:48 [09:36:48.496]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2cc8f2c0ddcee3e3 err=test error
{"provingJoId":"1:BASE_PARITY:2cc8f2c0ddcee3e3"}
09:36:48 [09:36:48.504]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNB6iHV/1 with map size 1073741824
09:36:48 [09:36:48.508]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.539]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNB6iHV/2 with map size 1073741824
09:36:48 [09:36:48.542]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.565]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.570]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:09890d9d846b5b11 settled=false
{"provingJobId":"1:BASE_PARITY:09890d9d846b5b11","status":"pending"}
09:36:48 [09:36:48.574]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dd8255c1e7a6f745 settled=false
{"provingJobId":"1:BASE_PARITY:dd8255c1e7a6f745","status":"pending"}
09:36:48 [09:36:48.591]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testszOm0o/1 with map size 1073741824
09:36:48 [09:36:48.595]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.616]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testszOm0o/2 with map size 1073741824
09:36:48 [09:36:48.619]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.663]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.666]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f379accd5225dd8f settled=true
{"provingJobId":"1:BASE_PARITY:f379accd5225dd8f","status":"fulfilled"}
09:36:48 [09:36:48.671]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:44ca3cc1b671dc50 settled=true
{"provingJobId":"2:BASE_PARITY:44ca3cc1b671dc50","status":"fulfilled"}
09:36:48 [09:36:48.678]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9JdKzt/1 with map size 1073741824
09:36:48 [09:36:48.681]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.713]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test9JdKzt/2 with map size 1073741824
09:36:48 [09:36:48.717]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.735]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.742]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5588f506438633bb settled=true
{"provingJobId":"1:BASE_PARITY:5588f506438633bb","status":"fulfilled"}
09:36:48 [09:36:48.744]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b14db4d1d3117bd1 settled=false
{"provingJobId":"1:BASE_PARITY:b14db4d1d3117bd1","status":"pending"}
09:36:48 [09:36:48.759]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.760]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d73d8a6e35c0cde epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d73d8a6e35c0cde"}
09:36:48 [09:36:48.761]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test71ZjYA/1 with map size 1073741824
09:36:48 [09:36:48.767]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.794]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.795]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5762edc9b2a589d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5762edc9b2a589d7"}
09:36:48 [09:36:48.796]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:5762edc9b2a589d7: Error: db error: Error: db error
09:36:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:5762edc9b2a589d7"}
09:36:48 [09:36:48.800]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.801]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a8de5309c499230b epochNumber=1
{"provingJobId":"1:BASE_PARITY:a8de5309c499230b"}
09:36:48 [09:36:48.801]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testigqfZx/1 with map size 1073741824
09:36:48 [09:36:48.804]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.829]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a8de5309c499230b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a8de5309c499230b"}
09:36:48 [09:36:48.830]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a8de5309c499230b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a8de5309c499230b"}
09:36:48 [09:36:48.851]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.852]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49134d1ef6588330 epochNumber=1
{"provingJobId":"1:BASE_PARITY:49134d1ef6588330"}
09:36:48 [09:36:48.855]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJNEMmR/1 with map size 1073741824
09:36:48 [09:36:48.864]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.884]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:49134d1ef6588330 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:49134d1ef6588330"}
09:36:48 [09:36:48.885]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:49134d1ef6588330 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:49134d1ef6588330"}
09:36:48 [09:36:48.886]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:49134d1ef6588330: Error: db error
09:36:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:49134d1ef6588330"}
09:36:48 [09:36:48.892]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.893]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:75941994627b64ae epochNumber=1
{"provingJobId":"1:BASE_PARITY:75941994627b64ae"}
09:36:48 [09:36:48.894]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWyuvc9/1 with map size 1073741824
09:36:48 [09:36:48.896]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.913]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:75941994627b64ae type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:75941994627b64ae"}
09:36:48 [09:36:48.914]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:75941994627b64ae type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:75941994627b64ae"}
09:36:48 [09:36:48.923]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.924]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e623060c58e59022 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e623060c58e59022"}
09:36:48 [09:36:48.924]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdl88rl/1 with map size 1073741824
09:36:48 [09:36:48.926]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:48 [09:36:48.946]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e623060c58e59022 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e623060c58e59022"}
09:36:48 [09:36:48.947]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e623060c58e59022 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e623060c58e59022"}
09:36:48 [09:36:48.952]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:e623060c58e59022 jobErr=test error: Error: db error
09:36:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:e623060c58e59022"}
09:36:48 [09:36:48.965]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.966]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a654c1113815b1ba not found
{"provingJobId":"1:BASE_PARITY:a654c1113815b1ba"}
09:36:48 [09:36:48.972]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.973]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d2f582377977de3d err=test error
{"provingJoId":"1:BASE_PARITY:d2f582377977de3d"}
09:36:48 [09:36:48.976]
INFO:
prover-client:proving-broker Proving Broker started
09:36:48 [09:36:48.987]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a052b1d3c9fe0dc epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a052b1d3c9fe0dc"}
09:36:48 [09:36:48.988]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuaSUYh/1 with map size 1073741824
09:36:48 [09:36:48.989]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:49 [09:36:49.006]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3a052b1d3c9fe0dc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3a052b1d3c9fe0dc"}
09:36:49 [09:36:49.007]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3a052b1d3c9fe0dc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3a052b1d3c9fe0dc"}
09:36:49 [09:36:49.024]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:1a4c2010f2de2489 epochNumber=2
{"provingJobId":"2:BASE_PARITY:1a4c2010f2de2489"}
09:36:49 [09:36:49.025]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testuaSUYh/2 with map size 1073741824
09:36:49 [09:36:49.029]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:49 [09:36:49.050]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:1a4c2010f2de2489 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:1a4c2010f2de2489"}
09:36:49 [09:36:49.051]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:1a4c2010f2de2489 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:1a4c2010f2de2489"}
09:36:49 [09:36:49.118]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:ca16611da83a87fe epochNumber=3
{"provingJobId":"3:BASE_PARITY:ca16611da83a87fe"}
09:36:49 [09:36:49.119]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testuaSUYh/3 with map size 1073741824
09:36:49 [09:36:49.121]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:49 [09:36:49.152]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:49 [09:36:49.154]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:36:49 [09:36:49.202]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:8474086f4666e2a1 epochNumber=4
{"provingJobId":"4:BASE_PARITY:8474086f4666e2a1"}
09:36:49 [09:36:49.203]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testuaSUYh/4 with map size 1073741824
09:36:49 [09:36:49.208]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:49 [09:36:49.234]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:49 [09:36:49.235]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
09:36:49 [09:36:49.288]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:c5880c9a8295db63 epochNumber=5
{"provingJobId":"5:BASE_PARITY:c5880c9a8295db63"}
09:36:49 [09:36:49.289]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testuaSUYh/5 with map size 1073741824
09:36:49 [09:36:49.291]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:36:49 [09:36:49.316]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:36:49 [09:36:49.317]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
09:36:49
PASS src/proving_broker/proving_broker.test.ts (
13.375 s)
09:36:49 ProvingBroker
09:36:49 Producer API
09:36:49
✓ refuses stale jobs (46 ms)
09:36:49
✓ enqueues jobs (3 ms)
09:36:49
✓ ignores duplicate jobs (2 ms)
09:36:49
✓ reports correct status when enqueuing repeat jobs (3 ms)
09:36:49
✓ throws an error in case of duplicate job IDs (21 ms)
09:36:49
✓ returns not-found status for non-existing jobs (1 ms)
09:36:49
✓ cancels jobs in queue (2 ms)
09:36:49
✓ cancels jobs in-progress (1 ms)
09:36:49
✓ returns job result if successful (1 ms)
09:36:49
✓ returns job error if failed (1 ms)
09:36:49
✓ correctly returns job status for concurrent writes (2 ms)
09:36:49
✓ refuses stale jobs (43 ms)
09:36:49
✓ enqueues jobs (31 ms)
09:36:49
✓ ignores duplicate jobs (26 ms)
09:36:49
✓ reports correct status when enqueuing repeat jobs (42 ms)
09:36:49
✓ throws an error in case of duplicate job IDs (25 ms)
09:36:49
✓ returns not-found status for non-existing jobs (2 ms)
09:36:49
✓ cancels jobs in queue (30 ms)
09:36:49
✓ cancels jobs in-progress (46 ms)
09:36:49
✓ returns job result if successful (33 ms)
09:36:49
✓ returns job error if failed (47 ms)
09:36:49
✓ correctly returns job status for concurrent writes (43 ms)
09:36:49 Consumer API
09:36:49
✓ returns undefined if no jobs are available (1 ms)
09:36:49
✓ returns jobs in priority order (2 ms)
09:36:49
✓ returns undefined if no jobs are available for the given allowList (8 ms)
09:36:49
✓ returns a job if it is in the allowList (2 ms)
09:36:49
✓ returns the most important job if it is in the allowList (2 ms)
09:36:49
✓ returns any job if filter is empty (1 ms)
09:36:49
✓ returns a new job when reporting job success (2 ms)
09:36:49
✓ returns a new job when reporting permanent error (2 ms)
09:36:49
✓ returns a new job when reporting retry-able error (2 ms)
09:36:49
✓ returns a new job when reporting progress if current one is cancelled (2 ms)
09:36:49
✓ returns a new job if job is already in progress elsewhere (1057 ms)
09:36:49
✓ avoids sending the same job to a new agent after a restart (1006 ms)
09:36:49
✓ avoids sending a completed job to a new agent after a restart (1006 ms)
09:36:49
✓ tracks job result if in progress (3 ms)
09:36:49
✓ tracks job result even if job is in queue (2 ms)
09:36:49
✓ ignores reported job error if unknown job (1 ms)
09:36:49
✓ ignores job result if unknown job (1 ms)
09:36:49
✓ returns undefined if no jobs are available (2 ms)
09:36:49
✓ returns jobs in priority order (42 ms)
09:36:49
✓ returns undefined if no jobs are available for the given allowList (23 ms)
09:36:49
✓ returns a job if it is in the allowList (56 ms)
09:36:49
✓ returns the most important job if it is in the allowList (50 ms)
09:36:49
✓ returns any job if filter is empty (49 ms)
09:36:49
✓ returns a new job when reporting job success (48 ms)
09:36:49
✓ returns a new job when reporting permanent error (37 ms)
09:36:49
✓ returns a new job when reporting retry-able error (35 ms)
09:36:49
✓ returns a new job when reporting progress if current one is cancelled (44 ms)
09:36:49
✓ returns a new job if job is already in progress elsewhere (1135 ms)
09:36:49
✓ avoids sending the same job to a new agent after a restart (1067 ms)
09:36:49
✓ avoids sending a completed job to a new agent after a restart (1074 ms)
09:36:49
✓ tracks job result if in progress (96 ms)
09:36:49
✓ tracks job result even if job is in queue (94 ms)
09:36:49
✓ ignores reported job error if unknown job (7 ms)
09:36:49
✓ ignores job result if unknown job (6 ms)
09:36:49 Timeouts
09:36:49
✓ tracks in progress jobs (1 ms)
09:36:49
✓ re-enqueues jobs that time out (109 ms)
09:36:49
✓ cancel stale jobs that time out (242 ms)
09:36:49
✓ keeps the jobs in progress while it is alive (206 ms)
09:36:49
✓ tracks in progress jobs (36 ms)
09:36:49
✓ re-enqueues jobs that time out (163 ms)
09:36:49
✓ cancel stale jobs that time out (288 ms)
09:36:49
✓ keeps the jobs in progress while it is alive (227 ms)
09:36:49 Retries
09:36:49
✓ retries jobs (6 ms)
09:36:49
✓ retries up to a maximum number of times (3 ms)
09:36:49
✓ passing retry=false does not retry (2 ms)
09:36:49
✓ does not retry if job is stale (29 ms)
09:36:49
✓ retries jobs (30 ms)
09:36:49
✓ retries up to a maximum number of times (39 ms)
09:36:49
✓ passing retry=false does not retry (36 ms)
09:36:49
✓ does not retry if job is stale (98 ms)
09:36:49 Database management
09:36:49
✓ re-enqueues proof requests on start (3 ms)
09:36:49
✓ restores proof results on start (1 ms)
09:36:49
✓ only re-enqueues unfinished jobs (2 ms)
09:36:49
✓ saves job when enqueued (4 ms)
09:36:49
✓ does not retain job if database fails to save (3 ms)
09:36:49
✓ saves job result (5 ms)
09:36:49
✓ saves result even if database fails to save (4 ms)
09:36:49
✓ saves job error (3 ms)
09:36:49
✓ saves job error even if database fails to save (4 ms)
09:36:49
✓ does not save job result if job is unknown (8 ms)
09:36:49
✓ does not save job error if job is unknown (2 ms)
09:36:49
✓ cleans up old jobs periodically (265 ms)
09:36:49
✓ re-enqueues proof requests on start (85 ms)
09:36:49
✓ restores proof results on start (90 ms)
09:36:49
✓ only re-enqueues unfinished jobs (76 ms)
09:36:49
✓ saves job when enqueued (38 ms)
09:36:49
✓ does not retain job if database fails to save (7 ms)
09:36:49
✓ saves job result (40 ms)
09:36:49
✓ saves result even if database fails to save (50 ms)
09:36:49
✓ saves job error (31 ms)
09:36:49
✓ saves job error even if database fails to save (35 ms)
09:36:49
✓ does not save job result if job is unknown (10 ms)
09:36:49
✓ does not save job error if job is unknown (6 ms)
09:36:49
✓ cleans up old jobs periodically (391 ms)
09:36:49
09:36:49
Test Suites: 1 passed, 1 total
09:36:49
Tests: 96 passed, 96 total
09:36:49
Snapshots: 0 total
09:36:49
Time: 13.475 s
09:36:49
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
09:36:49
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?