Parent Log:
http://ci.aztec-labs.com/dd06888a1deddcbe
Command: 48483af633053ffe yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/3b44f825dcd3249b2c8a1105905fa3394aa7213d
Env: REF_NAME=gh-readonly-queue/next/pr-15041-eeab628a4e44477cb6796c5a461ee0c0bb332d75 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 16 17:46:10 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15041_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
17:46:15 [17:46:15.039]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.045]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57dc68bf3e375424 epochNumber=42
{"provingJobId":"1:BASE_PARITY:57dc68bf3e375424"}
17:46:15 [17:46:15.048]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:44128dcf517be7b7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:44128dcf517be7b7"}
17:46:15 [17:46:15.082]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.083]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f295a92bd19f408e epochNumber=1
{"provingJobId":"1:BASE_PARITY:f295a92bd19f408e"}
17:46:15 [17:46:15.085]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:254cc99761d969c1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:254cc99761d969c1"}
17:46:15 [17:46:15.087]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.088]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:edc54bfa2076d28b epochNumber=1
{"provingJobId":"1:BASE_PARITY:edc54bfa2076d28b"}
17:46:15 [17:46:15.089]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:edc54bfa2076d28b epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:edc54bfa2076d28b"}
17:46:15 [17:46:15.091]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.092]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d454b22ba370ce3c epochNumber=1
{"provingJobId":"1:BASE_PARITY:d454b22ba370ce3c"}
17:46:15 [17:46:15.093]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d454b22ba370ce3c epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d454b22ba370ce3c"}
17:46:15 [17:46:15.093]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d454b22ba370ce3c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d454b22ba370ce3c"}
17:46:15 [17:46:15.094]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d454b22ba370ce3c epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d454b22ba370ce3c"}
17:46:15 [17:46:15.095]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.096]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:18f6d65f438f21dd epochNumber=1
{"provingJobId":"1:BASE_PARITY:18f6d65f438f21dd"}
17:46:15 [17:46:15.127]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.129]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.130]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:27cefc7434b27988 epochNumber=1
{"provingJobId":"1:BASE_PARITY:27cefc7434b27988"}
17:46:15 [17:46:15.130]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:27cefc7434b27988
{"provingJobId":"1:BASE_PARITY:27cefc7434b27988"}
17:46:15 [17:46:15.131]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:27cefc7434b27988 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:27cefc7434b27988"}
17:46:15 [17:46:15.131]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:27cefc7434b27988 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:27cefc7434b27988"}
17:46:15 [17:46:15.133]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.133]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c389b33c67bb844 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0c389b33c67bb844"}
17:46:15 [17:46:15.134]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0c389b33c67bb844
{"provingJobId":"1:BASE_PARITY:0c389b33c67bb844"}
17:46:15 [17:46:15.134]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0c389b33c67bb844 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:0c389b33c67bb844"}
17:46:15 [17:46:15.135]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.136]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:879b588fc7bccd04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:879b588fc7bccd04"}
17:46:15 [17:46:15.137]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:879b588fc7bccd04 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:879b588fc7bccd04"}
17:46:15 [17:46:15.137]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:879b588fc7bccd04 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:879b588fc7bccd04"}
17:46:15 [17:46:15.138]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.139]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:130ccf51c112c9b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:130ccf51c112c9b0"}
17:46:15 [17:46:15.139]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:130ccf51c112c9b0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:130ccf51c112c9b0"}
17:46:15 [17:46:15.139]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:130ccf51c112c9b0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:130ccf51c112c9b0"}
17:46:15 [17:46:15.141]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.141]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e171ab697f55ba7 epochNumber=0
{"provingJobId":"1:BASE_PARITY:6e171ab697f55ba7"}
17:46:15 [17:46:15.141]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6e171ab697f55ba7 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6e171ab697f55ba7"}
17:46:15 [17:46:15.142]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b11631c1b389aea epochNumber=0
{"provingJobId":"1:BASE_PARITY:7b11631c1b389aea"}
17:46:15 [17:46:15.142]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6e171ab697f55ba7 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6e171ab697f55ba7"}
17:46:15 [17:46:15.142]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:259ce06aadc6fb4e epochNumber=0
{"provingJobId":"1:BASE_PARITY:259ce06aadc6fb4e"}
17:46:15 [17:46:15.143]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.144]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.145]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0131d042103b3f18 epochNumber=2
{"provingJobId":"1:BASE_PARITY:0131d042103b3f18"}
17:46:15 [17:46:15.145]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:74271d8984b833b7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:74271d8984b833b7"}
17:46:15 [17:46:15.146]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.147]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8e547abab22b45b epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8e547abab22b45b"}
17:46:15 [17:46:15.148]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.148]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ea7ec2f11144a0c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ea7ec2f11144a0c5"}
17:46:15 [17:46:15.149]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a72ab2a3ff838ef5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a72ab2a3ff838ef5"}
17:46:15 [17:46:15.149]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:59e535f23765dad1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:59e535f23765dad1"}
17:46:15 [17:46:15.149]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:068a67a1ed117088 epochNumber=1
{"provingJobId":"1:BASE_PARITY:068a67a1ed117088"}
17:46:15 [17:46:15.150]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.151]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3167ba30d073135c epochNumber=1
{"provingJobId":"1:BASE_PARITY:3167ba30d073135c"}
17:46:15 [17:46:15.151]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ccf4ba80309a810f epochNumber=1
{"provingJobId":"1:BASE_PARITY:ccf4ba80309a810f"}
17:46:15 [17:46:15.151]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:500e2c4ca17df87d epochNumber=2
{"provingJobId":"1:BASE_PARITY:500e2c4ca17df87d"}
17:46:15 [17:46:15.152]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:efd6bb593040da51 epochNumber=1
{"provingJobId":"1:BASE_PARITY:efd6bb593040da51"}
17:46:15 [17:46:15.153]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.153]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c09eb5dd2b9b610d epochNumber=1
{"provingJobId":"1:BASE_PARITY:c09eb5dd2b9b610d"}
17:46:15 [17:46:15.153]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdfe2a8c4c52eafd epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdfe2a8c4c52eafd"}
17:46:15 [17:46:15.154]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:07a13c3c03a16526 epochNumber=2
{"provingJobId":"1:BASE_PARITY:07a13c3c03a16526"}
17:46:15 [17:46:15.154]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:971e3c62e63f6ffe epochNumber=1
{"provingJobId":"1:BASE_PARITY:971e3c62e63f6ffe"}
17:46:15 [17:46:15.155]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.155]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ddfaa78601337ba epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ddfaa78601337ba"}
17:46:15 [17:46:15.156]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8311a3fcb0c9386 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8311a3fcb0c9386"}
17:46:15 [17:46:15.156]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5ddfaa78601337ba type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5ddfaa78601337ba"}
17:46:15 [17:46:15.157]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.158]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1a88b15a2518e3be epochNumber=1
{"provingJobId":"1:BASE_PARITY:1a88b15a2518e3be"}
17:46:15 [17:46:15.158]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ee1e4baabe44645 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ee1e4baabe44645"}
17:46:15 [17:46:15.158]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1a88b15a2518e3be type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:1a88b15a2518e3be"}
17:46:15 [17:46:15.159]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.160]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:94d5ede429e30ac3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:94d5ede429e30ac3"}
17:46:15 [17:46:15.160]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e7911f5cac51149 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9e7911f5cac51149"}
17:46:15 [17:46:15.160]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:94d5ede429e30ac3 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:94d5ede429e30ac3"}
17:46:15 [17:46:15.161]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.162]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c0ba21297c9a4f5b epochNumber=1
{"provingJobId":"1:BASE_PARITY:c0ba21297c9a4f5b"}
17:46:15 [17:46:15.162]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c0ba21297c9a4f5b
{"provingJobId":"1:BASE_PARITY:c0ba21297c9a4f5b"}
17:46:15 [17:46:15.162]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c0ba21297c9a4f5b type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c0ba21297c9a4f5b"}
17:46:15 [17:46:15.163]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d368fe3e1896900f epochNumber=1
{"provingJobId":"1:BASE_PARITY:d368fe3e1896900f"}
17:46:15 [17:46:15.163]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c0ba21297c9a4f5b has already been completed
{"provingJobId":"1:BASE_PARITY:c0ba21297c9a4f5b"}
17:46:15 [17:46:15.164]
INFO:
prover-client:proving-broker Proving Broker started
17:46:15 [17:46:15.164]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9d4e21c578de508 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f9d4e21c578de508"}
17:46:15 [17:46:15.165]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ed9ebd173338db0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5ed9ebd173338db0"}
17:46:16 [17:46:16.217]
INFO:
prover-client:proving-broker Proving Broker started
17:46:16 [17:46:16.218]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f9d4e21c578de508 settled=false
{"provingJobId":"1:BASE_PARITY:f9d4e21c578de508","status":"pending"}
17:46:16 [17:46:16.219]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5ed9ebd173338db0 settled=false
{"provingJobId":"1:BASE_PARITY:5ed9ebd173338db0","status":"pending"}
17:46:16 [17:46:16.220]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f9d4e21c578de508 type=BASE_PARITY startedAt=1750095975165 older agent has taken job
{"provingJobId":"1:BASE_PARITY:f9d4e21c578de508"}
17:46:16 [17:46:16.220]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f9d4e21c578de508 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:f9d4e21c578de508"}
17:46:16 [17:46:16.222]
INFO:
prover-client:proving-broker Proving Broker started
17:46:16 [17:46:16.222]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2eda766d0b203502 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2eda766d0b203502"}
17:46:16 [17:46:16.223]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:21c36f3e5f135fb2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:21c36f3e5f135fb2"}
17:46:17 [17:46:17.224]
INFO:
prover-client:proving-broker Proving Broker started
17:46:17 [17:46:17.225]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2eda766d0b203502 settled=false
{"provingJobId":"1:BASE_PARITY:2eda766d0b203502","status":"pending"}
17:46:17 [17:46:17.225]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:21c36f3e5f135fb2 settled=false
{"provingJobId":"1:BASE_PARITY:21c36f3e5f135fb2","status":"pending"}
17:46:17 [17:46:17.226]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2eda766d0b203502 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:2eda766d0b203502"}
17:46:17 [17:46:17.227]
INFO:
prover-client:proving-broker Proving Broker started
17:46:17 [17:46:17.228]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bae1725b9d70359b epochNumber=1
{"provingJobId":"1:BASE_PARITY:bae1725b9d70359b"}
17:46:17 [17:46:17.228]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:82156c719cf9c215 epochNumber=2
{"provingJobId":"1:BASE_PARITY:82156c719cf9c215"}
17:46:18 [17:46:18.228]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.229]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bae1725b9d70359b settled=false
{"provingJobId":"1:BASE_PARITY:bae1725b9d70359b","status":"pending"}
17:46:18 [17:46:18.229]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:82156c719cf9c215 settled=false
{"provingJobId":"1:BASE_PARITY:82156c719cf9c215","status":"pending"}
17:46:18 [17:46:18.230]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bae1725b9d70359b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bae1725b9d70359b"}
17:46:18 [17:46:18.230]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bae1725b9d70359b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:bae1725b9d70359b"}
17:46:18 [17:46:18.231]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.233]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9326f2692da617c4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9326f2692da617c4"}
17:46:18 [17:46:18.233]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4a95219937e30ff9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:4a95219937e30ff9"}
17:46:18 [17:46:18.235]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9326f2692da617c4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9326f2692da617c4"}
17:46:18 [17:46:18.236]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4a95219937e30ff9 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4a95219937e30ff9"}
17:46:18 [17:46:18.237]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.238]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:45e03ac570854d86 epochNumber=1
{"provingJobId":"1:BASE_PARITY:45e03ac570854d86"}
17:46:18 [17:46:18.238]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4cbe64dd81fd7a50 epochNumber=2
{"provingJobId":"1:BASE_PARITY:4cbe64dd81fd7a50"}
17:46:18 [17:46:18.239]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:45e03ac570854d86 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:45e03ac570854d86"}
17:46:18 [17:46:18.241]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:45e03ac570854d86 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:45e03ac570854d86"}
17:46:18 [17:46:18.241]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4cbe64dd81fd7a50 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4cbe64dd81fd7a50"}
17:46:18 [17:46:18.242]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.243]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:485d2c22f200cae6 err=test error
{"provingJoId":"1:BASE_PARITY:485d2c22f200cae6"}
17:46:18 [17:46:18.243]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.244]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:56daeadf92b368f9 not found
{"provingJobId":"1:BASE_PARITY:56daeadf92b368f9"}
17:46:18 [17:46:18.245]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.245]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0dfbcba4785c2776 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0dfbcba4785c2776"}
17:46:18 [17:46:18.246]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6390151de3e6f5b epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6390151de3e6f5b"}
17:46:18 [17:46:18.351]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e6390151de3e6f5b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:e6390151de3e6f5b"}
17:46:18 [17:46:18.355]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.355]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c244d36b27f75b1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c244d36b27f75b1"}
17:46:18 [17:46:18.456]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2c244d36b27f75b1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2c244d36b27f75b1"}
17:46:18 [17:46:18.463]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4d7d5d382c526ddd epochNumber=10
{"provingJobId":"1:BASE_PARITY:4d7d5d382c526ddd"}
17:46:18 [17:46:18.483]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:18 [17:46:18.591]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.592]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4b382a16441b08fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:4b382a16441b08fc"}
17:46:18 [17:46:18.768]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4b382a16441b08fc timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:4b382a16441b08fc"}
17:46:18 [17:46:18.798]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.799]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fbbb9b70d3ab5652 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fbbb9b70d3ab5652"}
17:46:18 [17:46:18.800]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fbbb9b70d3ab5652 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:fbbb9b70d3ab5652"}
17:46:18 [17:46:18.801]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.802]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a0587191ef6473bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:a0587191ef6473bc"}
17:46:18 [17:46:18.802]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a0587191ef6473bc type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a0587191ef6473bc"}
17:46:18 [17:46:18.804]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a0587191ef6473bc type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:a0587191ef6473bc"}
17:46:18 [17:46:18.806]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.806]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0fe8b9fa4ebaa25b epochNumber=1
{"provingJobId":"1:BASE_PARITY:0fe8b9fa4ebaa25b"}
17:46:18 [17:46:18.810]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0fe8b9fa4ebaa25b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0fe8b9fa4ebaa25b"}
17:46:18 [17:46:18.812]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.812]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:58fe8b5dd3e6e391 epochNumber=1
{"provingJobId":"1:BASE_PARITY:58fe8b5dd3e6e391"}
17:46:18 [17:46:18.813]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:58fe8b5dd3e6e391 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:58fe8b5dd3e6e391"}
17:46:18 [17:46:18.814]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bce3da5dee08c8d8 epochNumber=3
{"provingJobId":"1:BASE_PARITY:bce3da5dee08c8d8"}
17:46:18 [17:46:18.838]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:18 [17:46:18.843]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:58fe8b5dd3e6e391 err=test error
{"provingJoId":"1:BASE_PARITY:58fe8b5dd3e6e391"}
17:46:18 [17:46:18.845]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.845]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1431e3cc94d9f74d settled=false
{"provingJobId":"1:BASE_PARITY:1431e3cc94d9f74d","status":"pending"}
17:46:18 [17:46:18.846]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d72922112ad0b63c settled=false
{"provingJobId":"1:BASE_PARITY:d72922112ad0b63c","status":"pending"}
17:46:18 [17:46:18.848]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.848]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:53c20ee3976e486b settled=true
{"provingJobId":"1:BASE_PARITY:53c20ee3976e486b","status":"fulfilled"}
17:46:18 [17:46:18.848]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:62081c59305e04ef settled=true
{"provingJobId":"2:BASE_PARITY:62081c59305e04ef","status":"fulfilled"}
17:46:18 [17:46:18.849]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.850]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3f2244f31795baf6 settled=true
{"provingJobId":"1:BASE_PARITY:3f2244f31795baf6","status":"fulfilled"}
17:46:18 [17:46:18.850]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:466d0d6211feff54 settled=false
{"provingJobId":"1:BASE_PARITY:466d0d6211feff54","status":"pending"}
17:46:18 [17:46:18.851]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.852]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a2681e9d9b23167 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a2681e9d9b23167"}
17:46:18 [17:46:18.853]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.854]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b45eccb95cbebad3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b45eccb95cbebad3"}
17:46:18 [17:46:18.854]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:b45eccb95cbebad3: Error: db error: Error: db error
17:46:18 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:b45eccb95cbebad3"}
17:46:18 [17:46:18.855]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.855]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:87e6f8031cd366ef epochNumber=1
{"provingJobId":"1:BASE_PARITY:87e6f8031cd366ef"}
17:46:18 [17:46:18.856]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:87e6f8031cd366ef type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:87e6f8031cd366ef"}
17:46:18 [17:46:18.856]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:87e6f8031cd366ef type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:87e6f8031cd366ef"}
17:46:18 [17:46:18.857]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.857]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8637ef883f2d4615 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8637ef883f2d4615"}
17:46:18 [17:46:18.857]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8637ef883f2d4615 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8637ef883f2d4615"}
17:46:18 [17:46:18.857]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8637ef883f2d4615 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8637ef883f2d4615"}
17:46:18 [17:46:18.858]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:8637ef883f2d4615: Error: db error
17:46:18 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:8637ef883f2d4615"}
17:46:18 [17:46:18.859]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.859]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6a3dfca009a2ffab epochNumber=1
{"provingJobId":"1:BASE_PARITY:6a3dfca009a2ffab"}
17:46:18 [17:46:18.860]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6a3dfca009a2ffab type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6a3dfca009a2ffab"}
17:46:18 [17:46:18.860]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6a3dfca009a2ffab type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6a3dfca009a2ffab"}
17:46:18 [17:46:18.861]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.861]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c55404ddfb4f396 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c55404ddfb4f396"}
17:46:18 [17:46:18.862]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5c55404ddfb4f396 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5c55404ddfb4f396"}
17:46:18 [17:46:18.862]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5c55404ddfb4f396 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:5c55404ddfb4f396"}
17:46:18 [17:46:18.862]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:5c55404ddfb4f396 jobErr=test error: Error: db error
17:46:18 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:5c55404ddfb4f396"}
17:46:18 [17:46:18.863]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.864]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5311d2c257d9fb8d not found
{"provingJobId":"1:BASE_PARITY:5311d2c257d9fb8d"}
17:46:18 [17:46:18.865]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.866]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:91ffd7c8e96ee3be err=test error
{"provingJoId":"1:BASE_PARITY:91ffd7c8e96ee3be"}
17:46:18 [17:46:18.867]
INFO:
prover-client:proving-broker Proving Broker started
17:46:18 [17:46:18.877]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5cb7c30e05c5aa29 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5cb7c30e05c5aa29"}
17:46:18 [17:46:18.878]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5cb7c30e05c5aa29 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5cb7c30e05c5aa29"}
17:46:18 [17:46:18.878]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5cb7c30e05c5aa29 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5cb7c30e05c5aa29"}
17:46:18 [17:46:18.889]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:e28df8b01a703e0c epochNumber=2
{"provingJobId":"2:BASE_PARITY:e28df8b01a703e0c"}
17:46:18 [17:46:18.890]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:e28df8b01a703e0c type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:e28df8b01a703e0c"}
17:46:18 [17:46:18.890]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:e28df8b01a703e0c type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:e28df8b01a703e0c"}
17:46:18 [17:46:18.952]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:51601109db403353 epochNumber=3
{"provingJobId":"3:BASE_PARITY:51601109db403353"}
17:46:18 [17:46:18.969]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:19 [17:46:19.013]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:6979693a9cce2e46 epochNumber=4
{"provingJobId":"4:BASE_PARITY:6979693a9cce2e46"}
17:46:19 [17:46:19.019]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:19 [17:46:19.075]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:65a0f076d13bd27b epochNumber=5
{"provingJobId":"5:BASE_PARITY:65a0f076d13bd27b"}
17:46:19 [17:46:19.094]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:19 [17:46:19.130]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.131]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6a2ce6205c26aeaf epochNumber=42
{"provingJobId":"1:BASE_PARITY:6a2ce6205c26aeaf"}
17:46:19 [17:46:19.132]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testbQgMPv/42 with map size 1073741824
17:46:19 [17:46:19.135]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.164]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:7e2ccb6aa7756ce9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e2ccb6aa7756ce9"}
17:46:19 [17:46:19.169]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.170]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5da0c8cab5ea01bb epochNumber=1
{"provingJobId":"1:BASE_PARITY:5da0c8cab5ea01bb"}
17:46:19 [17:46:19.172]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8F8GLc/1 with map size 1073741824
17:46:19 [17:46:19.173]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.183]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd899ee46d33a08f epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd899ee46d33a08f"}
17:46:19 [17:46:19.189]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.190]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:50208d4a50ce78f2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:50208d4a50ce78f2"}
17:46:19 [17:46:19.190]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuhKGmG/1 with map size 1073741824
17:46:19 [17:46:19.193]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.204]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:50208d4a50ce78f2 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:50208d4a50ce78f2"}
17:46:19 [17:46:19.208]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.209]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:39f50e8a8d6a10fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:39f50e8a8d6a10fd"}
17:46:19 [17:46:19.209]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testB1Xr29/1 with map size 1073741824
17:46:19 [17:46:19.211]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.230]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:39f50e8a8d6a10fd epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:39f50e8a8d6a10fd"}
17:46:19 [17:46:19.230]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:39f50e8a8d6a10fd type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:39f50e8a8d6a10fd"}
17:46:19 [17:46:19.238]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:39f50e8a8d6a10fd epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:39f50e8a8d6a10fd"}
17:46:19 [17:46:19.242]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.243]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:970046fa9126bf6e epochNumber=1
{"provingJobId":"1:BASE_PARITY:970046fa9126bf6e"}
17:46:19 [17:46:19.244]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9sBHgA/1 with map size 1073741824
17:46:19 [17:46:19.246]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.272]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.275]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.276]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f264d1260c7cc08 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f264d1260c7cc08"}
17:46:19 [17:46:19.277]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFw9anp/1 with map size 1073741824
17:46:19 [17:46:19.278]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.298]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7f264d1260c7cc08
{"provingJobId":"1:BASE_PARITY:7f264d1260c7cc08"}
17:46:19 [17:46:19.298]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7f264d1260c7cc08 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7f264d1260c7cc08"}
17:46:19 [17:46:19.299]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7f264d1260c7cc08 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:7f264d1260c7cc08"}
17:46:19 [17:46:19.310]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.311]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d356b5e704f128b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d356b5e704f128b4"}
17:46:19 [17:46:19.311]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGZ7rW6/1 with map size 1073741824
17:46:19 [17:46:19.313]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.331]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d356b5e704f128b4
{"provingJobId":"1:BASE_PARITY:d356b5e704f128b4"}
17:46:19 [17:46:19.332]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d356b5e704f128b4 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:d356b5e704f128b4"}
17:46:19 [17:46:19.341]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.342]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:017553e2ad4d33c7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:017553e2ad4d33c7"}
17:46:19 [17:46:19.343]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwsh6na/1 with map size 1073741824
17:46:19 [17:46:19.345]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.361]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:017553e2ad4d33c7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:017553e2ad4d33c7"}
17:46:19 [17:46:19.362]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:017553e2ad4d33c7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:017553e2ad4d33c7"}
17:46:19 [17:46:19.370]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.371]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d5fac874729bbf0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d5fac874729bbf0"}
17:46:19 [17:46:19.371]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuOu9HO/1 with map size 1073741824
17:46:19 [17:46:19.373]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.391]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7d5fac874729bbf0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7d5fac874729bbf0"}
17:46:19 [17:46:19.392]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7d5fac874729bbf0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7d5fac874729bbf0"}
17:46:19 [17:46:19.400]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.401]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c9f67c4ea23a062 epochNumber=0
{"provingJobId":"1:BASE_PARITY:0c9f67c4ea23a062"}
17:46:19 [17:46:19.401]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testN5RbmN/0 with map size 1073741824
17:46:19 [17:46:19.403]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.420]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0c9f67c4ea23a062 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0c9f67c4ea23a062"}
17:46:19 [17:46:19.421]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a44da6bfcb359a55 epochNumber=0
{"provingJobId":"1:BASE_PARITY:a44da6bfcb359a55"}
17:46:19 [17:46:19.421]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0c9f67c4ea23a062 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0c9f67c4ea23a062"}
17:46:19 [17:46:19.422]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf1db8ea9586405d epochNumber=0
{"provingJobId":"1:BASE_PARITY:bf1db8ea9586405d"}
17:46:19 [17:46:19.437]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.440]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb1d64d9e9987abb epochNumber=2
{"provingJobId":"1:BASE_PARITY:cb1d64d9e9987abb"}
17:46:19 [17:46:19.441]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUnvJN9/2 with map size 1073741824
17:46:19 [17:46:19.443]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.460]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1cbdef78a52ba4af epochNumber=1
{"provingJobId":"1:BASE_PARITY:1cbdef78a52ba4af"}
17:46:19 [17:46:19.461]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUnvJN9/1 with map size 1073741824
17:46:19 [17:46:19.462]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.482]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.483]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:84ade73a2635712d epochNumber=1
{"provingJobId":"1:BASE_PARITY:84ade73a2635712d"}
17:46:19 [17:46:19.484]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMDrluv/1 with map size 1073741824
17:46:19 [17:46:19.486]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.505]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.506]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d054ed4fe833695d epochNumber=1
{"provingJobId":"1:BASE_PARITY:d054ed4fe833695d"}
17:46:19 [17:46:19.506]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSIbPmk/1 with map size 1073741824
17:46:19 [17:46:19.508]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.525]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:22735d20dbe2354e epochNumber=1
{"provingJobId":"1:BASE_PARITY:22735d20dbe2354e"}
17:46:19 [17:46:19.531]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7a742d8ee1f0e8a epochNumber=2
{"provingJobId":"1:BASE_PARITY:d7a742d8ee1f0e8a"}
17:46:19 [17:46:19.532]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSIbPmk/2 with map size 1073741824
17:46:19 [17:46:19.534]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.552]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:582c1369df7acdef epochNumber=1
{"provingJobId":"1:BASE_PARITY:582c1369df7acdef"}
17:46:19 [17:46:19.561]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.562]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f33b556697ae879d epochNumber=1
{"provingJobId":"1:BASE_PARITY:f33b556697ae879d"}
17:46:19 [17:46:19.562]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpYIaoA/1 with map size 1073741824
17:46:19 [17:46:19.567]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.587]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:56a2d06d44a5bd29 epochNumber=1
{"provingJobId":"1:BASE_PARITY:56a2d06d44a5bd29"}
17:46:19 [17:46:19.593]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a8478acdfa75b63 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5a8478acdfa75b63"}
17:46:19 [17:46:19.594]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpYIaoA/2 with map size 1073741824
17:46:19 [17:46:19.596]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.612]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fca091869794868d epochNumber=1
{"provingJobId":"1:BASE_PARITY:fca091869794868d"}
17:46:19 [17:46:19.624]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.625]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:675127ef8cbddfad epochNumber=1
{"provingJobId":"1:BASE_PARITY:675127ef8cbddfad"}
17:46:19 [17:46:19.626]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiKfrLp/1 with map size 1073741824
17:46:19 [17:46:19.627]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.645]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:062f542a25f23132 epochNumber=1
{"provingJobId":"1:BASE_PARITY:062f542a25f23132"}
17:46:19 [17:46:19.653]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3d34c2d71262e72 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d3d34c2d71262e72"}
17:46:19 [17:46:19.654]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiKfrLp/2 with map size 1073741824
17:46:19 [17:46:19.656]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.675]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5470cc573a29cdbe epochNumber=1
{"provingJobId":"1:BASE_PARITY:5470cc573a29cdbe"}
17:46:19 [17:46:19.686]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.686]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:47e1e2fa3de11d2e epochNumber=1
{"provingJobId":"1:BASE_PARITY:47e1e2fa3de11d2e"}
17:46:19 [17:46:19.687]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7jxhC4/1 with map size 1073741824
17:46:19 [17:46:19.688]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.708]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b4037f7d21d5e046 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b4037f7d21d5e046"}
17:46:19 [17:46:19.714]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:47e1e2fa3de11d2e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:47e1e2fa3de11d2e"}
17:46:19 [17:46:19.725]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.726]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:188770cbeca1a976 epochNumber=1
{"provingJobId":"1:BASE_PARITY:188770cbeca1a976"}
17:46:19 [17:46:19.727]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAksmkO/1 with map size 1073741824
17:46:19 [17:46:19.729]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.749]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cc0080e3d9839eec epochNumber=1
{"provingJobId":"1:BASE_PARITY:cc0080e3d9839eec"}
17:46:19 [17:46:19.755]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:188770cbeca1a976 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:188770cbeca1a976"}
17:46:19 [17:46:19.766]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.767]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ad244fde641ce34 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9ad244fde641ce34"}
17:46:19 [17:46:19.768]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFE7tOZ/1 with map size 1073741824
17:46:19 [17:46:19.770]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.788]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de0034432776d391 epochNumber=1
{"provingJobId":"1:BASE_PARITY:de0034432776d391"}
17:46:19 [17:46:19.795]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9ad244fde641ce34 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:9ad244fde641ce34"}
17:46:19 [17:46:19.799]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.800]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aa5f5a6842f5c6c9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aa5f5a6842f5c6c9"}
17:46:19 [17:46:19.801]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDlK1ep/1 with map size 1073741824
17:46:19 [17:46:19.806]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.825]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:aa5f5a6842f5c6c9
{"provingJobId":"1:BASE_PARITY:aa5f5a6842f5c6c9"}
17:46:19 [17:46:19.825]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:aa5f5a6842f5c6c9 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:aa5f5a6842f5c6c9"}
17:46:19 [17:46:19.831]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:10d07611d1d24f2d epochNumber=1
{"provingJobId":"1:BASE_PARITY:10d07611d1d24f2d"}
17:46:19 [17:46:19.837]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:aa5f5a6842f5c6c9 has already been completed
{"provingJobId":"1:BASE_PARITY:aa5f5a6842f5c6c9"}
17:46:19 [17:46:19.843]
INFO:
prover-client:proving-broker Proving Broker started
17:46:19 [17:46:19.844]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bbf43a1a9a8b49b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bbf43a1a9a8b49b4"}
17:46:19 [17:46:19.845]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbZlNkX/1 with map size 1073741824
17:46:19 [17:46:19.847]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:19 [17:46:19.864]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a3ca1de5ab35f0d epochNumber=2
{"provingJobId":"1:BASE_PARITY:5a3ca1de5ab35f0d"}
17:46:19 [17:46:19.865]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testbZlNkX/2 with map size 1073741824
17:46:19 [17:46:19.866]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:20 [17:46:20.959]
INFO:
prover-client:proving-broker Proving Broker started
17:46:20 [17:46:20.966]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bbf43a1a9a8b49b4 settled=false
{"provingJobId":"1:BASE_PARITY:bbf43a1a9a8b49b4","status":"pending"}
17:46:20 [17:46:20.969]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5a3ca1de5ab35f0d settled=false
{"provingJobId":"1:BASE_PARITY:5a3ca1de5ab35f0d","status":"pending"}
17:46:20 [17:46:20.971]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bbf43a1a9a8b49b4 type=BASE_PARITY startedAt=1750095979908 older agent has taken job
{"provingJobId":"1:BASE_PARITY:bbf43a1a9a8b49b4"}
17:46:20 [17:46:20.972]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bbf43a1a9a8b49b4 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:bbf43a1a9a8b49b4"}
17:46:20 [17:46:20.977]
INFO:
prover-client:proving-broker Proving Broker started
17:46:20 [17:46:20.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cea40997cbc8051d epochNumber=1
{"provingJobId":"1:BASE_PARITY:cea40997cbc8051d"}
17:46:20 [17:46:20.978]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI29B5G/1 with map size 1073741824
17:46:20 [17:46:20.981]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:21 [17:46:21.004]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb523456ebe8d0b4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:cb523456ebe8d0b4"}
17:46:21 [17:46:21.006]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testI29B5G/2 with map size 1073741824
17:46:21 [17:46:21.010]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:22 [17:46:22.027]
INFO:
prover-client:proving-broker Proving Broker started
17:46:22 [17:46:22.031]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cea40997cbc8051d settled=false
{"provingJobId":"1:BASE_PARITY:cea40997cbc8051d","status":"pending"}
17:46:22 [17:46:22.034]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cb523456ebe8d0b4 settled=false
{"provingJobId":"1:BASE_PARITY:cb523456ebe8d0b4","status":"pending"}
17:46:22 [17:46:22.035]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cea40997cbc8051d type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:cea40997cbc8051d"}
17:46:22 [17:46:22.040]
INFO:
prover-client:proving-broker Proving Broker started
17:46:22 [17:46:22.040]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de8878a6ef997dce epochNumber=1
{"provingJobId":"1:BASE_PARITY:de8878a6ef997dce"}
17:46:22 [17:46:22.041]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2pYnLU/1 with map size 1073741824
17:46:22 [17:46:22.043]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:22 [17:46:22.058]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f34525325cfa3577 epochNumber=2
{"provingJobId":"1:BASE_PARITY:f34525325cfa3577"}
17:46:22 [17:46:22.059]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test2pYnLU/2 with map size 1073741824
17:46:22 [17:46:22.061]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.077]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.078]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:de8878a6ef997dce settled=false
{"provingJobId":"1:BASE_PARITY:de8878a6ef997dce","status":"pending"}
17:46:23 [17:46:23.080]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f34525325cfa3577 settled=false
{"provingJobId":"1:BASE_PARITY:f34525325cfa3577","status":"pending"}
17:46:23 [17:46:23.080]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:de8878a6ef997dce type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:de8878a6ef997dce"}
17:46:23 [17:46:23.081]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:de8878a6ef997dce type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:de8878a6ef997dce"}
17:46:23 [17:46:23.105]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.106]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c965229996a105cf epochNumber=1
{"provingJobId":"1:BASE_PARITY:c965229996a105cf"}
17:46:23 [17:46:23.106]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNcV2Al/1 with map size 1073741824
17:46:23 [17:46:23.110]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.128]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1ece6a6fd7b127b4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1ece6a6fd7b127b4"}
17:46:23 [17:46:23.129]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNcV2Al/2 with map size 1073741824
17:46:23 [17:46:23.135]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.152]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c965229996a105cf type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c965229996a105cf"}
17:46:23 [17:46:23.158]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1ece6a6fd7b127b4 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1ece6a6fd7b127b4"}
17:46:23 [17:46:23.169]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.170]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:114fd0eec23ba211 epochNumber=1
{"provingJobId":"1:BASE_PARITY:114fd0eec23ba211"}
17:46:23 [17:46:23.173]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmNYBDc/1 with map size 1073741824
17:46:23 [17:46:23.175]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.200]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a996c0dc16fe6bde epochNumber=2
{"provingJobId":"1:BASE_PARITY:a996c0dc16fe6bde"}
17:46:23 [17:46:23.202]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testmNYBDc/2 with map size 1073741824
17:46:23 [17:46:23.206]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.228]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:114fd0eec23ba211 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:114fd0eec23ba211"}
17:46:23 [17:46:23.229]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:114fd0eec23ba211 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:114fd0eec23ba211"}
17:46:23 [17:46:23.236]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a996c0dc16fe6bde type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a996c0dc16fe6bde"}
17:46:23 [17:46:23.247]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.248]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a0a21d8c57ad37cb err=test error
{"provingJoId":"1:BASE_PARITY:a0a21d8c57ad37cb"}
17:46:23 [17:46:23.254]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.256]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9a4bccd330abf423 not found
{"provingJobId":"1:BASE_PARITY:9a4bccd330abf423"}
17:46:23 [17:46:23.262]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.263]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aaf550e9056c57a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aaf550e9056c57a3"}
17:46:23 [17:46:23.264]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoGA2mZ/1 with map size 1073741824
17:46:23 [17:46:23.267]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.299]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.301]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c8acfb14d39fcc3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1c8acfb14d39fcc3"}
17:46:23 [17:46:23.303]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcbAsU4/1 with map size 1073741824
17:46:23 [17:46:23.308]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.452]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1c8acfb14d39fcc3 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:1c8acfb14d39fcc3"}
17:46:23 [17:46:23.460]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.462]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:05de1715b0790ecf epochNumber=1
{"provingJobId":"1:BASE_PARITY:05de1715b0790ecf"}
17:46:23 [17:46:23.464]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testop1sOR/1 with map size 1073741824
17:46:23 [17:46:23.466]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.599]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:05de1715b0790ecf timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:05de1715b0790ecf"}
17:46:23 [17:46:23.603]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b19f250302b2f31d epochNumber=10
{"provingJobId":"1:BASE_PARITY:b19f250302b2f31d"}
17:46:23 [17:46:23.605]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testop1sOR/10 with map size 1073741824
17:46:23 [17:46:23.608]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.652]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:23 [17:46:23.653]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:46:23 [17:46:23.764]
INFO:
prover-client:proving-broker Proving Broker started
17:46:23 [17:46:23.766]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2275d3478e6db2ec epochNumber=1
{"provingJobId":"1:BASE_PARITY:2275d3478e6db2ec"}
17:46:23 [17:46:23.767]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testy3iZi5/1 with map size 1073741824
17:46:23 [17:46:23.771]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:23 [17:46:23.946]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2275d3478e6db2ec timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2275d3478e6db2ec"}
17:46:24 [17:46:24.000]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.001]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e414ea4526c68cbf epochNumber=1
{"provingJobId":"1:BASE_PARITY:e414ea4526c68cbf"}
17:46:24 [17:46:24.002]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlF522p/1 with map size 1073741824
17:46:24 [17:46:24.009]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.029]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e414ea4526c68cbf type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e414ea4526c68cbf"}
17:46:24 [17:46:24.033]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.034]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49d5d78383efe5c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:49d5d78383efe5c0"}
17:46:24 [17:46:24.034]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkYR01F/1 with map size 1073741824
17:46:24 [17:46:24.037]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.056]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:49d5d78383efe5c0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:49d5d78383efe5c0"}
17:46:24 [17:46:24.056]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:49d5d78383efe5c0 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:49d5d78383efe5c0"}
17:46:24 [17:46:24.066]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.067]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c3e45fc250f2b753 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c3e45fc250f2b753"}
17:46:24 [17:46:24.068]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testA6XCDt/1 with map size 1073741824
17:46:24 [17:46:24.070]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.090]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c3e45fc250f2b753 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c3e45fc250f2b753"}
17:46:24 [17:46:24.109]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.109]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:359d5df32b106c77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:359d5df32b106c77"}
17:46:24 [17:46:24.110]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbFGUrc/1 with map size 1073741824
17:46:24 [17:46:24.113]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.130]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:359d5df32b106c77 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:359d5df32b106c77"}
17:46:24 [17:46:24.132]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eb1197536e87c93f epochNumber=3
{"provingJobId":"1:BASE_PARITY:eb1197536e87c93f"}
17:46:24 [17:46:24.133]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testbFGUrc/3 with map size 1073741824
17:46:24 [17:46:24.135]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.160]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:24 [17:46:24.161]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:46:24 [17:46:24.177]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:359d5df32b106c77 err=test error
{"provingJoId":"1:BASE_PARITY:359d5df32b106c77"}
17:46:24 [17:46:24.186]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLA9rfY/1 with map size 1073741824
17:46:24 [17:46:24.196]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.221]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLA9rfY/2 with map size 1073741824
17:46:24 [17:46:24.223]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.239]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.241]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4ca691e437138343 settled=false
{"provingJobId":"1:BASE_PARITY:4ca691e437138343","status":"pending"}
17:46:24 [17:46:24.243]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d8ab107f22d01ead settled=false
{"provingJobId":"1:BASE_PARITY:d8ab107f22d01ead","status":"pending"}
17:46:24 [17:46:24.254]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUYmJaN/1 with map size 1073741824
17:46:24 [17:46:24.256]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.279]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUYmJaN/2 with map size 1073741824
17:46:24 [17:46:24.282]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.322]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.326]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6c2815dbf194f5da settled=true
{"provingJobId":"1:BASE_PARITY:6c2815dbf194f5da","status":"fulfilled"}
17:46:24 [17:46:24.328]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:b2114e3a0736262b settled=true
{"provingJobId":"2:BASE_PARITY:b2114e3a0736262b","status":"fulfilled"}
17:46:24 [17:46:24.336]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbhPOG0/1 with map size 1073741824
17:46:24 [17:46:24.339]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.370]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testbhPOG0/2 with map size 1073741824
17:46:24 [17:46:24.373]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.390]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.392]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e7ddcc43386b1219 settled=true
{"provingJobId":"1:BASE_PARITY:e7ddcc43386b1219","status":"fulfilled"}
17:46:24 [17:46:24.395]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4867049219e72bbc settled=false
{"provingJobId":"1:BASE_PARITY:4867049219e72bbc","status":"pending"}
17:46:24 [17:46:24.403]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.404]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a9695454e4be0e43 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a9695454e4be0e43"}
17:46:24 [17:46:24.406]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaqvdFt/1 with map size 1073741824
17:46:24 [17:46:24.410]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.435]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.437]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:79fcc444907ea1f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:79fcc444907ea1f5"}
17:46:24 [17:46:24.437]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:79fcc444907ea1f5: Error: db error: Error: db error
17:46:24 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:79fcc444907ea1f5"}
17:46:24 [17:46:24.444]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:090c0be9a60341be epochNumber=1
{"provingJobId":"1:BASE_PARITY:090c0be9a60341be"}
17:46:24 [17:46:24.446]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxD2peJ/1 with map size 1073741824
17:46:24 [17:46:24.449]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.469]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:090c0be9a60341be type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:090c0be9a60341be"}
17:46:24 [17:46:24.469]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:090c0be9a60341be type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:090c0be9a60341be"}
17:46:24 [17:46:24.483]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.484]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ef894fb565603df4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ef894fb565603df4"}
17:46:24 [17:46:24.485]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQUXsAe/1 with map size 1073741824
17:46:24 [17:46:24.490]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.516]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ef894fb565603df4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ef894fb565603df4"}
17:46:24 [17:46:24.517]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ef894fb565603df4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ef894fb565603df4"}
17:46:24 [17:46:24.517]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:ef894fb565603df4: Error: db error
17:46:24 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:ef894fb565603df4"}
17:46:24 [17:46:24.522]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.522]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa73a2b36f69ccb0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa73a2b36f69ccb0"}
17:46:24 [17:46:24.524]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testG9HJAj/1 with map size 1073741824
17:46:24 [17:46:24.528]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.547]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fa73a2b36f69ccb0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fa73a2b36f69ccb0"}
17:46:24 [17:46:24.548]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fa73a2b36f69ccb0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fa73a2b36f69ccb0"}
17:46:24 [17:46:24.562]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.563]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8db58a0ebda36903 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8db58a0ebda36903"}
17:46:24 [17:46:24.564]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwk5b6O/1 with map size 1073741824
17:46:24 [17:46:24.566]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.588]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8db58a0ebda36903 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8db58a0ebda36903"}
17:46:24 [17:46:24.589]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8db58a0ebda36903 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8db58a0ebda36903"}
17:46:24 [17:46:24.590]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:8db58a0ebda36903 jobErr=test error: Error: db error
17:46:24 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:8db58a0ebda36903"}
17:46:24 [17:46:24.595]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.596]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4238a20a4e10806c not found
{"provingJobId":"1:BASE_PARITY:4238a20a4e10806c"}
17:46:24 [17:46:24.599]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.599]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:14773229449f216d err=test error
{"provingJoId":"1:BASE_PARITY:14773229449f216d"}
17:46:24 [17:46:24.604]
INFO:
prover-client:proving-broker Proving Broker started
17:46:24 [17:46:24.615]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f81db82472785ddb epochNumber=1
{"provingJobId":"1:BASE_PARITY:f81db82472785ddb"}
17:46:24 [17:46:24.617]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testthGnY0/1 with map size 1073741824
17:46:24 [17:46:24.619]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.640]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f81db82472785ddb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f81db82472785ddb"}
17:46:24 [17:46:24.641]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f81db82472785ddb type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f81db82472785ddb"}
17:46:24 [17:46:24.661]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:9f1f7e340ff1aa2f epochNumber=2
{"provingJobId":"2:BASE_PARITY:9f1f7e340ff1aa2f"}
17:46:24 [17:46:24.662]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testthGnY0/2 with map size 1073741824
17:46:24 [17:46:24.665]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.685]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:9f1f7e340ff1aa2f type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:9f1f7e340ff1aa2f"}
17:46:24 [17:46:24.686]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:9f1f7e340ff1aa2f type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:9f1f7e340ff1aa2f"}
17:46:24 [17:46:24.752]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:a8ccc56fa3d2a3c3 epochNumber=3
{"provingJobId":"3:BASE_PARITY:a8ccc56fa3d2a3c3"}
17:46:24 [17:46:24.753]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testthGnY0/3 with map size 1073741824
17:46:24 [17:46:24.760]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.809]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:24 [17:46:24.810]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:46:24 [17:46:24.853]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:dad5f647ebf38676 epochNumber=4
{"provingJobId":"4:BASE_PARITY:dad5f647ebf38676"}
17:46:24 [17:46:24.854]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testthGnY0/4 with map size 1073741824
17:46:24 [17:46:24.857]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.889]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:24 [17:46:24.890]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
17:46:24 [17:46:24.942]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:5480f44e0d6d21ab epochNumber=5
{"provingJobId":"5:BASE_PARITY:5480f44e0d6d21ab"}
17:46:24 [17:46:24.944]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testthGnY0/5 with map size 1073741824
17:46:24 [17:46:24.950]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:46:24 [17:46:24.974]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:46:24 [17:46:24.975]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
17:46:25
PASS src/proving_broker/proving_broker.test.ts (
13.326 s)
17:46:25 ProvingBroker
17:46:25 Producer API
17:46:25
✓ refuses stale jobs (44 ms)
17:46:25
✓ enqueues jobs (4 ms)
17:46:25
✓ ignores duplicate jobs (4 ms)
17:46:25
✓ reports correct status when enqueuing repeat jobs (4 ms)
17:46:25
✓ throws an error in case of duplicate job IDs (31 ms)
17:46:25
✓ returns not-found status for non-existing jobs (1 ms)
17:46:25
✓ cancels jobs in queue (3 ms)
17:46:25
✓ cancels jobs in-progress (2 ms)
17:46:25
✓ returns job result if successful (2 ms)
17:46:25
✓ returns job error if failed (2 ms)
17:46:25
✓ correctly returns job status for concurrent writes (3 ms)
17:46:25
✓ refuses stale jobs (40 ms)
17:46:25
✓ enqueues jobs (19 ms)
17:46:25
✓ ignores duplicate jobs (18 ms)
17:46:25
✓ reports correct status when enqueuing repeat jobs (34 ms)
17:46:25
✓ throws an error in case of duplicate job IDs (30 ms)
17:46:25
✓ returns not-found status for non-existing jobs (3 ms)
17:46:25
✓ cancels jobs in queue (33 ms)
17:46:25
✓ cancels jobs in-progress (31 ms)
17:46:25
✓ returns job result if successful (29 ms)
17:46:25
✓ returns job error if failed (29 ms)
17:46:25
✓ correctly returns job status for concurrent writes (36 ms)
17:46:25 Consumer API
17:46:25
✓ returns undefined if no jobs are available (1 ms)
17:46:25
✓ returns jobs in priority order (2 ms)
17:46:25
✓ returns undefined if no jobs are available for the given allowList (1 ms)
17:46:25
✓ returns a job if it is in the allowList (3 ms)
17:46:25
✓ returns the most important job if it is in the allowList (2 ms)
17:46:25
✓ returns any job if filter is empty (2 ms)
17:46:25
✓ returns a new job when reporting job success (2 ms)
17:46:25
✓ returns a new job when reporting permanent error (2 ms)
17:46:25
✓ returns a new job when reporting retry-able error (2 ms)
17:46:25
✓ returns a new job when reporting progress if current one is cancelled (2 ms)
17:46:25
✓ returns a new job if job is already in progress elsewhere (1057 ms)
17:46:25
✓ avoids sending the same job to a new agent after a restart (1006 ms)
17:46:25
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
17:46:25
✓ tracks job result if in progress (6 ms)
17:46:25
✓ tracks job result even if job is in queue (5 ms)
17:46:25
✓ ignores reported job error if unknown job (1 ms)
17:46:25
✓ ignores job result if unknown job (1 ms)
17:46:25
✓ returns undefined if no jobs are available (3 ms)
17:46:25
✓ returns jobs in priority order (41 ms)
17:46:25
✓ returns undefined if no jobs are available for the given allowList (22 ms)
17:46:25
✓ returns a job if it is in the allowList (55 ms)
17:46:25
✓ returns the most important job if it is in the allowList (62 ms)
17:46:25
✓ returns any job if filter is empty (61 ms)
17:46:25
✓ returns a new job when reporting job success (38 ms)
17:46:25
✓ returns a new job when reporting permanent error (41 ms)
17:46:25
✓ returns a new job when reporting retry-able error (33 ms)
17:46:25
✓ returns a new job when reporting progress if current one is cancelled (43 ms)
17:46:25
✓ returns a new job if job is already in progress elsewhere (1134 ms)
17:46:25
✓ avoids sending the same job to a new agent after a restart (1062 ms)
17:46:25
✓ avoids sending a completed job to a new agent after a restart (1062 ms)
17:46:25
✓ tracks job result if in progress (64 ms)
17:46:25
✓ tracks job result even if job is in queue (78 ms)
17:46:25
✓ ignores reported job error if unknown job (4 ms)
17:46:25
✓ ignores job result if unknown job (7 ms)
17:46:25 Timeouts
17:46:25
✓ tracks in progress jobs (1 ms)
17:46:25
✓ re-enqueues jobs that time out (108 ms)
17:46:25
✓ cancel stale jobs that time out (236 ms)
17:46:25
✓ keeps the jobs in progress while it is alive (207 ms)
17:46:25
✓ tracks in progress jobs (37 ms)
17:46:25
✓ re-enqueues jobs that time out (159 ms)
17:46:25
✓ cancel stale jobs that time out (303 ms)
17:46:25
✓ keeps the jobs in progress while it is alive (238 ms)
17:46:25 Retries
17:46:25
✓ retries jobs (3 ms)
17:46:25
✓ retries up to a maximum number of times (4 ms)
17:46:25
✓ passing retry=false does not retry (6 ms)
17:46:25
✓ does not retry if job is stale (33 ms)
17:46:25
✓ retries jobs (33 ms)
17:46:25
✓ retries up to a maximum number of times (32 ms)
17:46:25
✓ passing retry=false does not retry (39 ms)
17:46:25
✓ does not retry if job is stale (76 ms)
17:46:25 Database management
17:46:25
✓ re-enqueues proof requests on start (3 ms)
17:46:25
✓ restores proof results on start (2 ms)
17:46:25
✓ only re-enqueues unfinished jobs (1 ms)
17:46:25
✓ saves job when enqueued (2 ms)
17:46:25
✓ does not retain job if database fails to save (2 ms)
17:46:25
✓ saves job result (1 ms)
17:46:25
✓ saves result even if database fails to save (2 ms)
17:46:25
✓ saves job error (2 ms)
17:46:25
✓ saves job error even if database fails to save (2 ms)
17:46:25
✓ does not save job result if job is unknown (2 ms)
17:46:25
✓ does not save job error if job is unknown (1 ms)
17:46:25
✓ cleans up old jobs periodically (261 ms)
17:46:25
✓ re-enqueues proof requests on start (68 ms)
17:46:25
✓ restores proof results on start (81 ms)
17:46:25
✓ only re-enqueues unfinished jobs (67 ms)
17:46:25
✓ saves job when enqueued (33 ms)
17:46:25
✓ does not retain job if database fails to save (6 ms)
17:46:25
✓ saves job result (39 ms)
17:46:25
✓ saves result even if database fails to save (40 ms)
17:46:25
✓ saves job error (39 ms)
17:46:25
✓ saves job error even if database fails to save (32 ms)
17:46:25
✓ does not save job result if job is unknown (4 ms)
17:46:25
✓ does not save job error if job is unknown (3 ms)
17:46:25
✓ cleans up old jobs periodically (427 ms)
17:46:25
17:46:25
Test Suites: 1 passed, 1 total
17:46:25
Tests: 96 passed, 96 total
17:46:25
Snapshots: 0 total
17:46:25
Time: 13.431 s
17:46:25
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
17:46:25
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?