Parent Log: http://ci.aztec-labs.com/cfa3a08d8c538c28 Command: 1905308686f8cbc3 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/30660269b33bab8cca354c41659533acf4d48e07 Env: REF_NAME=gh-readonly-queue/next/pr-15026-d96baf1c44329e8b2e3a432ad803f702f5184a62 CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Fri Jun 13 09:36:37 UTC 2025 System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15026_arm64_a1-fast Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 09:36:40 [09:36:40.044] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.049] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0a8d48b91fbb3344 epochNumber=42 {"provingJobId":"1:BASE_PARITY:0a8d48b91fbb3344"} 09:36:40 [09:36:40.052] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:7ef1d1aedf7bc006 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7ef1d1aedf7bc006"} 09:36:40 [09:36:40.075] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5c518a60647e517 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5c518a60647e517"} 09:36:40 [09:36:40.077] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb47c3a50b439f99 epochNumber=1 {"provingJobId":"1:BASE_PARITY:eb47c3a50b439f99"} 09:36:40 [09:36:40.078] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.079] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7288808198be7d40 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7288808198be7d40"} 09:36:40 [09:36:40.080] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7288808198be7d40 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7288808198be7d40"} 09:36:40 [09:36:40.082] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.083] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7ac90813ca54050b epochNumber=1 {"provingJobId":"1:BASE_PARITY:7ac90813ca54050b"} 09:36:40 [09:36:40.084] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7ac90813ca54050b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7ac90813ca54050b"} 09:36:40 [09:36:40.085] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7ac90813ca54050b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7ac90813ca54050b"} 09:36:40 [09:36:40.085] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7ac90813ca54050b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7ac90813ca54050b"} 09:36:40 [09:36:40.086] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.087] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:92a4a274b2f96da5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:92a4a274b2f96da5"} 09:36:40 [09:36:40.109] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.110] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:92ec4e63490b78f8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:92ec4e63490b78f8"} 09:36:40 [09:36:40.111] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:92ec4e63490b78f8 {"provingJobId":"1:BASE_PARITY:92ec4e63490b78f8"} 09:36:40 [09:36:40.111] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:92ec4e63490b78f8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:92ec4e63490b78f8"} 09:36:40 [09:36:40.112] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:92ec4e63490b78f8 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:92ec4e63490b78f8"} 09:36:40 [09:36:40.113] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.113] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aad1d6c6fb2bb8a0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aad1d6c6fb2bb8a0"} 09:36:40 [09:36:40.114] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:aad1d6c6fb2bb8a0 {"provingJobId":"1:BASE_PARITY:aad1d6c6fb2bb8a0"} 09:36:40 [09:36:40.114] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:aad1d6c6fb2bb8a0 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:aad1d6c6fb2bb8a0"} 09:36:40 [09:36:40.115] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.116] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:80fed18c33f0200d epochNumber=1 {"provingJobId":"1:BASE_PARITY:80fed18c33f0200d"} 09:36:40 [09:36:40.117] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:80fed18c33f0200d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:80fed18c33f0200d"} 09:36:40 [09:36:40.117] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:80fed18c33f0200d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:80fed18c33f0200d"} 09:36:40 [09:36:40.118] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:24876efab598236e epochNumber=1 {"provingJobId":"1:BASE_PARITY:24876efab598236e"} 09:36:40 [09:36:40.119] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:24876efab598236e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:24876efab598236e"} 09:36:40 [09:36:40.119] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:24876efab598236e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:24876efab598236e"} 09:36:40 [09:36:40.120] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.120] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:44269de40eaae25e epochNumber=0 {"provingJobId":"1:BASE_PARITY:44269de40eaae25e"} 09:36:40 [09:36:40.121] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:44269de40eaae25e epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:44269de40eaae25e"} 09:36:40 [09:36:40.121] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b543ca41dabfc298 epochNumber=0 {"provingJobId":"1:BASE_PARITY:b543ca41dabfc298"} 09:36:40 [09:36:40.121] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:44269de40eaae25e epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:44269de40eaae25e"} 09:36:40 [09:36:40.121] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b222ceebecd7110 epochNumber=0 {"provingJobId":"1:BASE_PARITY:6b222ceebecd7110"} 09:36:40 [09:36:40.122] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.123] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1eee0bd89f97014f epochNumber=2 {"provingJobId":"1:BASE_PARITY:1eee0bd89f97014f"} 09:36:40 [09:36:40.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:890ba0119ed6c70d epochNumber=1 {"provingJobId":"1:BASE_PARITY:890ba0119ed6c70d"} 09:36:40 [09:36:40.125] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.125] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:83e879f5190fc938 epochNumber=1 {"provingJobId":"1:BASE_PARITY:83e879f5190fc938"} 09:36:40 [09:36:40.126] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.126] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a4f2baad8131b7b epochNumber=1 {"provingJobId":"1:BASE_PARITY:6a4f2baad8131b7b"} 09:36:40 [09:36:40.127] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e70bb2b7cfc62216 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e70bb2b7cfc62216"} 09:36:40 [09:36:40.127] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2b92b1623ea74be epochNumber=2 {"provingJobId":"1:BASE_PARITY:e2b92b1623ea74be"} 09:36:40 [09:36:40.127] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8e9288dc976a8d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a8e9288dc976a8d3"} 09:36:40 [09:36:40.128] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a5c34c46853a8861 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a5c34c46853a8861"} 09:36:40 [09:36:40.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:627d900222910024 epochNumber=1 {"provingJobId":"1:BASE_PARITY:627d900222910024"} 09:36:40 [09:36:40.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2c531dc9c02ce2d2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2c531dc9c02ce2d2"} 09:36:40 [09:36:40.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7be5379599d3223f epochNumber=1 {"provingJobId":"1:BASE_PARITY:7be5379599d3223f"} 09:36:40 [09:36:40.130] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.131] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0de8325f9ee84bc epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0de8325f9ee84bc"} 09:36:40 [09:36:40.131] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:172cad2627bc6608 epochNumber=1 {"provingJobId":"1:BASE_PARITY:172cad2627bc6608"} 09:36:40 [09:36:40.131] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f9f6b3216d54475f epochNumber=2 {"provingJobId":"1:BASE_PARITY:f9f6b3216d54475f"} 09:36:40 [09:36:40.131] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a7a0d675d337932 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a7a0d675d337932"} 09:36:40 [09:36:40.132] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.133] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e7f9852717e97c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e7f9852717e97c1"} 09:36:40 [09:36:40.133] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aae9f69ca92038ff epochNumber=1 {"provingJobId":"1:BASE_PARITY:aae9f69ca92038ff"} 09:36:40 [09:36:40.133] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7e7f9852717e97c1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7e7f9852717e97c1"} 09:36:40 [09:36:40.134] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.135] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:47308cb269bc560f epochNumber=1 {"provingJobId":"1:BASE_PARITY:47308cb269bc560f"} 09:36:40 [09:36:40.135] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15b1cc3f10bb9121 epochNumber=1 {"provingJobId":"1:BASE_PARITY:15b1cc3f10bb9121"} 09:36:40 [09:36:40.135] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:47308cb269bc560f type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:47308cb269bc560f"} 09:36:40 [09:36:40.136] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.136] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e7af9289be43b20f epochNumber=1 {"provingJobId":"1:BASE_PARITY:e7af9289be43b20f"} 09:36:40 [09:36:40.137] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cbc911b70cbe89b8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cbc911b70cbe89b8"} 09:36:40 [09:36:40.137] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e7af9289be43b20f type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:e7af9289be43b20f"} 09:36:40 [09:36:40.138] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.138] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5505dd23963877ce epochNumber=1 {"provingJobId":"1:BASE_PARITY:5505dd23963877ce"} 09:36:40 [09:36:40.139] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5505dd23963877ce {"provingJobId":"1:BASE_PARITY:5505dd23963877ce"} 09:36:40 [09:36:40.139] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5505dd23963877ce type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:5505dd23963877ce"} 09:36:40 [09:36:40.139] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:92879dc19bda609f epochNumber=1 {"provingJobId":"1:BASE_PARITY:92879dc19bda609f"} 09:36:40 [09:36:40.140] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5505dd23963877ce has already been completed {"provingJobId":"1:BASE_PARITY:5505dd23963877ce"} 09:36:40 [09:36:40.140] INFO: prover-client:proving-broker Proving Broker started 09:36:40 [09:36:40.141] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4cb52e5756f97522 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4cb52e5756f97522"} 09:36:40 [09:36:40.141] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5aba214e1b570b7a epochNumber=2 {"provingJobId":"1:BASE_PARITY:5aba214e1b570b7a"} 09:36:41 [09:36:41.192] INFO: prover-client:proving-broker Proving Broker started 09:36:41 [09:36:41.192] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4cb52e5756f97522 settled=false {"provingJobId":"1:BASE_PARITY:4cb52e5756f97522","status":"pending"} 09:36:41 [09:36:41.193] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5aba214e1b570b7a settled=false {"provingJobId":"1:BASE_PARITY:5aba214e1b570b7a","status":"pending"} 09:36:41 [09:36:41.193] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:4cb52e5756f97522 type=BASE_PARITY startedAt=1749807400141 older agent has taken job {"provingJobId":"1:BASE_PARITY:4cb52e5756f97522"} 09:36:41 [09:36:41.194] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4cb52e5756f97522 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:4cb52e5756f97522"} 09:36:41 [09:36:41.195] INFO: prover-client:proving-broker Proving Broker started 09:36:41 [09:36:41.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a6922ead055048d epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a6922ead055048d"} 09:36:41 [09:36:41.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a25d4aab5fb2d5e9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a25d4aab5fb2d5e9"} 09:36:42 [09:36:42.196] INFO: prover-client:proving-broker Proving Broker started 09:36:42 [09:36:42.196] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5a6922ead055048d settled=false {"provingJobId":"1:BASE_PARITY:5a6922ead055048d","status":"pending"} 09:36:42 [09:36:42.197] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a25d4aab5fb2d5e9 settled=false {"provingJobId":"1:BASE_PARITY:a25d4aab5fb2d5e9","status":"pending"} 09:36:42 [09:36:42.197] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5a6922ead055048d type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:5a6922ead055048d"} 09:36:42 [09:36:42.198] INFO: prover-client:proving-broker Proving Broker started 09:36:42 [09:36:42.199] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f455ceec7017c8bb epochNumber=1 {"provingJobId":"1:BASE_PARITY:f455ceec7017c8bb"} 09:36:42 [09:36:42.199] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:767a2be8962eb7d9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:767a2be8962eb7d9"} 09:36:43 [09:36:43.200] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.201] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f455ceec7017c8bb settled=false {"provingJobId":"1:BASE_PARITY:f455ceec7017c8bb","status":"pending"} 09:36:43 [09:36:43.201] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:767a2be8962eb7d9 settled=false {"provingJobId":"1:BASE_PARITY:767a2be8962eb7d9","status":"pending"} 09:36:43 [09:36:43.201] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f455ceec7017c8bb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f455ceec7017c8bb"} 09:36:43 [09:36:43.202] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f455ceec7017c8bb type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f455ceec7017c8bb"} 09:36:43 [09:36:43.203] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a93746543b639b48 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a93746543b639b48"} 09:36:43 [09:36:43.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2283c5c3fdde367d epochNumber=2 {"provingJobId":"1:BASE_PARITY:2283c5c3fdde367d"} 09:36:43 [09:36:43.204] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a93746543b639b48 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a93746543b639b48"} 09:36:43 [09:36:43.204] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2283c5c3fdde367d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2283c5c3fdde367d"} 09:36:43 [09:36:43.205] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.206] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:592e106d95cccaac epochNumber=1 {"provingJobId":"1:BASE_PARITY:592e106d95cccaac"} 09:36:43 [09:36:43.206] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:357dd2679e493e6c epochNumber=2 {"provingJobId":"1:BASE_PARITY:357dd2679e493e6c"} 09:36:43 [09:36:43.206] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:592e106d95cccaac type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:592e106d95cccaac"} 09:36:43 [09:36:43.206] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:592e106d95cccaac type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:592e106d95cccaac"} 09:36:43 [09:36:43.207] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:357dd2679e493e6c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:357dd2679e493e6c"} 09:36:43 [09:36:43.208] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.208] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1d016b36361fb0d6 err=test error {"provingJoId":"1:BASE_PARITY:1d016b36361fb0d6"} 09:36:43 [09:36:43.209] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.209] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8d6ef06585bd6d18 not found {"provingJobId":"1:BASE_PARITY:8d6ef06585bd6d18"} 09:36:43 [09:36:43.210] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c9c9cb415abd1046 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c9c9cb415abd1046"} 09:36:43 [09:36:43.211] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.212] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:41fc6e96b44db669 epochNumber=1 {"provingJobId":"1:BASE_PARITY:41fc6e96b44db669"} 09:36:43 [09:36:43.313] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:41fc6e96b44db669 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:41fc6e96b44db669"} 09:36:43 [09:36:43.320] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.321] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a32939b850fa7266 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a32939b850fa7266"} 09:36:43 [09:36:43.421] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a32939b850fa7266 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:a32939b850fa7266"} 09:36:43 [09:36:43.427] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9d0e84863e8a5686 epochNumber=10 {"provingJobId":"1:BASE_PARITY:9d0e84863e8a5686"} 09:36:43 [09:36:43.447] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:43 [09:36:43.553] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.554] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e693a7653d35d453 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e693a7653d35d453"} 09:36:43 [09:36:43.705] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e693a7653d35d453 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:e693a7653d35d453"} 09:36:43 [09:36:43.756] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.757] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a4172ea1b374c92f epochNumber=1 {"provingJobId":"1:BASE_PARITY:a4172ea1b374c92f"} 09:36:43 [09:36:43.757] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a4172ea1b374c92f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:a4172ea1b374c92f"} 09:36:43 [09:36:43.758] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.759] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:415a509ded798d61 epochNumber=1 {"provingJobId":"1:BASE_PARITY:415a509ded798d61"} 09:36:43 [09:36:43.759] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:415a509ded798d61 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:415a509ded798d61"} 09:36:43 [09:36:43.760] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:415a509ded798d61 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:415a509ded798d61"} 09:36:43 [09:36:43.760] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.761] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4fda06e12f14a4f4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4fda06e12f14a4f4"} 09:36:43 [09:36:43.761] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4fda06e12f14a4f4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4fda06e12f14a4f4"} 09:36:43 [09:36:43.762] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.762] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8bdfcc7f30d26fdb epochNumber=1 {"provingJobId":"1:BASE_PARITY:8bdfcc7f30d26fdb"} 09:36:43 [09:36:43.763] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8bdfcc7f30d26fdb type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:8bdfcc7f30d26fdb"} 09:36:43 [09:36:43.763] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6427834b26f1afc7 epochNumber=3 {"provingJobId":"1:BASE_PARITY:6427834b26f1afc7"} 09:36:43 [09:36:43.787] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:43 [09:36:43.789] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8bdfcc7f30d26fdb err=test error {"provingJoId":"1:BASE_PARITY:8bdfcc7f30d26fdb"} 09:36:43 [09:36:43.790] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.790] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d405eb79d59ce019 settled=false {"provingJobId":"1:BASE_PARITY:d405eb79d59ce019","status":"pending"} 09:36:43 [09:36:43.790] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e07bbf6f3ca7685a settled=false {"provingJobId":"1:BASE_PARITY:e07bbf6f3ca7685a","status":"pending"} 09:36:43 [09:36:43.792] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.792] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9dc3b589b4f1c553 settled=true {"provingJobId":"1:BASE_PARITY:9dc3b589b4f1c553","status":"fulfilled"} 09:36:43 [09:36:43.792] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8e020929fb6e722e settled=true {"provingJobId":"2:BASE_PARITY:8e020929fb6e722e","status":"fulfilled"} 09:36:43 [09:36:43.793] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.793] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3bb85822f860b70f settled=true {"provingJobId":"1:BASE_PARITY:3bb85822f860b70f","status":"fulfilled"} 09:36:43 [09:36:43.794] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:40bc0ecd31b8fd70 settled=false {"provingJobId":"1:BASE_PARITY:40bc0ecd31b8fd70","status":"pending"} 09:36:43 [09:36:43.794] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.795] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7902e5280666c35d epochNumber=1 {"provingJobId":"1:BASE_PARITY:7902e5280666c35d"} 09:36:43 [09:36:43.797] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.797] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5745603f7b84bf27 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5745603f7b84bf27"} 09:36:43 [09:36:43.798] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:5745603f7b84bf27: Error: db error: Error: db error 09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:5745603f7b84bf27"} 09:36:43 [09:36:43.799] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.799] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:26817c9d9412d0b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:26817c9d9412d0b5"} 09:36:43 [09:36:43.800] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:26817c9d9412d0b5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:26817c9d9412d0b5"} 09:36:43 [09:36:43.800] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:26817c9d9412d0b5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:26817c9d9412d0b5"} 09:36:43 [09:36:43.801] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.801] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d8c38ad59a819bb epochNumber=1 {"provingJobId":"1:BASE_PARITY:6d8c38ad59a819bb"} 09:36:43 [09:36:43.801] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6d8c38ad59a819bb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6d8c38ad59a819bb"} 09:36:43 [09:36:43.801] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6d8c38ad59a819bb type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6d8c38ad59a819bb"} 09:36:43 [09:36:43.802] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:6d8c38ad59a819bb: Error: db error 09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:6d8c38ad59a819bb"} 09:36:43 [09:36:43.803] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.803] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7d1726d9064c9499 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7d1726d9064c9499"} 09:36:43 [09:36:43.803] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7d1726d9064c9499 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7d1726d9064c9499"} 09:36:43 [09:36:43.804] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7d1726d9064c9499 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:7d1726d9064c9499"} 09:36:43 [09:36:43.804] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.805] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d2c945ae0300eb8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6d2c945ae0300eb8"} 09:36:43 [09:36:43.805] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6d2c945ae0300eb8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6d2c945ae0300eb8"} 09:36:43 [09:36:43.805] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6d2c945ae0300eb8 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:6d2c945ae0300eb8"} 09:36:43 [09:36:43.806] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:6d2c945ae0300eb8 jobErr=test error: Error: db error 09:36:43 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:6d2c945ae0300eb8"} 09:36:43 [09:36:43.807] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.807] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34b459a4a5be3779 not found {"provingJobId":"1:BASE_PARITY:34b459a4a5be3779"} 09:36:43 [09:36:43.808] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.809] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c8eed5e1fe0df191 err=test error {"provingJoId":"1:BASE_PARITY:c8eed5e1fe0df191"} 09:36:43 [09:36:43.810] INFO: prover-client:proving-broker Proving Broker started 09:36:43 [09:36:43.820] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:820261fcf97e4477 epochNumber=1 {"provingJobId":"1:BASE_PARITY:820261fcf97e4477"} 09:36:43 [09:36:43.820] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:820261fcf97e4477 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:820261fcf97e4477"} 09:36:43 [09:36:43.820] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:820261fcf97e4477 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:820261fcf97e4477"} 09:36:43 [09:36:43.831] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:49c0b6bd69702d31 epochNumber=2 {"provingJobId":"2:BASE_PARITY:49c0b6bd69702d31"} 09:36:43 [09:36:43.831] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:49c0b6bd69702d31 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:49c0b6bd69702d31"} 09:36:43 [09:36:43.832] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:49c0b6bd69702d31 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:49c0b6bd69702d31"} 09:36:43 [09:36:43.893] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:81a9ba48d651a174 epochNumber=3 {"provingJobId":"3:BASE_PARITY:81a9ba48d651a174"} 09:36:43 [09:36:43.911] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:43 [09:36:43.955] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:d1fe450c677637c2 epochNumber=4 {"provingJobId":"4:BASE_PARITY:d1fe450c677637c2"} 09:36:43 [09:36:43.962] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:44 [09:36:44.016] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:aeeb85cda0c6fd0a epochNumber=5 {"provingJobId":"5:BASE_PARITY:aeeb85cda0c6fd0a"} 09:36:44 [09:36:44.038] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:44 [09:36:44.069] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.069] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1aaa084f3cfc9655 epochNumber=42 {"provingJobId":"1:BASE_PARITY:1aaa084f3cfc9655"} 09:36:44 [09:36:44.070] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testeSERep/42 with map size 1073741824 09:36:44 [09:36:44.072] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.094] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:368b1bb8df57ef14 epochNumber=1 {"provingJobId":"1:BASE_PARITY:368b1bb8df57ef14"} 09:36:44 [09:36:44.097] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.098] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe3ec2cb8374197a epochNumber=1 {"provingJobId":"1:BASE_PARITY:fe3ec2cb8374197a"} 09:36:44 [09:36:44.098] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHmYCPG/1 with map size 1073741824 09:36:44 [09:36:44.099] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6ec959715df2f905 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6ec959715df2f905"} 09:36:44 [09:36:44.125] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.126] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c587b9743916e07f epochNumber=1 {"provingJobId":"1:BASE_PARITY:c587b9743916e07f"} 09:36:44 [09:36:44.126] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnslXSN/1 with map size 1073741824 09:36:44 [09:36:44.128] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.147] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c587b9743916e07f epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c587b9743916e07f"} 09:36:44 [09:36:44.149] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.150] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dda1c40b8d52484b epochNumber=1 {"provingJobId":"1:BASE_PARITY:dda1c40b8d52484b"} 09:36:44 [09:36:44.150] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaRwPws/1 with map size 1073741824 09:36:44 [09:36:44.151] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.167] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dda1c40b8d52484b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:dda1c40b8d52484b"} 09:36:44 [09:36:44.167] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:dda1c40b8d52484b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:dda1c40b8d52484b"} 09:36:44 [09:36:44.172] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dda1c40b8d52484b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:dda1c40b8d52484b"} 09:36:44 [09:36:44.174] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.175] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2546435e71d82876 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2546435e71d82876"} 09:36:44 [09:36:44.175] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNC0Ia6/1 with map size 1073741824 09:36:44 [09:36:44.178] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.194] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.196] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.197] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2e03d029e32c68e epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2e03d029e32c68e"} 09:36:44 [09:36:44.197] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testo2R18r/1 with map size 1073741824 09:36:44 [09:36:44.198] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.213] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c2e03d029e32c68e {"provingJobId":"1:BASE_PARITY:c2e03d029e32c68e"} 09:36:44 [09:36:44.214] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c2e03d029e32c68e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c2e03d029e32c68e"} 09:36:44 [09:36:44.214] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c2e03d029e32c68e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c2e03d029e32c68e"} 09:36:44 [09:36:44.221] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.222] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:78ead915d3aeec4e epochNumber=1 {"provingJobId":"1:BASE_PARITY:78ead915d3aeec4e"} 09:36:44 [09:36:44.222] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxamhMX/1 with map size 1073741824 09:36:44 [09:36:44.223] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.237] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:78ead915d3aeec4e {"provingJobId":"1:BASE_PARITY:78ead915d3aeec4e"} 09:36:44 [09:36:44.238] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:78ead915d3aeec4e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:78ead915d3aeec4e"} 09:36:44 [09:36:44.245] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.246] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:24f5481356a43703 epochNumber=1 {"provingJobId":"1:BASE_PARITY:24f5481356a43703"} 09:36:44 [09:36:44.248] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfKkySc/1 with map size 1073741824 09:36:44 [09:36:44.249] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.263] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:24f5481356a43703 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:24f5481356a43703"} 09:36:44 [09:36:44.263] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:24f5481356a43703 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:24f5481356a43703"} 09:36:44 [09:36:44.270] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b25314e2b68b951f epochNumber=1 {"provingJobId":"1:BASE_PARITY:b25314e2b68b951f"} 09:36:44 [09:36:44.271] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvoPO2Z/1 with map size 1073741824 09:36:44 [09:36:44.273] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.293] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b25314e2b68b951f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b25314e2b68b951f"} 09:36:44 [09:36:44.294] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b25314e2b68b951f type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b25314e2b68b951f"} 09:36:44 [09:36:44.301] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.302] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4fd92c913a98a957 epochNumber=0 {"provingJobId":"1:BASE_PARITY:4fd92c913a98a957"} 09:36:44 [09:36:44.303] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testKEFwBM/0 with map size 1073741824 09:36:44 [09:36:44.305] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.328] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4fd92c913a98a957 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:4fd92c913a98a957"} 09:36:44 [09:36:44.328] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3aecac323ca889fc epochNumber=0 {"provingJobId":"1:BASE_PARITY:3aecac323ca889fc"} 09:36:44 [09:36:44.329] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4fd92c913a98a957 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:4fd92c913a98a957"} 09:36:44 [09:36:44.329] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6abf04a192a9ac1c epochNumber=0 {"provingJobId":"1:BASE_PARITY:6abf04a192a9ac1c"} 09:36:44 [09:36:44.342] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.343] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.343] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8016d4d81c2f878 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a8016d4d81c2f878"} 09:36:44 [09:36:44.345] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtpL9Bf/2 with map size 1073741824 09:36:44 [09:36:44.346] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.366] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:947b4df231202793 epochNumber=1 {"provingJobId":"1:BASE_PARITY:947b4df231202793"} 09:36:44 [09:36:44.366] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtpL9Bf/1 with map size 1073741824 09:36:44 [09:36:44.369] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.387] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.388] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a84e8739d642a13b epochNumber=1 {"provingJobId":"1:BASE_PARITY:a84e8739d642a13b"} 09:36:44 [09:36:44.388] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFUZSZ4/1 with map size 1073741824 09:36:44 [09:36:44.389] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.408] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.409] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b06d901e2a96e2f0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b06d901e2a96e2f0"} 09:36:44 [09:36:44.410] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpUnpTK/1 with map size 1073741824 09:36:44 [09:36:44.411] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.428] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:633abe7e0e77b23e epochNumber=1 {"provingJobId":"1:BASE_PARITY:633abe7e0e77b23e"} 09:36:44 [09:36:44.435] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e25bdf1de829da53 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e25bdf1de829da53"} 09:36:44 [09:36:44.435] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpUnpTK/2 with map size 1073741824 09:36:44 [09:36:44.436] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.454] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7761256e72857222 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7761256e72857222"} 09:36:44 [09:36:44.461] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.462] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6e1790bd64e60ab3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6e1790bd64e60ab3"} 09:36:44 [09:36:44.462] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testABNvgM/1 with map size 1073741824 09:36:44 [09:36:44.463] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.478] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:42d6751cfd83a627 epochNumber=1 {"provingJobId":"1:BASE_PARITY:42d6751cfd83a627"} 09:36:44 [09:36:44.483] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf1c6d2ce63a1d20 epochNumber=2 {"provingJobId":"1:BASE_PARITY:cf1c6d2ce63a1d20"} 09:36:44 [09:36:44.483] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testABNvgM/2 with map size 1073741824 09:36:44 [09:36:44.484] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.499] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0cb5180181272b3f epochNumber=1 {"provingJobId":"1:BASE_PARITY:0cb5180181272b3f"} 09:36:44 [09:36:44.506] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.506] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bb6ab76625261747 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bb6ab76625261747"} 09:36:44 [09:36:44.508] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGFgyaA/1 with map size 1073741824 09:36:44 [09:36:44.509] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.523] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afb420eb87ea3e40 epochNumber=1 {"provingJobId":"1:BASE_PARITY:afb420eb87ea3e40"} 09:36:44 [09:36:44.528] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:af6798d301f61b73 epochNumber=2 {"provingJobId":"1:BASE_PARITY:af6798d301f61b73"} 09:36:44 [09:36:44.528] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testGFgyaA/2 with map size 1073741824 09:36:44 [09:36:44.530] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.544] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d18df9d298e15950 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d18df9d298e15950"} 09:36:44 [09:36:44.551] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.551] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7287f58df24fe6f3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7287f58df24fe6f3"} 09:36:44 [09:36:44.552] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test63hbIE/1 with map size 1073741824 09:36:44 [09:36:44.553] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.568] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5bab0076796ba93 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5bab0076796ba93"} 09:36:44 [09:36:44.573] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7287f58df24fe6f3 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7287f58df24fe6f3"} 09:36:44 [09:36:44.580] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.581] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aceaeb66ad696059 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aceaeb66ad696059"} 09:36:44 [09:36:44.581] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testj7pGwh/1 with map size 1073741824 09:36:44 [09:36:44.584] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.599] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc5545f5764ac5c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc5545f5764ac5c4"} 09:36:44 [09:36:44.604] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:aceaeb66ad696059 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:aceaeb66ad696059"} 09:36:44 [09:36:44.612] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.612] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d0623f413b8c91e epochNumber=1 {"provingJobId":"1:BASE_PARITY:6d0623f413b8c91e"} 09:36:44 [09:36:44.613] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbaVeK9/1 with map size 1073741824 09:36:44 [09:36:44.615] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.633] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1fe100e1fb274ecc epochNumber=1 {"provingJobId":"1:BASE_PARITY:1fe100e1fb274ecc"} 09:36:44 [09:36:44.642] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6d0623f413b8c91e type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:6d0623f413b8c91e"} 09:36:44 [09:36:44.645] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.645] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4fc90f44caeacdc epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4fc90f44caeacdc"} 09:36:44 [09:36:44.646] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyn4vYG/1 with map size 1073741824 09:36:44 [09:36:44.647] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.667] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c4fc90f44caeacdc {"provingJobId":"1:BASE_PARITY:c4fc90f44caeacdc"} 09:36:44 [09:36:44.668] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c4fc90f44caeacdc type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c4fc90f44caeacdc"} 09:36:44 [09:36:44.676] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:355a8159d495ef80 epochNumber=1 {"provingJobId":"1:BASE_PARITY:355a8159d495ef80"} 09:36:44 [09:36:44.683] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c4fc90f44caeacdc has already been completed {"provingJobId":"1:BASE_PARITY:c4fc90f44caeacdc"} 09:36:44 [09:36:44.686] INFO: prover-client:proving-broker Proving Broker started 09:36:44 [09:36:44.687] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9d2e6a41e680aec5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9d2e6a41e680aec5"} 09:36:44 [09:36:44.688] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtn8yPg/1 with map size 1073741824 09:36:44 [09:36:44.689] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:44 [09:36:44.707] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:485f0d475bfe1f35 epochNumber=2 {"provingJobId":"1:BASE_PARITY:485f0d475bfe1f35"} 09:36:44 [09:36:44.708] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtn8yPg/2 with map size 1073741824 09:36:44 [09:36:44.711] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:45 [09:36:45.780] INFO: prover-client:proving-broker Proving Broker started 09:36:45 [09:36:45.784] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9d2e6a41e680aec5 settled=false {"provingJobId":"1:BASE_PARITY:9d2e6a41e680aec5","status":"pending"} 09:36:45 [09:36:45.785] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:485f0d475bfe1f35 settled=false {"provingJobId":"1:BASE_PARITY:485f0d475bfe1f35","status":"pending"} 09:36:45 [09:36:45.786] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:9d2e6a41e680aec5 type=BASE_PARITY startedAt=1749807404730 older agent has taken job {"provingJobId":"1:BASE_PARITY:9d2e6a41e680aec5"} 09:36:45 [09:36:45.787] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9d2e6a41e680aec5 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:9d2e6a41e680aec5"} 09:36:45 [09:36:45.789] INFO: prover-client:proving-broker Proving Broker started 09:36:45 [09:36:45.789] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:703cc7f5519dd2e9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:703cc7f5519dd2e9"} 09:36:45 [09:36:45.790] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVYfHse/1 with map size 1073741824 09:36:45 [09:36:45.791] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:45 [09:36:45.806] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2143dd5d4b9e377d epochNumber=2 {"provingJobId":"1:BASE_PARITY:2143dd5d4b9e377d"} 09:36:45 [09:36:45.806] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVYfHse/2 with map size 1073741824 09:36:45 [09:36:45.809] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:46 [09:36:46.824] INFO: prover-client:proving-broker Proving Broker started 09:36:46 [09:36:46.826] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:703cc7f5519dd2e9 settled=false {"provingJobId":"1:BASE_PARITY:703cc7f5519dd2e9","status":"pending"} 09:36:46 [09:36:46.827] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2143dd5d4b9e377d settled=false {"provingJobId":"1:BASE_PARITY:2143dd5d4b9e377d","status":"pending"} 09:36:46 [09:36:46.828] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:703cc7f5519dd2e9 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:703cc7f5519dd2e9"} 09:36:46 [09:36:46.830] INFO: prover-client:proving-broker Proving Broker started 09:36:46 [09:36:46.831] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51fae0623378c618 epochNumber=1 {"provingJobId":"1:BASE_PARITY:51fae0623378c618"} 09:36:46 [09:36:46.832] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjZfDzQ/1 with map size 1073741824 09:36:46 [09:36:46.833] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:46 [09:36:46.847] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d8e177396ab233f7 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d8e177396ab233f7"} 09:36:46 [09:36:46.848] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testjZfDzQ/2 with map size 1073741824 09:36:46 [09:36:46.849] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.863] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.865] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:51fae0623378c618 settled=false {"provingJobId":"1:BASE_PARITY:51fae0623378c618","status":"pending"} 09:36:47 [09:36:47.867] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d8e177396ab233f7 settled=false {"provingJobId":"1:BASE_PARITY:d8e177396ab233f7","status":"pending"} 09:36:47 [09:36:47.868] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:51fae0623378c618 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:51fae0623378c618"} 09:36:47 [09:36:47.868] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:51fae0623378c618 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:51fae0623378c618"} 09:36:47 [09:36:47.877] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.878] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f07bf1d43a90abb0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f07bf1d43a90abb0"} 09:36:47 [09:36:47.879] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyiEmp6/1 with map size 1073741824 09:36:47 [09:36:47.880] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.896] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b7a80d033811c73 epochNumber=2 {"provingJobId":"1:BASE_PARITY:6b7a80d033811c73"} 09:36:47 [09:36:47.896] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyiEmp6/2 with map size 1073741824 09:36:47 [09:36:47.898] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.912] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f07bf1d43a90abb0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f07bf1d43a90abb0"} 09:36:47 [09:36:47.918] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6b7a80d033811c73 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:6b7a80d033811c73"} 09:36:47 [09:36:47.926] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.926] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:006a5e63278d8c67 epochNumber=1 {"provingJobId":"1:BASE_PARITY:006a5e63278d8c67"} 09:36:47 [09:36:47.927] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFfPnpv/1 with map size 1073741824 09:36:47 [09:36:47.928] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.943] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b0e3ad285cb230d epochNumber=2 {"provingJobId":"1:BASE_PARITY:9b0e3ad285cb230d"} 09:36:47 [09:36:47.943] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testFfPnpv/2 with map size 1073741824 09:36:47 [09:36:47.945] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.958] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:006a5e63278d8c67 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:006a5e63278d8c67"} 09:36:47 [09:36:47.959] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:006a5e63278d8c67 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:006a5e63278d8c67"} 09:36:47 [09:36:47.964] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9b0e3ad285cb230d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:9b0e3ad285cb230d"} 09:36:47 [09:36:47.971] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.972] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:4e8aef33579c61ba err=test error {"provingJoId":"1:BASE_PARITY:4e8aef33579c61ba"} 09:36:47 [09:36:47.974] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.975] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5ba9c28178d92e16 not found {"provingJobId":"1:BASE_PARITY:5ba9c28178d92e16"} 09:36:47 [09:36:47.978] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.979] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4d02bd09a4aa4016 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4d02bd09a4aa4016"} 09:36:47 [09:36:47.979] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testusHlXA/1 with map size 1073741824 09:36:47 [09:36:47.980] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:47 [09:36:47.997] INFO: prover-client:proving-broker Proving Broker started 09:36:47 [09:36:47.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5b44aed98a6a25fe epochNumber=1 {"provingJobId":"1:BASE_PARITY:5b44aed98a6a25fe"} 09:36:47 [09:36:47.998] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6LoXvo/1 with map size 1073741824 09:36:47 [09:36:47.999] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.122] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5b44aed98a6a25fe timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:5b44aed98a6a25fe"} 09:36:48 [09:36:48.128] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a23680c6da7edc87 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a23680c6da7edc87"} 09:36:48 [09:36:48.129] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFbsubm/1 with map size 1073741824 09:36:48 [09:36:48.132] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.255] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a23680c6da7edc87 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:a23680c6da7edc87"} 09:36:48 [09:36:48.258] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c1d377fa4d54dd19 epochNumber=10 {"provingJobId":"1:BASE_PARITY:c1d377fa4d54dd19"} 09:36:48 [09:36:48.258] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testFbsubm/10 with map size 1073741824 09:36:48 [09:36:48.260] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.280] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:48 [09:36:48.280] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 09:36:48 [09:36:48.402] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.403] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7d14540429f8bc2a epochNumber=1 {"provingJobId":"1:BASE_PARITY:7d14540429f8bc2a"} 09:36:48 [09:36:48.404] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBsWIoY/1 with map size 1073741824 09:36:48 [09:36:48.405] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.579] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7d14540429f8bc2a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:7d14540429f8bc2a"} 09:36:48 [09:36:48.623] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.623] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a444cc87e01100f epochNumber=1 {"provingJobId":"1:BASE_PARITY:6a444cc87e01100f"} 09:36:48 [09:36:48.624] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiom1YR/1 with map size 1073741824 09:36:48 [09:36:48.625] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.640] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6a444cc87e01100f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:6a444cc87e01100f"} 09:36:48 [09:36:48.643] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.643] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:01a2774980c0f255 epochNumber=1 {"provingJobId":"1:BASE_PARITY:01a2774980c0f255"} 09:36:48 [09:36:48.644] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWGNYbW/1 with map size 1073741824 09:36:48 [09:36:48.645] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.659] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:01a2774980c0f255 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:01a2774980c0f255"} 09:36:48 [09:36:48.660] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:01a2774980c0f255 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:01a2774980c0f255"} 09:36:48 [09:36:48.667] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4e9eced19dd61aff epochNumber=1 {"provingJobId":"1:BASE_PARITY:4e9eced19dd61aff"} 09:36:48 [09:36:48.668] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test29w64Y/1 with map size 1073741824 09:36:48 [09:36:48.669] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.683] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4e9eced19dd61aff type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4e9eced19dd61aff"} 09:36:48 [09:36:48.690] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf17f1ee8bb27a58 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bf17f1ee8bb27a58"} 09:36:48 [09:36:48.692] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4GAtBj/1 with map size 1073741824 09:36:48 [09:36:48.693] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.706] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bf17f1ee8bb27a58 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:bf17f1ee8bb27a58"} 09:36:48 [09:36:48.707] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11466b1e060cf3f7 epochNumber=3 {"provingJobId":"1:BASE_PARITY:11466b1e060cf3f7"} 09:36:48 [09:36:48.708] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test4GAtBj/3 with map size 1073741824 09:36:48 [09:36:48.709] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.740] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:48 [09:36:48.740] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 09:36:48 [09:36:48.747] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:bf17f1ee8bb27a58 err=test error {"provingJoId":"1:BASE_PARITY:bf17f1ee8bb27a58"} 09:36:48 [09:36:48.750] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTfyPCD/1 with map size 1073741824 09:36:48 [09:36:48.751] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.768] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTfyPCD/2 with map size 1073741824 09:36:48 [09:36:48.769] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.784] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.785] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8089748e00c70eda settled=false {"provingJobId":"1:BASE_PARITY:8089748e00c70eda","status":"pending"} 09:36:48 [09:36:48.786] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:04b8108f08b15105 settled=false {"provingJobId":"1:BASE_PARITY:04b8108f08b15105","status":"pending"} 09:36:48 [09:36:48.791] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYoCSvU/1 with map size 1073741824 09:36:48 [09:36:48.793] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.807] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testYoCSvU/2 with map size 1073741824 09:36:48 [09:36:48.808] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.834] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.835] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3e27027df3c57b21 settled=true {"provingJobId":"1:BASE_PARITY:3e27027df3c57b21","status":"fulfilled"} 09:36:48 [09:36:48.837] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:e721ae2413952497 settled=true {"provingJobId":"2:BASE_PARITY:e721ae2413952497","status":"fulfilled"} 09:36:48 [09:36:48.841] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcckLja/1 with map size 1073741824 09:36:48 [09:36:48.843] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.863] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testcckLja/2 with map size 1073741824 09:36:48 [09:36:48.864] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.879] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.880] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0c40499c728bb334 settled=true {"provingJobId":"1:BASE_PARITY:0c40499c728bb334","status":"fulfilled"} 09:36:48 [09:36:48.883] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5bcb32e95d02d6b1 settled=false {"provingJobId":"1:BASE_PARITY:5bcb32e95d02d6b1","status":"pending"} 09:36:48 [09:36:48.886] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.887] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:431a0c82984a4d24 epochNumber=1 {"provingJobId":"1:BASE_PARITY:431a0c82984a4d24"} 09:36:48 [09:36:48.887] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5WfQO7/1 with map size 1073741824 09:36:48 [09:36:48.888] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.907] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.907] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1853a6f4fd8c46fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:1853a6f4fd8c46fb"} 09:36:48 [09:36:48.908] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:1853a6f4fd8c46fb: Error: db error: Error: db error 09:36:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:1853a6f4fd8c46fb"} 09:36:48 [09:36:48.911] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.911] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4452852e92b97de6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4452852e92b97de6"} 09:36:48 [09:36:48.912] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNoOGNb/1 with map size 1073741824 09:36:48 [09:36:48.913] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.930] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4452852e92b97de6 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4452852e92b97de6"} 09:36:48 [09:36:48.930] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4452852e92b97de6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4452852e92b97de6"} 09:36:48 [09:36:48.939] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.940] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:96f6852adac51eb0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:96f6852adac51eb0"} 09:36:48 [09:36:48.940] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7XwTtI/1 with map size 1073741824 09:36:48 [09:36:48.941] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.955] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:96f6852adac51eb0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:96f6852adac51eb0"} 09:36:48 [09:36:48.956] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:96f6852adac51eb0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:96f6852adac51eb0"} 09:36:48 [09:36:48.956] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:96f6852adac51eb0: Error: db error 09:36:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:96f6852adac51eb0"} 09:36:48 [09:36:48.959] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.960] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cefd853c22563ae7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cefd853c22563ae7"} 09:36:48 [09:36:48.961] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0QEvG8/1 with map size 1073741824 09:36:48 [09:36:48.962] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:48 [09:36:48.976] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cefd853c22563ae7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:cefd853c22563ae7"} 09:36:48 [09:36:48.977] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cefd853c22563ae7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:cefd853c22563ae7"} 09:36:48 [09:36:48.984] INFO: prover-client:proving-broker Proving Broker started 09:36:48 [09:36:48.985] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:818789f4eadf22d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:818789f4eadf22d7"} 09:36:48 [09:36:48.986] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD818wl/1 with map size 1073741824 09:36:48 [09:36:48.987] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.001] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:818789f4eadf22d7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:818789f4eadf22d7"} 09:36:49 [09:36:49.002] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:818789f4eadf22d7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:818789f4eadf22d7"} 09:36:49 [09:36:49.002] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:818789f4eadf22d7 jobErr=test error: Error: db error 09:36:49 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:818789f4eadf22d7"} 09:36:49 [09:36:49.006] INFO: prover-client:proving-broker Proving Broker started 09:36:49 [09:36:49.007] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c7067875f8f6df41 not found {"provingJobId":"1:BASE_PARITY:c7067875f8f6df41"} 09:36:49 [09:36:49.009] INFO: prover-client:proving-broker Proving Broker started 09:36:49 [09:36:49.010] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:cf16175598aad96e err=test error {"provingJoId":"1:BASE_PARITY:cf16175598aad96e"} 09:36:49 [09:36:49.013] INFO: prover-client:proving-broker Proving Broker started 09:36:49 [09:36:49.023] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:644a4f8f5b0dbef7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:644a4f8f5b0dbef7"} 09:36:49 [09:36:49.024] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3JeeBL/1 with map size 1073741824 09:36:49 [09:36:49.025] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.040] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:644a4f8f5b0dbef7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:644a4f8f5b0dbef7"} 09:36:49 [09:36:49.040] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:644a4f8f5b0dbef7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:644a4f8f5b0dbef7"} 09:36:49 [09:36:49.056] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:f7874dee40767952 epochNumber=2 {"provingJobId":"2:BASE_PARITY:f7874dee40767952"} 09:36:49 [09:36:49.057] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3JeeBL/2 with map size 1073741824 09:36:49 [09:36:49.058] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.074] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:f7874dee40767952 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:f7874dee40767952"} 09:36:49 [09:36:49.074] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:f7874dee40767952 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:f7874dee40767952"} 09:36:49 [09:36:49.140] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:543944842af86803 epochNumber=3 {"provingJobId":"3:BASE_PARITY:543944842af86803"} 09:36:49 [09:36:49.141] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test3JeeBL/3 with map size 1073741824 09:36:49 [09:36:49.142] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.165] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:49 [09:36:49.165] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 09:36:49 [09:36:49.218] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:d799afde6904ea32 epochNumber=4 {"provingJobId":"4:BASE_PARITY:d799afde6904ea32"} 09:36:49 [09:36:49.218] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test3JeeBL/4 with map size 1073741824 09:36:49 [09:36:49.220] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.242] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:49 [09:36:49.243] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 09:36:49 [09:36:49.295] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:4078fc976d04ee87 epochNumber=5 {"provingJobId":"5:BASE_PARITY:4078fc976d04ee87"} 09:36:49 [09:36:49.296] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test3JeeBL/5 with map size 1073741824 09:36:49 [09:36:49.297] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 09:36:49 [09:36:49.321] INFO: prover-client:proving-broker Cleaned up jobs=1 09:36:49 [09:36:49.321] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 09:36:49 PASS src/proving_broker/proving_broker.test.ts (10.951 s) 09:36:49 ProvingBroker 09:36:49 Producer API 09:36:49 ✓ refuses stale jobs (33 ms) 09:36:49 ✓ enqueues jobs (4 ms) 09:36:49 ✓ ignores duplicate jobs (4 ms) 09:36:49 ✓ reports correct status when enqueuing repeat jobs (4 ms) 09:36:49 ✓ throws an error in case of duplicate job IDs (22 ms) 09:36:49 ✓ returns not-found status for non-existing jobs (2 ms) 09:36:49 ✓ cancels jobs in queue (2 ms) 09:36:49 ✓ cancels jobs in-progress (1 ms) 09:36:49 ✓ returns job result if successful (2 ms) 09:36:49 ✓ returns job error if failed (2 ms) 09:36:49 ✓ correctly returns job status for concurrent writes (2 ms) 09:36:49 ✓ refuses stale jobs (29 ms) 09:36:49 ✓ enqueues jobs (28 ms) 09:36:49 ✓ ignores duplicate jobs (24 ms) 09:36:49 ✓ reports correct status when enqueuing repeat jobs (25 ms) 09:36:49 ✓ throws an error in case of duplicate job IDs (20 ms) 09:36:49 ✓ returns not-found status for non-existing jobs (2 ms) 09:36:49 ✓ cancels jobs in queue (25 ms) 09:36:49 ✓ cancels jobs in-progress (24 ms) 09:36:49 ✓ returns job result if successful (25 ms) 09:36:49 ✓ returns job error if failed (31 ms) 09:36:49 ✓ correctly returns job status for concurrent writes (41 ms) 09:36:49 Consumer API 09:36:49 ✓ returns undefined if no jobs are available (1 ms) 09:36:49 ✓ returns jobs in priority order (2 ms) 09:36:49 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 09:36:49 ✓ returns a job if it is in the allowList (2 ms) 09:36:49 ✓ returns the most important job if it is in the allowList (2 ms) 09:36:49 ✓ returns any job if filter is empty (2 ms) 09:36:49 ✓ returns a new job when reporting job success (2 ms) 09:36:49 ✓ returns a new job when reporting permanent error (2 ms) 09:36:49 ✓ returns a new job when reporting retry-able error (1 ms) 09:36:49 ✓ returns a new job when reporting progress if current one is cancelled (2 ms) 09:36:49 ✓ returns a new job if job is already in progress elsewhere (1054 ms) 09:36:49 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 09:36:49 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 09:36:49 ✓ tracks job result if in progress (3 ms) 09:36:49 ✓ tracks job result even if job is in queue (2 ms) 09:36:49 ✓ ignores reported job error if unknown job (1 ms) 09:36:49 ✓ ignores job result if unknown job (1 ms) 09:36:49 ✓ returns undefined if no jobs are available (1 ms) 09:36:49 ✓ returns jobs in priority order (44 ms) 09:36:49 ✓ returns undefined if no jobs are available for the given allowList (20 ms) 09:36:49 ✓ returns a job if it is in the allowList (52 ms) 09:36:49 ✓ returns the most important job if it is in the allowList (45 ms) 09:36:49 ✓ returns any job if filter is empty (45 ms) 09:36:49 ✓ returns a new job when reporting job success (29 ms) 09:36:49 ✓ returns a new job when reporting permanent error (32 ms) 09:36:49 ✓ returns a new job when reporting retry-able error (33 ms) 09:36:49 ✓ returns a new job when reporting progress if current one is cancelled (41 ms) 09:36:49 ✓ returns a new job if job is already in progress elsewhere (1103 ms) 09:36:49 ✓ avoids sending the same job to a new agent after a restart (1041 ms) 09:36:49 ✓ avoids sending a completed job to a new agent after a restart (1047 ms) 09:36:49 ✓ tracks job result if in progress (48 ms) 09:36:49 ✓ tracks job result even if job is in queue (46 ms) 09:36:49 ✓ ignores reported job error if unknown job (3 ms) 09:36:49 ✓ ignores job result if unknown job (3 ms) 09:36:49 Timeouts 09:36:49 ✓ tracks in progress jobs (1 ms) 09:36:49 ✓ re-enqueues jobs that time out (109 ms) 09:36:49 ✓ cancel stale jobs that time out (233 ms) 09:36:49 ✓ keeps the jobs in progress while it is alive (202 ms) 09:36:49 ✓ tracks in progress jobs (19 ms) 09:36:49 ✓ re-enqueues jobs that time out (131 ms) 09:36:49 ✓ cancel stale jobs that time out (274 ms) 09:36:49 ✓ keeps the jobs in progress while it is alive (220 ms) 09:36:49 Retries 09:36:49 ✓ retries jobs (2 ms) 09:36:49 ✓ retries up to a maximum number of times (2 ms) 09:36:49 ✓ passing retry=false does not retry (2 ms) 09:36:49 ✓ does not retry if job is stale (27 ms) 09:36:49 ✓ retries jobs (19 ms) 09:36:49 ✓ retries up to a maximum number of times (23 ms) 09:36:49 ✓ passing retry=false does not retry (23 ms) 09:36:49 ✓ does not retry if job is stale (59 ms) 09:36:49 Database management 09:36:49 ✓ re-enqueues proof requests on start (2 ms) 09:36:49 ✓ restores proof results on start (2 ms) 09:36:49 ✓ only re-enqueues unfinished jobs (1 ms) 09:36:49 ✓ saves job when enqueued (2 ms) 09:36:49 ✓ does not retain job if database fails to save (1 ms) 09:36:49 ✓ saves job result (1 ms) 09:36:49 ✓ saves result even if database fails to save (2 ms) 09:36:49 ✓ saves job error (2 ms) 09:36:49 ✓ saves job error even if database fails to save (2 ms) 09:36:49 ✓ does not save job result if job is unknown (2 ms) 09:36:49 ✓ does not save job error if job is unknown (1 ms) 09:36:49 ✓ cleans up old jobs periodically (258 ms) 09:36:49 ✓ re-enqueues proof requests on start (39 ms) 09:36:49 ✓ restores proof results on start (50 ms) 09:36:49 ✓ only re-enqueues unfinished jobs (45 ms) 09:36:49 ✓ saves job when enqueued (20 ms) 09:36:49 ✓ does not retain job if database fails to save (3 ms) 09:36:49 ✓ saves job result (28 ms) 09:36:49 ✓ saves result even if database fails to save (20 ms) 09:36:49 ✓ saves job error (25 ms) 09:36:49 ✓ saves job error even if database fails to save (21 ms) 09:36:49 ✓ does not save job result if job is unknown (3 ms) 09:36:49 ✓ does not save job error if job is unknown (3 ms) 09:36:49 ✓ cleans up old jobs periodically (356 ms) 09:36:49 09:36:49 Test Suites: 1 passed, 1 total 09:36:49 Tests: 96 passed, 96 total 09:36:49 Snapshots: 0 total 09:36:49 Time: 11.018 s 09:36:49 Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i. 09:36:49 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?