Parent Log: http://ci.aztec-labs.com/844afe2f59201208 Command: 7d3a4ee7e143e35a yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/3b5835195491b53670250790b18c14d922718574 Env: REF_NAME=gh-readonly-queue/next/pr-15094-1542a83b8e029762e119474de1725d84a1aee3d2 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Tue Jun 17 10:24:05 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15094_amd64_x4-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 10:24:09 [10:24:09.974] INFO: prover-client:proving-broker Proving Broker started 10:24:09 [10:24:09.982] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8fbc7d0ecad946be epochNumber=42 {"provingJobId":"1:BASE_PARITY:8fbc7d0ecad946be"} 10:24:09 [10:24:09.986] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:5a1478ca42761588 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a1478ca42761588"} 10:24:10 [10:24:10.025] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.026] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4c325b6efb78d0e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4c325b6efb78d0e5"} 10:24:10 [10:24:10.028] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6401ef5e637dd2cf epochNumber=1 {"provingJobId":"1:BASE_PARITY:6401ef5e637dd2cf"} 10:24:10 [10:24:10.030] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.031] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0e8a18886057b127 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0e8a18886057b127"} 10:24:10 [10:24:10.033] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0e8a18886057b127 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:0e8a18886057b127"} 10:24:10 [10:24:10.035] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.036] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64bc7a479fc1b401 epochNumber=1 {"provingJobId":"1:BASE_PARITY:64bc7a479fc1b401"} 10:24:10 [10:24:10.037] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:64bc7a479fc1b401 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:64bc7a479fc1b401"} 10:24:10 [10:24:10.038] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:64bc7a479fc1b401 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:64bc7a479fc1b401"} 10:24:10 [10:24:10.039] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:64bc7a479fc1b401 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:64bc7a479fc1b401"} 10:24:10 [10:24:10.041] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.041] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf1fcb18a196f38c epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf1fcb18a196f38c"} 10:24:10 [10:24:10.072] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.075] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:42f430ed9836f04b epochNumber=1 {"provingJobId":"1:BASE_PARITY:42f430ed9836f04b"} 10:24:10 [10:24:10.077] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:42f430ed9836f04b {"provingJobId":"1:BASE_PARITY:42f430ed9836f04b"} 10:24:10 [10:24:10.077] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:42f430ed9836f04b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:42f430ed9836f04b"} 10:24:10 [10:24:10.078] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:42f430ed9836f04b type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:42f430ed9836f04b"} 10:24:10 [10:24:10.080] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.080] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6e14f6432a364bd5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6e14f6432a364bd5"} 10:24:10 [10:24:10.081] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:6e14f6432a364bd5 {"provingJobId":"1:BASE_PARITY:6e14f6432a364bd5"} 10:24:10 [10:24:10.082] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6e14f6432a364bd5 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:6e14f6432a364bd5"} 10:24:10 [10:24:10.084] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06cbe9f3cad42253 epochNumber=1 {"provingJobId":"1:BASE_PARITY:06cbe9f3cad42253"} 10:24:10 [10:24:10.085] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:06cbe9f3cad42253 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:06cbe9f3cad42253"} 10:24:10 [10:24:10.086] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:06cbe9f3cad42253 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:06cbe9f3cad42253"} 10:24:10 [10:24:10.088] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.089] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:03246d2faba36d79 epochNumber=1 {"provingJobId":"1:BASE_PARITY:03246d2faba36d79"} 10:24:10 [10:24:10.089] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:03246d2faba36d79 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:03246d2faba36d79"} 10:24:10 [10:24:10.090] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:03246d2faba36d79 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:03246d2faba36d79"} 10:24:10 [10:24:10.091] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.092] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2332d119a7480ca epochNumber=0 {"provingJobId":"1:BASE_PARITY:c2332d119a7480ca"} 10:24:10 [10:24:10.092] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c2332d119a7480ca epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c2332d119a7480ca"} 10:24:10 [10:24:10.093] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:98729a72c077a445 epochNumber=0 {"provingJobId":"1:BASE_PARITY:98729a72c077a445"} 10:24:10 [10:24:10.093] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c2332d119a7480ca epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c2332d119a7480ca"} 10:24:10 [10:24:10.093] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1626d6e7d2c8c352 epochNumber=0 {"provingJobId":"1:BASE_PARITY:1626d6e7d2c8c352"} 10:24:10 [10:24:10.094] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.095] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.096] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:170c3924de0aac84 epochNumber=2 {"provingJobId":"1:BASE_PARITY:170c3924de0aac84"} 10:24:10 [10:24:10.096] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0652af88834d9ba5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0652af88834d9ba5"} 10:24:10 [10:24:10.097] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.098] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8651d9471c35cd9d epochNumber=1 {"provingJobId":"1:BASE_PARITY:8651d9471c35cd9d"} 10:24:10 [10:24:10.098] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.099] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f6e9c86d34474887 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f6e9c86d34474887"} 10:24:10 [10:24:10.099] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a3087fc9f78447b epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a3087fc9f78447b"} 10:24:10 [10:24:10.099] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9755586782dd1355 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9755586782dd1355"} 10:24:10 [10:24:10.099] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b761468b51a3bfa0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b761468b51a3bfa0"} 10:24:10 [10:24:10.100] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.101] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6fdc09884e331bee epochNumber=1 {"provingJobId":"1:BASE_PARITY:6fdc09884e331bee"} 10:24:10 [10:24:10.101] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d33e4e343353d7d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d33e4e343353d7d7"} 10:24:10 [10:24:10.101] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:300aa11cd0508f75 epochNumber=2 {"provingJobId":"1:BASE_PARITY:300aa11cd0508f75"} 10:24:10 [10:24:10.101] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c548e2bf34ec51d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c548e2bf34ec51d5"} 10:24:10 [10:24:10.103] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.103] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:88b6550454bbb9eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:88b6550454bbb9eb"} 10:24:10 [10:24:10.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:44f0d6fa75dd2c0d epochNumber=1 {"provingJobId":"1:BASE_PARITY:44f0d6fa75dd2c0d"} 10:24:10 [10:24:10.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:71a50c899fa12298 epochNumber=2 {"provingJobId":"1:BASE_PARITY:71a50c899fa12298"} 10:24:10 [10:24:10.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:10e7820bd2b2bd00 epochNumber=1 {"provingJobId":"1:BASE_PARITY:10e7820bd2b2bd00"} 10:24:10 [10:24:10.106] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.106] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:29a96fe97eda0620 epochNumber=1 {"provingJobId":"1:BASE_PARITY:29a96fe97eda0620"} 10:24:10 [10:24:10.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb4e98446829d9aa epochNumber=1 {"provingJobId":"1:BASE_PARITY:eb4e98446829d9aa"} 10:24:10 [10:24:10.107] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:29a96fe97eda0620 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:29a96fe97eda0620"} 10:24:10 [10:24:10.109] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0c3dee2c37070cb4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0c3dee2c37070cb4"} 10:24:10 [10:24:10.111] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:94565e8ce6a69919 epochNumber=1 {"provingJobId":"1:BASE_PARITY:94565e8ce6a69919"} 10:24:10 [10:24:10.112] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0c3dee2c37070cb4 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:0c3dee2c37070cb4"} 10:24:10 [10:24:10.113] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.114] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:484069cb60037094 epochNumber=1 {"provingJobId":"1:BASE_PARITY:484069cb60037094"} 10:24:10 [10:24:10.114] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04850466c097bb31 epochNumber=1 {"provingJobId":"1:BASE_PARITY:04850466c097bb31"} 10:24:10 [10:24:10.114] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:484069cb60037094 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:484069cb60037094"} 10:24:10 [10:24:10.116] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.116] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4a1f145ea4cea22e epochNumber=1 {"provingJobId":"1:BASE_PARITY:4a1f145ea4cea22e"} 10:24:10 [10:24:10.116] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:4a1f145ea4cea22e {"provingJobId":"1:BASE_PARITY:4a1f145ea4cea22e"} 10:24:10 [10:24:10.117] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4a1f145ea4cea22e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:4a1f145ea4cea22e"} 10:24:10 [10:24:10.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a9b325aea765f672 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a9b325aea765f672"} 10:24:10 [10:24:10.117] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4a1f145ea4cea22e has already been completed {"provingJobId":"1:BASE_PARITY:4a1f145ea4cea22e"} 10:24:10 [10:24:10.128] INFO: prover-client:proving-broker Proving Broker started 10:24:10 [10:24:10.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c264bfff69fad4c9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c264bfff69fad4c9"} 10:24:10 [10:24:10.130] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6096555417f903af epochNumber=2 {"provingJobId":"1:BASE_PARITY:6096555417f903af"} 10:24:11 [10:24:11.182] INFO: prover-client:proving-broker Proving Broker started 10:24:11 [10:24:11.182] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c264bfff69fad4c9 settled=false {"provingJobId":"1:BASE_PARITY:c264bfff69fad4c9","status":"pending"} 10:24:11 [10:24:11.183] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6096555417f903af settled=false {"provingJobId":"1:BASE_PARITY:6096555417f903af","status":"pending"} 10:24:11 [10:24:11.184] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:c264bfff69fad4c9 type=BASE_PARITY startedAt=1750155850130 older agent has taken job {"provingJobId":"1:BASE_PARITY:c264bfff69fad4c9"} 10:24:11 [10:24:11.184] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c264bfff69fad4c9 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:c264bfff69fad4c9"} 10:24:11 [10:24:11.187] INFO: prover-client:proving-broker Proving Broker started 10:24:11 [10:24:11.188] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:68feabf68b36a8ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:68feabf68b36a8ba"} 10:24:11 [10:24:11.188] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:24303c194f7c6fef epochNumber=2 {"provingJobId":"1:BASE_PARITY:24303c194f7c6fef"} 10:24:12 [10:24:12.189] INFO: prover-client:proving-broker Proving Broker started 10:24:12 [10:24:12.194] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:68feabf68b36a8ba settled=false {"provingJobId":"1:BASE_PARITY:68feabf68b36a8ba","status":"pending"} 10:24:12 [10:24:12.194] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:24303c194f7c6fef settled=false {"provingJobId":"1:BASE_PARITY:24303c194f7c6fef","status":"pending"} 10:24:12 [10:24:12.195] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:68feabf68b36a8ba type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:68feabf68b36a8ba"} 10:24:12 [10:24:12.197] INFO: prover-client:proving-broker Proving Broker started 10:24:12 [10:24:12.197] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1c97fc0158dfdda epochNumber=1 {"provingJobId":"1:BASE_PARITY:a1c97fc0158dfdda"} 10:24:12 [10:24:12.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b522b7594d9b4edd epochNumber=2 {"provingJobId":"1:BASE_PARITY:b522b7594d9b4edd"} 10:24:13 [10:24:13.200] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.201] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a1c97fc0158dfdda settled=false {"provingJobId":"1:BASE_PARITY:a1c97fc0158dfdda","status":"pending"} 10:24:13 [10:24:13.201] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b522b7594d9b4edd settled=false {"provingJobId":"1:BASE_PARITY:b522b7594d9b4edd","status":"pending"} 10:24:13 [10:24:13.202] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a1c97fc0158dfdda type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a1c97fc0158dfdda"} 10:24:13 [10:24:13.202] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a1c97fc0158dfdda type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a1c97fc0158dfdda"} 10:24:13 [10:24:13.204] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:821f8085d5f7cecf epochNumber=1 {"provingJobId":"1:BASE_PARITY:821f8085d5f7cecf"} 10:24:13 [10:24:13.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b9a8fbae0f514ec3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b9a8fbae0f514ec3"} 10:24:13 [10:24:13.206] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:821f8085d5f7cecf type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:821f8085d5f7cecf"} 10:24:13 [10:24:13.207] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b9a8fbae0f514ec3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b9a8fbae0f514ec3"} 10:24:13 [10:24:13.208] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.209] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5e8ac711384e23d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5e8ac711384e23d3"} 10:24:13 [10:24:13.209] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f2aefc5425429a7e epochNumber=2 {"provingJobId":"1:BASE_PARITY:f2aefc5425429a7e"} 10:24:13 [10:24:13.209] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5e8ac711384e23d3 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5e8ac711384e23d3"} 10:24:13 [10:24:13.209] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5e8ac711384e23d3 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5e8ac711384e23d3"} 10:24:13 [10:24:13.210] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f2aefc5425429a7e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f2aefc5425429a7e"} 10:24:13 [10:24:13.210] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.211] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f73a9e2256effc2d err=test error {"provingJoId":"1:BASE_PARITY:f73a9e2256effc2d"} 10:24:13 [10:24:13.212] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.212] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7e2cdcaf2f13b20b not found {"provingJobId":"1:BASE_PARITY:7e2cdcaf2f13b20b"} 10:24:13 [10:24:13.213] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5710fa8c33824fe9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5710fa8c33824fe9"} 10:24:13 [10:24:13.215] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.216] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:518ae0e748b8961a epochNumber=1 {"provingJobId":"1:BASE_PARITY:518ae0e748b8961a"} 10:24:13 [10:24:13.317] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:518ae0e748b8961a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:518ae0e748b8961a"} 10:24:13 [10:24:13.324] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.325] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7ea13cd8346ec8c9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7ea13cd8346ec8c9"} 10:24:13 [10:24:13.426] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7ea13cd8346ec8c9 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:7ea13cd8346ec8c9"} 10:24:13 [10:24:13.433] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2b3a3854edaa3d8b epochNumber=10 {"provingJobId":"1:BASE_PARITY:2b3a3854edaa3d8b"} 10:24:13 [10:24:13.452] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:13 [10:24:13.561] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.561] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:428485851aa1eb2a epochNumber=1 {"provingJobId":"1:BASE_PARITY:428485851aa1eb2a"} 10:24:13 [10:24:13.737] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:428485851aa1eb2a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:428485851aa1eb2a"} 10:24:13 [10:24:13.766] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.767] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bc25da45f5c67047 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bc25da45f5c67047"} 10:24:13 [10:24:13.767] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bc25da45f5c67047 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:bc25da45f5c67047"} 10:24:13 [10:24:13.769] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.769] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:16328d5adaef895d epochNumber=1 {"provingJobId":"1:BASE_PARITY:16328d5adaef895d"} 10:24:13 [10:24:13.769] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:16328d5adaef895d type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:16328d5adaef895d"} 10:24:13 [10:24:13.770] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:16328d5adaef895d type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:16328d5adaef895d"} 10:24:13 [10:24:13.771] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.771] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b4f5ecd55c375243 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b4f5ecd55c375243"} 10:24:13 [10:24:13.771] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b4f5ecd55c375243 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b4f5ecd55c375243"} 10:24:13 [10:24:13.772] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.772] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fd2af2a13ae63b95 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fd2af2a13ae63b95"} 10:24:13 [10:24:13.773] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fd2af2a13ae63b95 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:fd2af2a13ae63b95"} 10:24:13 [10:24:13.773] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:556393f1be44573c epochNumber=3 {"provingJobId":"1:BASE_PARITY:556393f1be44573c"} 10:24:13 [10:24:13.797] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:13 [10:24:13.799] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:fd2af2a13ae63b95 err=test error {"provingJoId":"1:BASE_PARITY:fd2af2a13ae63b95"} 10:24:13 [10:24:13.802] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.802] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4cd389ecc7e8cad2 settled=false {"provingJobId":"1:BASE_PARITY:4cd389ecc7e8cad2","status":"pending"} 10:24:13 [10:24:13.802] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:630046957c25614b settled=false {"provingJobId":"1:BASE_PARITY:630046957c25614b","status":"pending"} 10:24:13 [10:24:13.806] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.806] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f8f09baec5d99813 settled=true {"provingJobId":"1:BASE_PARITY:f8f09baec5d99813","status":"fulfilled"} 10:24:13 [10:24:13.807] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:0e395a62614f6cbb settled=true {"provingJobId":"2:BASE_PARITY:0e395a62614f6cbb","status":"fulfilled"} 10:24:13 [10:24:13.809] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.809] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d888496bfc87bda5 settled=true {"provingJobId":"1:BASE_PARITY:d888496bfc87bda5","status":"fulfilled"} 10:24:13 [10:24:13.809] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:322aaf301608fa7f settled=false {"provingJobId":"1:BASE_PARITY:322aaf301608fa7f","status":"pending"} 10:24:13 [10:24:13.811] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.812] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cabb343114a7ea28 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cabb343114a7ea28"} 10:24:13 [10:24:13.814] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.814] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e95bbc9696363519 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e95bbc9696363519"} 10:24:13 [10:24:13.815] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:e95bbc9696363519: Error: db error: Error: db error 10:24:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:e95bbc9696363519"} 10:24:13 [10:24:13.817] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.817] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1cd8bb224ced2567 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1cd8bb224ced2567"} 10:24:13 [10:24:13.817] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1cd8bb224ced2567 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1cd8bb224ced2567"} 10:24:13 [10:24:13.818] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1cd8bb224ced2567 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1cd8bb224ced2567"} 10:24:13 [10:24:13.819] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.820] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7be06833d2ab6b1d epochNumber=1 {"provingJobId":"1:BASE_PARITY:7be06833d2ab6b1d"} 10:24:13 [10:24:13.820] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7be06833d2ab6b1d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7be06833d2ab6b1d"} 10:24:13 [10:24:13.820] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7be06833d2ab6b1d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7be06833d2ab6b1d"} 10:24:13 [10:24:13.821] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:7be06833d2ab6b1d: Error: db error 10:24:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:7be06833d2ab6b1d"} 10:24:13 [10:24:13.822] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.823] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:611996af1a162a3b epochNumber=1 {"provingJobId":"1:BASE_PARITY:611996af1a162a3b"} 10:24:13 [10:24:13.823] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:611996af1a162a3b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:611996af1a162a3b"} 10:24:13 [10:24:13.824] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:611996af1a162a3b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:611996af1a162a3b"} 10:24:13 [10:24:13.825] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.825] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:63e5426318861df7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:63e5426318861df7"} 10:24:13 [10:24:13.826] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:63e5426318861df7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:63e5426318861df7"} 10:24:13 [10:24:13.826] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:63e5426318861df7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:63e5426318861df7"} 10:24:13 [10:24:13.827] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:63e5426318861df7 jobErr=test error: Error: db error 10:24:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:63e5426318861df7"} 10:24:13 [10:24:13.828] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.828] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a59503bae1e49e90 not found {"provingJobId":"1:BASE_PARITY:a59503bae1e49e90"} 10:24:13 [10:24:13.830] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.830] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a1945abbd4268f1f err=test error {"provingJoId":"1:BASE_PARITY:a1945abbd4268f1f"} 10:24:13 [10:24:13.832] INFO: prover-client:proving-broker Proving Broker started 10:24:13 [10:24:13.843] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6c0c410713d8175b epochNumber=1 {"provingJobId":"1:BASE_PARITY:6c0c410713d8175b"} 10:24:13 [10:24:13.844] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6c0c410713d8175b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6c0c410713d8175b"} 10:24:13 [10:24:13.844] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6c0c410713d8175b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6c0c410713d8175b"} 10:24:13 [10:24:13.855] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:5cd9fb11c449bbb7 epochNumber=2 {"provingJobId":"2:BASE_PARITY:5cd9fb11c449bbb7"} 10:24:13 [10:24:13.855] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:5cd9fb11c449bbb7 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:5cd9fb11c449bbb7"} 10:24:13 [10:24:13.856] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:5cd9fb11c449bbb7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:5cd9fb11c449bbb7"} 10:24:13 [10:24:13.917] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:f0751a1ff492ba70 epochNumber=3 {"provingJobId":"3:BASE_PARITY:f0751a1ff492ba70"} 10:24:13 [10:24:13.935] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:13 [10:24:13.979] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:878cc9481e72b93a epochNumber=4 {"provingJobId":"4:BASE_PARITY:878cc9481e72b93a"} 10:24:13 [10:24:13.987] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:14 [10:24:14.044] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:cfcea31680f8fb72 epochNumber=5 {"provingJobId":"5:BASE_PARITY:cfcea31680f8fb72"} 10:24:14 [10:24:14.064] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:14 [10:24:14.099] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.100] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:952ad13f1bfc8dfc epochNumber=42 {"provingJobId":"1:BASE_PARITY:952ad13f1bfc8dfc"} 10:24:14 [10:24:14.101] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testE1PGxi/42 with map size 1073741824 10:24:14 [10:24:14.105] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.131] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:3f2a5b555cb55087 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f2a5b555cb55087"} 10:24:14 [10:24:14.137] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.137] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a32d78881d2dcca epochNumber=1 {"provingJobId":"1:BASE_PARITY:6a32d78881d2dcca"} 10:24:14 [10:24:14.138] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAeoIrA/1 with map size 1073741824 10:24:14 [10:24:14.140] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.159] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac93a876b2a61455 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac93a876b2a61455"} 10:24:14 [10:24:14.168] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.169] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15231bf0a69deabb epochNumber=1 {"provingJobId":"1:BASE_PARITY:15231bf0a69deabb"} 10:24:14 [10:24:14.170] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGG40SX/1 with map size 1073741824 10:24:14 [10:24:14.172] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.191] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:15231bf0a69deabb epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:15231bf0a69deabb"} 10:24:14 [10:24:14.195] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:17c146d8a4b2e62d epochNumber=1 {"provingJobId":"1:BASE_PARITY:17c146d8a4b2e62d"} 10:24:14 [10:24:14.196] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzbiE5A/1 with map size 1073741824 10:24:14 [10:24:14.198] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.217] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:17c146d8a4b2e62d epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:17c146d8a4b2e62d"} 10:24:14 [10:24:14.217] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:17c146d8a4b2e62d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:17c146d8a4b2e62d"} 10:24:14 [10:24:14.224] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:17c146d8a4b2e62d epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:17c146d8a4b2e62d"} 10:24:14 [10:24:14.228] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:861b65139756b255 epochNumber=1 {"provingJobId":"1:BASE_PARITY:861b65139756b255"} 10:24:14 [10:24:14.229] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXdqNAR/1 with map size 1073741824 10:24:14 [10:24:14.231] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.256] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.259] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.260] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:37cdeca7d1a4def8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:37cdeca7d1a4def8"} 10:24:14 [10:24:14.261] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2JTTHh/1 with map size 1073741824 10:24:14 [10:24:14.264] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.285] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:37cdeca7d1a4def8 {"provingJobId":"1:BASE_PARITY:37cdeca7d1a4def8"} 10:24:14 [10:24:14.286] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:37cdeca7d1a4def8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:37cdeca7d1a4def8"} 10:24:14 [10:24:14.286] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:37cdeca7d1a4def8 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:37cdeca7d1a4def8"} 10:24:14 [10:24:14.297] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.298] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e6d40f012d437c21 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e6d40f012d437c21"} 10:24:14 [10:24:14.299] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGmiy2o/1 with map size 1073741824 10:24:14 [10:24:14.301] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.320] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e6d40f012d437c21 {"provingJobId":"1:BASE_PARITY:e6d40f012d437c21"} 10:24:14 [10:24:14.321] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e6d40f012d437c21 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:e6d40f012d437c21"} 10:24:14 [10:24:14.330] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.331] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0c14dd06636f5c23 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0c14dd06636f5c23"} 10:24:14 [10:24:14.332] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testW0KGW0/1 with map size 1073741824 10:24:14 [10:24:14.334] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.352] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0c14dd06636f5c23 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0c14dd06636f5c23"} 10:24:14 [10:24:14.353] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0c14dd06636f5c23 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:0c14dd06636f5c23"} 10:24:14 [10:24:14.361] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.362] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f941efb0c39eb2fe epochNumber=1 {"provingJobId":"1:BASE_PARITY:f941efb0c39eb2fe"} 10:24:14 [10:24:14.363] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8gBIb0/1 with map size 1073741824 10:24:14 [10:24:14.365] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.382] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f941efb0c39eb2fe type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f941efb0c39eb2fe"} 10:24:14 [10:24:14.383] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f941efb0c39eb2fe type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f941efb0c39eb2fe"} 10:24:14 [10:24:14.391] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.392] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7f8462bdc86bdcc6 epochNumber=0 {"provingJobId":"1:BASE_PARITY:7f8462bdc86bdcc6"} 10:24:14 [10:24:14.392] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testHjp2iB/0 with map size 1073741824 10:24:14 [10:24:14.394] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.410] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7f8462bdc86bdcc6 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7f8462bdc86bdcc6"} 10:24:14 [10:24:14.411] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3c5419bb7c00e94 epochNumber=0 {"provingJobId":"1:BASE_PARITY:e3c5419bb7c00e94"} 10:24:14 [10:24:14.411] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7f8462bdc86bdcc6 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7f8462bdc86bdcc6"} 10:24:14 [10:24:14.411] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0de6aaa607406703 epochNumber=0 {"provingJobId":"1:BASE_PARITY:0de6aaa607406703"} 10:24:14 [10:24:14.425] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.427] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.428] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da3fcfe6791485b2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:da3fcfe6791485b2"} 10:24:14 [10:24:14.430] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPr79T9/2 with map size 1073741824 10:24:14 [10:24:14.432] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.452] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2f90a35f3c84ec9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2f90a35f3c84ec9"} 10:24:14 [10:24:14.453] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPr79T9/1 with map size 1073741824 10:24:14 [10:24:14.455] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.476] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.477] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afbd77b23525257e epochNumber=1 {"provingJobId":"1:BASE_PARITY:afbd77b23525257e"} 10:24:14 [10:24:14.478] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testy36oZf/1 with map size 1073741824 10:24:14 [10:24:14.479] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.504] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.505] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d23a098f2a65cdcb epochNumber=1 {"provingJobId":"1:BASE_PARITY:d23a098f2a65cdcb"} 10:24:14 [10:24:14.506] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3KbbYL/1 with map size 1073741824 10:24:14 [10:24:14.507] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.526] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08fadda01ced99b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:08fadda01ced99b2"} 10:24:14 [10:24:14.532] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:655886648753870c epochNumber=2 {"provingJobId":"1:BASE_PARITY:655886648753870c"} 10:24:14 [10:24:14.534] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3KbbYL/2 with map size 1073741824 10:24:14 [10:24:14.536] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.554] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a059aac0a8c8749e epochNumber=1 {"provingJobId":"1:BASE_PARITY:a059aac0a8c8749e"} 10:24:14 [10:24:14.564] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.565] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a3af8add43395ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:6a3af8add43395ba"} 10:24:14 [10:24:14.566] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIRGzBV/1 with map size 1073741824 10:24:14 [10:24:14.567] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.586] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:72d2a413863c3901 epochNumber=1 {"provingJobId":"1:BASE_PARITY:72d2a413863c3901"} 10:24:14 [10:24:14.592] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab46c3b5fc44fb46 epochNumber=2 {"provingJobId":"1:BASE_PARITY:ab46c3b5fc44fb46"} 10:24:14 [10:24:14.593] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testIRGzBV/2 with map size 1073741824 10:24:14 [10:24:14.595] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.612] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5de560852a5098b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5de560852a5098b0"} 10:24:14 [10:24:14.620] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.621] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1d57f0eb55c37c60 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1d57f0eb55c37c60"} 10:24:14 [10:24:14.622] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAhLUo4/1 with map size 1073741824 10:24:14 [10:24:14.624] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.642] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2f0cd2f28e4ab82 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2f0cd2f28e4ab82"} 10:24:14 [10:24:14.649] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f4053628ef37b234 epochNumber=2 {"provingJobId":"1:BASE_PARITY:f4053628ef37b234"} 10:24:14 [10:24:14.650] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAhLUo4/2 with map size 1073741824 10:24:14 [10:24:14.652] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.670] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cd4eb55c89578f19 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cd4eb55c89578f19"} 10:24:14 [10:24:14.681] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.682] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c3fe86926eb17552 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c3fe86926eb17552"} 10:24:14 [10:24:14.683] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWlRMZA/1 with map size 1073741824 10:24:14 [10:24:14.685] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.710] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e44d0657f64ebd14 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e44d0657f64ebd14"} 10:24:14 [10:24:14.717] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c3fe86926eb17552 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c3fe86926eb17552"} 10:24:14 [10:24:14.727] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.728] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5ebba9b6ab2dce88 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5ebba9b6ab2dce88"} 10:24:14 [10:24:14.729] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9cid6i/1 with map size 1073741824 10:24:14 [10:24:14.730] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.749] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb065d4c0315c314 epochNumber=1 {"provingJobId":"1:BASE_PARITY:eb065d4c0315c314"} 10:24:14 [10:24:14.755] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5ebba9b6ab2dce88 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:5ebba9b6ab2dce88"} 10:24:14 [10:24:14.765] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.765] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e11eb302f5376746 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e11eb302f5376746"} 10:24:14 [10:24:14.766] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNjY2LK/1 with map size 1073741824 10:24:14 [10:24:14.767] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.787] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afc442f87da3f5d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:afc442f87da3f5d7"} 10:24:14 [10:24:14.794] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e11eb302f5376746 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:e11eb302f5376746"} 10:24:14 [10:24:14.799] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.801] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7b4de17476a3f054 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7b4de17476a3f054"} 10:24:14 [10:24:14.802] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSUOPu7/1 with map size 1073741824 10:24:14 [10:24:14.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.821] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7b4de17476a3f054 {"provingJobId":"1:BASE_PARITY:7b4de17476a3f054"} 10:24:14 [10:24:14.822] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7b4de17476a3f054 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:7b4de17476a3f054"} 10:24:14 [10:24:14.829] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afed3bd9f8a5a7f0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:afed3bd9f8a5a7f0"} 10:24:14 [10:24:14.836] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7b4de17476a3f054 has already been completed {"provingJobId":"1:BASE_PARITY:7b4de17476a3f054"} 10:24:14 [10:24:14.840] INFO: prover-client:proving-broker Proving Broker started 10:24:14 [10:24:14.841] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba0660c6e39a6594 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba0660c6e39a6594"} 10:24:14 [10:24:14.841] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAaHdmr/1 with map size 1073741824 10:24:14 [10:24:14.843] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:14 [10:24:14.859] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7d6bc10e82e5e6c3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:7d6bc10e82e5e6c3"} 10:24:14 [10:24:14.860] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAaHdmr/2 with map size 1073741824 10:24:14 [10:24:14.861] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:15 [10:24:15.931] INFO: prover-client:proving-broker Proving Broker started 10:24:15 [10:24:15.937] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ba0660c6e39a6594 settled=false {"provingJobId":"1:BASE_PARITY:ba0660c6e39a6594","status":"pending"} 10:24:15 [10:24:15.939] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7d6bc10e82e5e6c3 settled=false {"provingJobId":"1:BASE_PARITY:7d6bc10e82e5e6c3","status":"pending"} 10:24:15 [10:24:15.939] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:ba0660c6e39a6594 type=BASE_PARITY startedAt=1750155854879 older agent has taken job {"provingJobId":"1:BASE_PARITY:ba0660c6e39a6594"} 10:24:15 [10:24:15.940] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ba0660c6e39a6594 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:ba0660c6e39a6594"} 10:24:15 [10:24:15.943] INFO: prover-client:proving-broker Proving Broker started 10:24:15 [10:24:15.945] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a99bb936ba9b6f33 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a99bb936ba9b6f33"} 10:24:15 [10:24:15.945] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoFPL9R/1 with map size 1073741824 10:24:15 [10:24:15.950] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:15 [10:24:15.969] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7bfe578895671781 epochNumber=2 {"provingJobId":"1:BASE_PARITY:7bfe578895671781"} 10:24:15 [10:24:15.970] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testoFPL9R/2 with map size 1073741824 10:24:15 [10:24:15.972] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:16 [10:24:16.997] INFO: prover-client:proving-broker Proving Broker started 10:24:17 [10:24:17.000] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a99bb936ba9b6f33 settled=false {"provingJobId":"1:BASE_PARITY:a99bb936ba9b6f33","status":"pending"} 10:24:17 [10:24:17.004] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7bfe578895671781 settled=false {"provingJobId":"1:BASE_PARITY:7bfe578895671781","status":"pending"} 10:24:17 [10:24:17.005] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a99bb936ba9b6f33 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:a99bb936ba9b6f33"} 10:24:17 [10:24:17.010] INFO: prover-client:proving-broker Proving Broker started 10:24:17 [10:24:17.011] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3161b3377ce0effc epochNumber=1 {"provingJobId":"1:BASE_PARITY:3161b3377ce0effc"} 10:24:17 [10:24:17.013] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testv99ztv/1 with map size 1073741824 10:24:17 [10:24:17.016] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:17 [10:24:17.032] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4e15e76d78a8aeda epochNumber=2 {"provingJobId":"1:BASE_PARITY:4e15e76d78a8aeda"} 10:24:17 [10:24:17.033] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testv99ztv/2 with map size 1073741824 10:24:17 [10:24:17.035] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.051] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.054] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3161b3377ce0effc settled=false {"provingJobId":"1:BASE_PARITY:3161b3377ce0effc","status":"pending"} 10:24:18 [10:24:18.056] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4e15e76d78a8aeda settled=false {"provingJobId":"1:BASE_PARITY:4e15e76d78a8aeda","status":"pending"} 10:24:18 [10:24:18.057] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3161b3377ce0effc type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3161b3377ce0effc"} 10:24:18 [10:24:18.058] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3161b3377ce0effc type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3161b3377ce0effc"} 10:24:18 [10:24:18.068] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.068] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:96040485ff0aabf2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:96040485ff0aabf2"} 10:24:18 [10:24:18.069] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testz8wiUC/1 with map size 1073741824 10:24:18 [10:24:18.071] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:280df78047b8363e epochNumber=2 {"provingJobId":"1:BASE_PARITY:280df78047b8363e"} 10:24:18 [10:24:18.086] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testz8wiUC/2 with map size 1073741824 10:24:18 [10:24:18.087] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.104] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:96040485ff0aabf2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:96040485ff0aabf2"} 10:24:18 [10:24:18.110] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:280df78047b8363e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:280df78047b8363e"} 10:24:18 [10:24:18.120] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.121] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:00597b3e1615cc26 epochNumber=1 {"provingJobId":"1:BASE_PARITY:00597b3e1615cc26"} 10:24:18 [10:24:18.122] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg9VSoU/1 with map size 1073741824 10:24:18 [10:24:18.124] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.140] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3133f81fd302f4d9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3133f81fd302f4d9"} 10:24:18 [10:24:18.141] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testg9VSoU/2 with map size 1073741824 10:24:18 [10:24:18.143] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.163] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:00597b3e1615cc26 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:00597b3e1615cc26"} 10:24:18 [10:24:18.164] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:00597b3e1615cc26 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:00597b3e1615cc26"} 10:24:18 [10:24:18.170] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3133f81fd302f4d9 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3133f81fd302f4d9"} 10:24:18 [10:24:18.183] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.185] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:10ae52971395ac30 err=test error {"provingJoId":"1:BASE_PARITY:10ae52971395ac30"} 10:24:18 [10:24:18.190] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.191] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ce751532448b86c8 not found {"provingJobId":"1:BASE_PARITY:ce751532448b86c8"} 10:24:18 [10:24:18.194] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c14457a215289f3b epochNumber=1 {"provingJobId":"1:BASE_PARITY:c14457a215289f3b"} 10:24:18 [10:24:18.196] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrkPPf5/1 with map size 1073741824 10:24:18 [10:24:18.198] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.219] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.220] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2b083f95f5002cab epochNumber=1 {"provingJobId":"1:BASE_PARITY:2b083f95f5002cab"} 10:24:18 [10:24:18.221] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCt9yvx/1 with map size 1073741824 10:24:18 [10:24:18.222] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.347] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2b083f95f5002cab timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:2b083f95f5002cab"} 10:24:18 [10:24:18.352] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.353] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b970c58701a0048 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9b970c58701a0048"} 10:24:18 [10:24:18.354] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVEVTaF/1 with map size 1073741824 10:24:18 [10:24:18.357] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.480] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9b970c58701a0048 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:9b970c58701a0048"} 10:24:18 [10:24:18.482] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:57fa5a79ef85d587 epochNumber=10 {"provingJobId":"1:BASE_PARITY:57fa5a79ef85d587"} 10:24:18 [10:24:18.482] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testVEVTaF/10 with map size 1073741824 10:24:18 [10:24:18.486] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.532] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:18 [10:24:18.533] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:24:18 [10:24:18.639] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.640] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f7114dfe2a552bac epochNumber=1 {"provingJobId":"1:BASE_PARITY:f7114dfe2a552bac"} 10:24:18 [10:24:18.641] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOXPpwr/1 with map size 1073741824 10:24:18 [10:24:18.643] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.820] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f7114dfe2a552bac timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f7114dfe2a552bac"} 10:24:18 [10:24:18.866] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.867] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:70ff44d3799e9a60 epochNumber=1 {"provingJobId":"1:BASE_PARITY:70ff44d3799e9a60"} 10:24:18 [10:24:18.868] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1941jm/1 with map size 1073741824 10:24:18 [10:24:18.870] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.895] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:70ff44d3799e9a60 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:70ff44d3799e9a60"} 10:24:18 [10:24:18.900] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.901] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:19cc6a6d8099f8cf epochNumber=1 {"provingJobId":"1:BASE_PARITY:19cc6a6d8099f8cf"} 10:24:18 [10:24:18.901] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnnli4A/1 with map size 1073741824 10:24:18 [10:24:18.905] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.926] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:19cc6a6d8099f8cf type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:19cc6a6d8099f8cf"} 10:24:18 [10:24:18.927] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:19cc6a6d8099f8cf type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:19cc6a6d8099f8cf"} 10:24:18 [10:24:18.941] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.943] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4f3feb34c44e411 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4f3feb34c44e411"} 10:24:18 [10:24:18.945] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9guj2f/1 with map size 1073741824 10:24:18 [10:24:18.948] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:18 [10:24:18.971] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d4f3feb34c44e411 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:d4f3feb34c44e411"} 10:24:18 [10:24:18.990] INFO: prover-client:proving-broker Proving Broker started 10:24:18 [10:24:18.991] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f05e8c1d7f989dc4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f05e8c1d7f989dc4"} 10:24:18 [10:24:18.993] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAPRSCS/1 with map size 1073741824 10:24:18 [10:24:19.000] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.022] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f05e8c1d7f989dc4 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:f05e8c1d7f989dc4"} 10:24:19 [10:24:19.024] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba745884b3341700 epochNumber=3 {"provingJobId":"1:BASE_PARITY:ba745884b3341700"} 10:24:19 [10:24:19.024] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testAPRSCS/3 with map size 1073741824 10:24:19 [10:24:19.033] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.066] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:19 [10:24:19.066] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:24:19 [10:24:19.084] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f05e8c1d7f989dc4 err=test error {"provingJoId":"1:BASE_PARITY:f05e8c1d7f989dc4"} 10:24:19 [10:24:19.095] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfwGKCw/1 with map size 1073741824 10:24:19 [10:24:19.098] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.119] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfwGKCw/2 with map size 1073741824 10:24:19 [10:24:19.125] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.147] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.150] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8d35ff8e515cf5b3 settled=false {"provingJobId":"1:BASE_PARITY:8d35ff8e515cf5b3","status":"pending"} 10:24:19 [10:24:19.151] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d84b7abc0c78f90e settled=false {"provingJobId":"1:BASE_PARITY:d84b7abc0c78f90e","status":"pending"} 10:24:19 [10:24:19.156] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHNu9cw/1 with map size 1073741824 10:24:19 [10:24:19.159] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.180] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testHNu9cw/2 with map size 1073741824 10:24:19 [10:24:19.190] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.229] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.232] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1e8aba8d48f56fdf settled=true {"provingJobId":"1:BASE_PARITY:1e8aba8d48f56fdf","status":"fulfilled"} 10:24:19 [10:24:19.233] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:c03473cd7ebf9a2f settled=true {"provingJobId":"2:BASE_PARITY:c03473cd7ebf9a2f","status":"fulfilled"} 10:24:19 [10:24:19.242] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVoF01e/1 with map size 1073741824 10:24:19 [10:24:19.247] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.279] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVoF01e/2 with map size 1073741824 10:24:19 [10:24:19.287] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.308] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.310] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1b0efeac13f0b1f5 settled=true {"provingJobId":"1:BASE_PARITY:1b0efeac13f0b1f5","status":"fulfilled"} 10:24:19 [10:24:19.312] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b66e80e27b1974aa settled=false {"provingJobId":"1:BASE_PARITY:b66e80e27b1974aa","status":"pending"} 10:24:19 [10:24:19.318] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.319] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3976fc4cd0be6af4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3976fc4cd0be6af4"} 10:24:19 [10:24:19.321] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqRgj80/1 with map size 1073741824 10:24:19 [10:24:19.323] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.349] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.350] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:746855a559c62857 epochNumber=1 {"provingJobId":"1:BASE_PARITY:746855a559c62857"} 10:24:19 [10:24:19.350] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:746855a559c62857: Error: db error: Error: db error 10:24:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:746855a559c62857"} 10:24:19 [10:24:19.354] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.354] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5187b05f876f3002 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5187b05f876f3002"} 10:24:19 [10:24:19.355] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXYee4g/1 with map size 1073741824 10:24:19 [10:24:19.358] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.384] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5187b05f876f3002 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5187b05f876f3002"} 10:24:19 [10:24:19.385] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5187b05f876f3002 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5187b05f876f3002"} 10:24:19 [10:24:19.399] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.400] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf747e6ab74264e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf747e6ab74264e5"} 10:24:19 [10:24:19.401] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0mMLcZ/1 with map size 1073741824 10:24:19 [10:24:19.404] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.422] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cf747e6ab74264e5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:cf747e6ab74264e5"} 10:24:19 [10:24:19.423] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:cf747e6ab74264e5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:cf747e6ab74264e5"} 10:24:19 [10:24:19.424] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:cf747e6ab74264e5: Error: db error 10:24:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:cf747e6ab74264e5"} 10:24:19 [10:24:19.431] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.432] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8b5eef2b627c5a5c epochNumber=1 {"provingJobId":"1:BASE_PARITY:8b5eef2b627c5a5c"} 10:24:19 [10:24:19.433] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzgBLg3/1 with map size 1073741824 10:24:19 [10:24:19.436] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.456] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8b5eef2b627c5a5c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8b5eef2b627c5a5c"} 10:24:19 [10:24:19.456] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8b5eef2b627c5a5c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8b5eef2b627c5a5c"} 10:24:19 [10:24:19.470] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.471] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5712e078cdf8c53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b5712e078cdf8c53"} 10:24:19 [10:24:19.472] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrpPWni/1 with map size 1073741824 10:24:19 [10:24:19.478] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.496] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b5712e078cdf8c53 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b5712e078cdf8c53"} 10:24:19 [10:24:19.497] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b5712e078cdf8c53 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b5712e078cdf8c53"} 10:24:19 [10:24:19.498] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:b5712e078cdf8c53 jobErr=test error: Error: db error 10:24:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:b5712e078cdf8c53"} 10:24:19 [10:24:19.503] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.504] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cb5521a0b1a6b514 not found {"provingJobId":"1:BASE_PARITY:cb5521a0b1a6b514"} 10:24:19 [10:24:19.511] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.511] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e902a7209ddee45c err=test error {"provingJoId":"1:BASE_PARITY:e902a7209ddee45c"} 10:24:19 [10:24:19.516] INFO: prover-client:proving-broker Proving Broker started 10:24:19 [10:24:19.527] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:126b039af1d3edc7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:126b039af1d3edc7"} 10:24:19 [10:24:19.529] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1GUPwa/1 with map size 1073741824 10:24:19 [10:24:19.531] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.548] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:126b039af1d3edc7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:126b039af1d3edc7"} 10:24:19 [10:24:19.549] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:126b039af1d3edc7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:126b039af1d3edc7"} 10:24:19 [10:24:19.567] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:0e7add70a4ef5642 epochNumber=2 {"provingJobId":"2:BASE_PARITY:0e7add70a4ef5642"} 10:24:19 [10:24:19.568] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test1GUPwa/2 with map size 1073741824 10:24:19 [10:24:19.571] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.598] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:0e7add70a4ef5642 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:0e7add70a4ef5642"} 10:24:19 [10:24:19.599] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:0e7add70a4ef5642 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:0e7add70a4ef5642"} 10:24:19 [10:24:19.666] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:0a6860a50f79bde0 epochNumber=3 {"provingJobId":"3:BASE_PARITY:0a6860a50f79bde0"} 10:24:19 [10:24:19.667] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test1GUPwa/3 with map size 1073741824 10:24:19 [10:24:19.672] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.721] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:19 [10:24:19.722] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:24:19 [10:24:19.765] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:8a06aed88ff6482c epochNumber=4 {"provingJobId":"4:BASE_PARITY:8a06aed88ff6482c"} 10:24:19 [10:24:19.766] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test1GUPwa/4 with map size 1073741824 10:24:19 [10:24:19.771] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.803] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:19 [10:24:19.804] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 10:24:19 [10:24:19.858] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:8c17c0c714953897 epochNumber=5 {"provingJobId":"5:BASE_PARITY:8c17c0c714953897"} 10:24:19 [10:24:19.860] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test1GUPwa/5 with map size 1073741824 10:24:19 [10:24:19.863] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:24:19 [10:24:19.912] INFO: prover-client:proving-broker Cleaned up jobs=1 10:24:19 [10:24:19.913] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 10:24:19 PASS src/proving_broker/proving_broker.test.ts (13.443 s) 10:24:19 ProvingBroker 10:24:19 Producer API 10:24:19 ✓ refuses stale jobs (54 ms) 10:24:19 ✓ enqueues jobs (5 ms) 10:24:19 ✓ ignores duplicate jobs (5 ms) 10:24:19 ✓ reports correct status when enqueuing repeat jobs (5 ms) 10:24:19 ✓ throws an error in case of duplicate job IDs (31 ms) 10:24:19 ✓ returns not-found status for non-existing jobs (2 ms) 10:24:19 ✓ cancels jobs in queue (4 ms) 10:24:19 ✓ cancels jobs in-progress (3 ms) 10:24:19 ✓ returns job result if successful (3 ms) 10:24:19 ✓ returns job error if failed (3 ms) 10:24:19 ✓ correctly returns job status for concurrent writes (3 ms) 10:24:19 ✓ refuses stale jobs (38 ms) 10:24:19 ✓ enqueues jobs (31 ms) 10:24:19 ✓ ignores duplicate jobs (27 ms) 10:24:19 ✓ reports correct status when enqueuing repeat jobs (33 ms) 10:24:19 ✓ throws an error in case of duplicate job IDs (28 ms) 10:24:19 ✓ returns not-found status for non-existing jobs (3 ms) 10:24:19 ✓ cancels jobs in queue (37 ms) 10:24:19 ✓ cancels jobs in-progress (32 ms) 10:24:19 ✓ returns job result if successful (31 ms) 10:24:19 ✓ returns job error if failed (29 ms) 10:24:19 ✓ correctly returns job status for concurrent writes (33 ms) 10:24:19 Consumer API 10:24:19 ✓ returns undefined if no jobs are available (1 ms) 10:24:19 ✓ returns jobs in priority order (2 ms) 10:24:19 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 10:24:19 ✓ returns a job if it is in the allowList (2 ms) 10:24:19 ✓ returns the most important job if it is in the allowList (2 ms) 10:24:19 ✓ returns any job if filter is empty (3 ms) 10:24:19 ✓ returns a new job when reporting job success (3 ms) 10:24:19 ✓ returns a new job when reporting permanent error (4 ms) 10:24:19 ✓ returns a new job when reporting retry-able error (2 ms) 10:24:19 ✓ returns a new job when reporting progress if current one is cancelled (11 ms) 10:24:19 ✓ returns a new job if job is already in progress elsewhere (1059 ms) 10:24:19 ✓ avoids sending the same job to a new agent after a restart (1009 ms) 10:24:19 ✓ avoids sending a completed job to a new agent after a restart (1007 ms) 10:24:19 ✓ tracks job result if in progress (3 ms) 10:24:19 ✓ tracks job result even if job is in queue (2 ms) 10:24:19 ✓ ignores reported job error if unknown job (1 ms) 10:24:19 ✓ ignores job result if unknown job (1 ms) 10:24:19 ✓ returns undefined if no jobs are available (2 ms) 10:24:19 ✓ returns jobs in priority order (48 ms) 10:24:19 ✓ returns undefined if no jobs are available for the given allowList (27 ms) 10:24:19 ✓ returns a job if it is in the allowList (60 ms) 10:24:19 ✓ returns the most important job if it is in the allowList (55 ms) 10:24:19 ✓ returns any job if filter is empty (60 ms) 10:24:19 ✓ returns a new job when reporting job success (46 ms) 10:24:19 ✓ returns a new job when reporting permanent error (37 ms) 10:24:19 ✓ returns a new job when reporting retry-able error (34 ms) 10:24:19 ✓ returns a new job when reporting progress if current one is cancelled (41 ms) 10:24:19 ✓ returns a new job if job is already in progress elsewhere (1102 ms) 10:24:19 ✓ avoids sending the same job to a new agent after a restart (1066 ms) 10:24:19 ✓ avoids sending a completed job to a new agent after a restart (1058 ms) 10:24:19 ✓ tracks job result if in progress (52 ms) 10:24:19 ✓ tracks job result even if job is in queue (62 ms) 10:24:19 ✓ ignores reported job error if unknown job (6 ms) 10:24:19 ✓ ignores job result if unknown job (6 ms) 10:24:19 Timeouts 10:24:19 ✓ tracks in progress jobs (2 ms) 10:24:19 ✓ re-enqueues jobs that time out (108 ms) 10:24:19 ✓ cancel stale jobs that time out (237 ms) 10:24:19 ✓ keeps the jobs in progress while it is alive (205 ms) 10:24:19 ✓ tracks in progress jobs (24 ms) 10:24:19 ✓ re-enqueues jobs that time out (132 ms) 10:24:19 ✓ cancel stale jobs that time out (287 ms) 10:24:19 ✓ keeps the jobs in progress while it is alive (227 ms) 10:24:19 Retries 10:24:19 ✓ retries jobs (2 ms) 10:24:19 ✓ retries up to a maximum number of times (2 ms) 10:24:19 ✓ passing retry=false does not retry (2 ms) 10:24:19 ✓ does not retry if job is stale (28 ms) 10:24:19 ✓ retries jobs (33 ms) 10:24:19 ✓ retries up to a maximum number of times (40 ms) 10:24:19 ✓ passing retry=false does not retry (44 ms) 10:24:19 ✓ does not retry if job is stale (107 ms) 10:24:19 Database management 10:24:19 ✓ re-enqueues proof requests on start (4 ms) 10:24:19 ✓ restores proof results on start (3 ms) 10:24:19 ✓ only re-enqueues unfinished jobs (2 ms) 10:24:19 ✓ saves job when enqueued (3 ms) 10:24:19 ✓ does not retain job if database fails to save (3 ms) 10:24:19 ✓ saves job result (2 ms) 10:24:19 ✓ saves result even if database fails to save (3 ms) 10:24:19 ✓ saves job error (2 ms) 10:24:19 ✓ saves job error even if database fails to save (2 ms) 10:24:19 ✓ does not save job result if job is unknown (2 ms) 10:24:19 ✓ does not save job error if job is unknown (2 ms) 10:24:19 ✓ cleans up old jobs periodically (265 ms) 10:24:19 ✓ re-enqueues proof requests on start (62 ms) 10:24:19 ✓ restores proof results on start (84 ms) 10:24:19 ✓ only re-enqueues unfinished jobs (76 ms) 10:24:19 ✓ saves job when enqueued (31 ms) 10:24:19 ✓ does not retain job if database fails to save (5 ms) 10:24:19 ✓ saves job result (45 ms) 10:24:19 ✓ saves result even if database fails to save (30 ms) 10:24:19 ✓ saves job error (39 ms) 10:24:19 ✓ saves job error even if database fails to save (33 ms) 10:24:19 ✓ does not save job result if job is unknown (6 ms) 10:24:19 ✓ does not save job error if job is unknown (7 ms) 10:24:19 ✓ cleans up old jobs periodically (434 ms) 10:24:19 10:24:19 Test Suites: 1 passed, 1 total 10:24:19 Tests: 96 passed, 96 total 10:24:19 Snapshots: 0 total 10:24:19 Time: 13.559 s 10:24:19 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 10:24:19 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?