Parent Log: http://ci.aztec-labs.com/85449a36e9425412 Command: ebfed4061172f77e yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/07aee119a58dd438ddfb5bd53e124e7269e6c651 Env: REF_NAME=gh-readonly-queue/next/pr-15083-9b801754ba98479ee645488c29d2fda1eecbfd85 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Mon Jun 16 21:05:26 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15083_amd64_x2-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 21:05:30 [21:05:30.943] INFO: prover-client:proving-broker Proving Broker started 21:05:30 [21:05:30.950] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0fff7a3e035d059 epochNumber=42 {"provingJobId":"1:BASE_PARITY:c0fff7a3e035d059"} 21:05:30 [21:05:30.953] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:231225ac15dec8de epochNumber=1 {"provingJobId":"1:BASE_PARITY:231225ac15dec8de"} 21:05:30 [21:05:30.997] INFO: prover-client:proving-broker Proving Broker started 21:05:30 [21:05:30.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7a1927f6759895f4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7a1927f6759895f4"} 21:05:31 [21:05:30.999] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d25ef77ac2f16eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d25ef77ac2f16eb"} 21:05:31 [21:05:31.001] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.002] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8cd08709328b03e6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8cd08709328b03e6"} 21:05:31 [21:05:31.003] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:8cd08709328b03e6 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:8cd08709328b03e6"} 21:05:31 [21:05:31.005] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.006] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:af4022613ebf5057 epochNumber=1 {"provingJobId":"1:BASE_PARITY:af4022613ebf5057"} 21:05:31 [21:05:31.007] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:af4022613ebf5057 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:af4022613ebf5057"} 21:05:31 [21:05:31.008] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:af4022613ebf5057 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:af4022613ebf5057"} 21:05:31 [21:05:31.008] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:af4022613ebf5057 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:af4022613ebf5057"} 21:05:31 [21:05:31.009] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.010] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b204263875c7cc0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9b204263875c7cc0"} 21:05:31 [21:05:31.035] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.037] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.038] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0f1db2a0df9ad1e0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0f1db2a0df9ad1e0"} 21:05:31 [21:05:31.038] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0f1db2a0df9ad1e0 {"provingJobId":"1:BASE_PARITY:0f1db2a0df9ad1e0"} 21:05:31 [21:05:31.039] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0f1db2a0df9ad1e0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0f1db2a0df9ad1e0"} 21:05:31 [21:05:31.039] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0f1db2a0df9ad1e0 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0f1db2a0df9ad1e0"} 21:05:31 [21:05:31.041] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.042] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:84da30808beeaf4a epochNumber=1 {"provingJobId":"1:BASE_PARITY:84da30808beeaf4a"} 21:05:31 [21:05:31.042] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:84da30808beeaf4a {"provingJobId":"1:BASE_PARITY:84da30808beeaf4a"} 21:05:31 [21:05:31.043] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:84da30808beeaf4a type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:84da30808beeaf4a"} 21:05:31 [21:05:31.044] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.045] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8a4c2655600b90d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8a4c2655600b90d3"} 21:05:31 [21:05:31.045] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8a4c2655600b90d3 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8a4c2655600b90d3"} 21:05:31 [21:05:31.046] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8a4c2655600b90d3 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:8a4c2655600b90d3"} 21:05:31 [21:05:31.048] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.049] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0ab12add3c2ad9cf epochNumber=1 {"provingJobId":"1:BASE_PARITY:0ab12add3c2ad9cf"} 21:05:31 [21:05:31.050] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0ab12add3c2ad9cf type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0ab12add3c2ad9cf"} 21:05:31 [21:05:31.050] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0ab12add3c2ad9cf type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:0ab12add3c2ad9cf"} 21:05:31 [21:05:31.051] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.052] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ebc64abb3474b8e3 epochNumber=0 {"provingJobId":"1:BASE_PARITY:ebc64abb3474b8e3"} 21:05:31 [21:05:31.052] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ebc64abb3474b8e3 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ebc64abb3474b8e3"} 21:05:31 [21:05:31.053] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d59931831af8b5b epochNumber=0 {"provingJobId":"1:BASE_PARITY:3d59931831af8b5b"} 21:05:31 [21:05:31.053] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ebc64abb3474b8e3 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ebc64abb3474b8e3"} 21:05:31 [21:05:31.053] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61df8cc33322e6be epochNumber=0 {"provingJobId":"1:BASE_PARITY:61df8cc33322e6be"} 21:05:31 [21:05:31.054] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.055] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.056] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e553d4c017e973b7 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e553d4c017e973b7"} 21:05:31 [21:05:31.056] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6f2f6513eb03f1b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6f2f6513eb03f1b6"} 21:05:31 [21:05:31.057] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.057] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c8f3c6ac0411badb epochNumber=1 {"provingJobId":"1:BASE_PARITY:c8f3c6ac0411badb"} 21:05:31 [21:05:31.058] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.058] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d39ea36243dfeb21 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d39ea36243dfeb21"} 21:05:31 [21:05:31.059] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d99301e6679d1a97 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d99301e6679d1a97"} 21:05:31 [21:05:31.059] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da15cbc0cf416fa1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:da15cbc0cf416fa1"} 21:05:31 [21:05:31.059] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:56fa5dc3a8c0ef45 epochNumber=1 {"provingJobId":"1:BASE_PARITY:56fa5dc3a8c0ef45"} 21:05:31 [21:05:31.060] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.060] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8a87cb465a5c6a80 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8a87cb465a5c6a80"} 21:05:31 [21:05:31.061] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33a0d3fcdb6f7ba4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:33a0d3fcdb6f7ba4"} 21:05:31 [21:05:31.061] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3ea62e748a09e4cf epochNumber=2 {"provingJobId":"1:BASE_PARITY:3ea62e748a09e4cf"} 21:05:31 [21:05:31.061] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:320a21f53df12f5c epochNumber=1 {"provingJobId":"1:BASE_PARITY:320a21f53df12f5c"} 21:05:31 [21:05:31.062] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.062] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9fee65ea70938904 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9fee65ea70938904"} 21:05:31 [21:05:31.063] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e558d881ecafdba0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e558d881ecafdba0"} 21:05:31 [21:05:31.063] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b225fc2c236a966f epochNumber=2 {"provingJobId":"1:BASE_PARITY:b225fc2c236a966f"} 21:05:31 [21:05:31.063] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:27479def67c67e75 epochNumber=1 {"provingJobId":"1:BASE_PARITY:27479def67c67e75"} 21:05:31 [21:05:31.064] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:24568228c01e9979 epochNumber=1 {"provingJobId":"1:BASE_PARITY:24568228c01e9979"} 21:05:31 [21:05:31.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7b602b072086fec1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7b602b072086fec1"} 21:05:31 [21:05:31.065] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:24568228c01e9979 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:24568228c01e9979"} 21:05:31 [21:05:31.067] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.067] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec89411a0697ef55 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ec89411a0697ef55"} 21:05:31 [21:05:31.067] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:43afbae833a3717b epochNumber=1 {"provingJobId":"1:BASE_PARITY:43afbae833a3717b"} 21:05:31 [21:05:31.067] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ec89411a0697ef55 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:ec89411a0697ef55"} 21:05:31 [21:05:31.068] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.069] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba1e7abee85d0127 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba1e7abee85d0127"} 21:05:31 [21:05:31.070] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:462e7aad8bac931c epochNumber=1 {"provingJobId":"1:BASE_PARITY:462e7aad8bac931c"} 21:05:31 [21:05:31.070] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ba1e7abee85d0127 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:ba1e7abee85d0127"} 21:05:31 [21:05:31.071] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.072] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:83fa1fc818e38076 epochNumber=1 {"provingJobId":"1:BASE_PARITY:83fa1fc818e38076"} 21:05:31 [21:05:31.083] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:83fa1fc818e38076 {"provingJobId":"1:BASE_PARITY:83fa1fc818e38076"} 21:05:31 [21:05:31.083] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:83fa1fc818e38076 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:83fa1fc818e38076"} 21:05:31 [21:05:31.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a4507da2d7037cbb epochNumber=1 {"provingJobId":"1:BASE_PARITY:a4507da2d7037cbb"} 21:05:31 [21:05:31.086] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:83fa1fc818e38076 has already been completed {"provingJobId":"1:BASE_PARITY:83fa1fc818e38076"} 21:05:31 [21:05:31.087] INFO: prover-client:proving-broker Proving Broker started 21:05:31 [21:05:31.088] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c8e6e2f9395125c epochNumber=1 {"provingJobId":"1:BASE_PARITY:1c8e6e2f9395125c"} 21:05:31 [21:05:31.089] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a9df4ec9098ee3c1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a9df4ec9098ee3c1"} 21:05:32 [21:05:32.141] INFO: prover-client:proving-broker Proving Broker started 21:05:32 [21:05:32.142] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1c8e6e2f9395125c settled=false {"provingJobId":"1:BASE_PARITY:1c8e6e2f9395125c","status":"pending"} 21:05:32 [21:05:32.143] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a9df4ec9098ee3c1 settled=false {"provingJobId":"1:BASE_PARITY:a9df4ec9098ee3c1","status":"pending"} 21:05:32 [21:05:32.143] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:1c8e6e2f9395125c type=BASE_PARITY startedAt=1750107931089 older agent has taken job {"provingJobId":"1:BASE_PARITY:1c8e6e2f9395125c"} 21:05:32 [21:05:32.144] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1c8e6e2f9395125c type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:1c8e6e2f9395125c"} 21:05:32 [21:05:32.145] INFO: prover-client:proving-broker Proving Broker started 21:05:32 [21:05:32.146] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d9fbe046c4fa084 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d9fbe046c4fa084"} 21:05:32 [21:05:32.146] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ffc6f49343bfe6ea epochNumber=2 {"provingJobId":"1:BASE_PARITY:ffc6f49343bfe6ea"} 21:05:33 [21:05:33.147] INFO: prover-client:proving-broker Proving Broker started 21:05:33 [21:05:33.148] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3d9fbe046c4fa084 settled=false {"provingJobId":"1:BASE_PARITY:3d9fbe046c4fa084","status":"pending"} 21:05:33 [21:05:33.148] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ffc6f49343bfe6ea settled=false {"provingJobId":"1:BASE_PARITY:ffc6f49343bfe6ea","status":"pending"} 21:05:33 [21:05:33.149] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3d9fbe046c4fa084 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:3d9fbe046c4fa084"} 21:05:33 [21:05:33.150] INFO: prover-client:proving-broker Proving Broker started 21:05:33 [21:05:33.151] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dcb9e3db6126c775 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dcb9e3db6126c775"} 21:05:33 [21:05:33.151] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:645056920de392b1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:645056920de392b1"} 21:05:34 [21:05:34.152] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.153] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dcb9e3db6126c775 settled=false {"provingJobId":"1:BASE_PARITY:dcb9e3db6126c775","status":"pending"} 21:05:34 [21:05:34.153] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:645056920de392b1 settled=false {"provingJobId":"1:BASE_PARITY:645056920de392b1","status":"pending"} 21:05:34 [21:05:34.154] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:dcb9e3db6126c775 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:dcb9e3db6126c775"} 21:05:34 [21:05:34.154] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:dcb9e3db6126c775 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:dcb9e3db6126c775"} 21:05:34 [21:05:34.155] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.156] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:50f8c27638dea37a epochNumber=1 {"provingJobId":"1:BASE_PARITY:50f8c27638dea37a"} 21:05:34 [21:05:34.156] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0e165480c374dea2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:0e165480c374dea2"} 21:05:34 [21:05:34.156] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:50f8c27638dea37a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:50f8c27638dea37a"} 21:05:34 [21:05:34.157] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0e165480c374dea2 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:0e165480c374dea2"} 21:05:34 [21:05:34.158] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.158] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4e3624c5ed0cc19 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4e3624c5ed0cc19"} 21:05:34 [21:05:34.158] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:69e3c930f8cdb315 epochNumber=2 {"provingJobId":"1:BASE_PARITY:69e3c930f8cdb315"} 21:05:34 [21:05:34.158] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c4e3624c5ed0cc19 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c4e3624c5ed0cc19"} 21:05:34 [21:05:34.159] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c4e3624c5ed0cc19 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c4e3624c5ed0cc19"} 21:05:34 [21:05:34.159] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:69e3c930f8cdb315 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:69e3c930f8cdb315"} 21:05:34 [21:05:34.160] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.160] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2382e1c6530aaf3a err=test error {"provingJoId":"1:BASE_PARITY:2382e1c6530aaf3a"} 21:05:34 [21:05:34.161] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.162] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:817366a38219ab5f not found {"provingJobId":"1:BASE_PARITY:817366a38219ab5f"} 21:05:34 [21:05:34.163] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.163] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:24b317630e9b5116 epochNumber=1 {"provingJobId":"1:BASE_PARITY:24b317630e9b5116"} 21:05:34 [21:05:34.164] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d43c06e6cbf3cceb epochNumber=1 {"provingJobId":"1:BASE_PARITY:d43c06e6cbf3cceb"} 21:05:34 [21:05:34.266] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d43c06e6cbf3cceb timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d43c06e6cbf3cceb"} 21:05:34 [21:05:34.273] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.273] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f6bc10f1d20ac695 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f6bc10f1d20ac695"} 21:05:34 [21:05:34.375] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f6bc10f1d20ac695 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f6bc10f1d20ac695"} 21:05:34 [21:05:34.384] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d15fc324e5a6609 epochNumber=10 {"provingJobId":"1:BASE_PARITY:6d15fc324e5a6609"} 21:05:34 [21:05:34.401] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:34 [21:05:34.512] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.513] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:97bc19315d3b3e82 epochNumber=1 {"provingJobId":"1:BASE_PARITY:97bc19315d3b3e82"} 21:05:34 [21:05:34.667] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:97bc19315d3b3e82 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:97bc19315d3b3e82"} 21:05:34 [21:05:34.716] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.717] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d3ad0073a1190626 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d3ad0073a1190626"} 21:05:34 [21:05:34.718] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d3ad0073a1190626 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:d3ad0073a1190626"} 21:05:34 [21:05:34.719] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.719] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:66940be238e67db7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:66940be238e67db7"} 21:05:34 [21:05:34.720] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:66940be238e67db7 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:66940be238e67db7"} 21:05:34 [21:05:34.720] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:66940be238e67db7 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:66940be238e67db7"} 21:05:34 [21:05:34.721] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.721] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7fdec616e9007d85 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7fdec616e9007d85"} 21:05:34 [21:05:34.722] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7fdec616e9007d85 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:7fdec616e9007d85"} 21:05:34 [21:05:34.722] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.723] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fbfc6e83c38dccaf epochNumber=1 {"provingJobId":"1:BASE_PARITY:fbfc6e83c38dccaf"} 21:05:34 [21:05:34.723] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fbfc6e83c38dccaf type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:fbfc6e83c38dccaf"} 21:05:34 [21:05:34.724] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:89e9c84cda85b53b epochNumber=3 {"provingJobId":"1:BASE_PARITY:89e9c84cda85b53b"} 21:05:34 [21:05:34.748] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:34 [21:05:34.749] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:fbfc6e83c38dccaf err=test error {"provingJoId":"1:BASE_PARITY:fbfc6e83c38dccaf"} 21:05:34 [21:05:34.751] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.751] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b13f8fc31b4f1383 settled=false {"provingJobId":"1:BASE_PARITY:b13f8fc31b4f1383","status":"pending"} 21:05:34 [21:05:34.752] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b30715138711612d settled=false {"provingJobId":"1:BASE_PARITY:b30715138711612d","status":"pending"} 21:05:34 [21:05:34.753] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.753] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2164abde0f38dfaf settled=true {"provingJobId":"1:BASE_PARITY:2164abde0f38dfaf","status":"fulfilled"} 21:05:34 [21:05:34.754] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:545e96670da4c84e settled=true {"provingJobId":"2:BASE_PARITY:545e96670da4c84e","status":"fulfilled"} 21:05:34 [21:05:34.755] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.755] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3b566993c62b0bf3 settled=true {"provingJobId":"1:BASE_PARITY:3b566993c62b0bf3","status":"fulfilled"} 21:05:34 [21:05:34.755] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1beba0ce1e44680f settled=false {"provingJobId":"1:BASE_PARITY:1beba0ce1e44680f","status":"pending"} 21:05:34 [21:05:34.756] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.757] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3ccdc741f5b2737b epochNumber=1 {"provingJobId":"1:BASE_PARITY:3ccdc741f5b2737b"} 21:05:34 [21:05:34.759] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.760] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:62ffdd3accf6124e epochNumber=1 {"provingJobId":"1:BASE_PARITY:62ffdd3accf6124e"} 21:05:34 [21:05:34.761] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:62ffdd3accf6124e: Error: db error: Error: db error 21:05:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:62ffdd3accf6124e"} 21:05:34 [21:05:34.763] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.763] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a0f30c5331ce923 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a0f30c5331ce923"} 21:05:34 [21:05:34.763] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2a0f30c5331ce923 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2a0f30c5331ce923"} 21:05:34 [21:05:34.764] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2a0f30c5331ce923 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2a0f30c5331ce923"} 21:05:34 [21:05:34.765] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.766] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1703b014eaf997b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1703b014eaf997b0"} 21:05:34 [21:05:34.766] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1703b014eaf997b0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1703b014eaf997b0"} 21:05:34 [21:05:34.766] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1703b014eaf997b0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1703b014eaf997b0"} 21:05:34 [21:05:34.767] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:1703b014eaf997b0: Error: db error 21:05:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:1703b014eaf997b0"} 21:05:34 [21:05:34.768] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.768] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dd90a3036e0779c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dd90a3036e0779c4"} 21:05:34 [21:05:34.769] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:dd90a3036e0779c4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:dd90a3036e0779c4"} 21:05:34 [21:05:34.769] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dd90a3036e0779c4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:dd90a3036e0779c4"} 21:05:34 [21:05:34.770] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.770] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34c6835aa08337a1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:34c6835aa08337a1"} 21:05:34 [21:05:34.771] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34c6835aa08337a1 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:34c6835aa08337a1"} 21:05:34 [21:05:34.771] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:34c6835aa08337a1 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:34c6835aa08337a1"} 21:05:34 [21:05:34.771] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:34c6835aa08337a1 jobErr=test error: Error: db error 21:05:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:34c6835aa08337a1"} 21:05:34 [21:05:34.772] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.773] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6cc3c98e23ce5c5f not found {"provingJobId":"1:BASE_PARITY:6cc3c98e23ce5c5f"} 21:05:34 [21:05:34.774] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.774] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8563aad0cfef8676 err=test error {"provingJoId":"1:BASE_PARITY:8563aad0cfef8676"} 21:05:34 [21:05:34.775] INFO: prover-client:proving-broker Proving Broker started 21:05:34 [21:05:34.786] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:74772ec3a03d0d59 epochNumber=1 {"provingJobId":"1:BASE_PARITY:74772ec3a03d0d59"} 21:05:34 [21:05:34.786] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:74772ec3a03d0d59 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:74772ec3a03d0d59"} 21:05:34 [21:05:34.787] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:74772ec3a03d0d59 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:74772ec3a03d0d59"} 21:05:34 [21:05:34.798] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:871ecacbf7939979 epochNumber=2 {"provingJobId":"2:BASE_PARITY:871ecacbf7939979"} 21:05:34 [21:05:34.798] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:871ecacbf7939979 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:871ecacbf7939979"} 21:05:34 [21:05:34.799] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:871ecacbf7939979 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:871ecacbf7939979"} 21:05:34 [21:05:34.861] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:774da4ab72cccb90 epochNumber=3 {"provingJobId":"3:BASE_PARITY:774da4ab72cccb90"} 21:05:34 [21:05:34.877] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:34 [21:05:34.923] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:69ddd96f2bfb695f epochNumber=4 {"provingJobId":"4:BASE_PARITY:69ddd96f2bfb695f"} 21:05:34 [21:05:34.928] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:34 [21:05:34.985] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:b4a3d000736dc516 epochNumber=5 {"provingJobId":"5:BASE_PARITY:b4a3d000736dc516"} 21:05:35 [21:05:35.004] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:35 [21:05:35.039] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.039] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9863b7033d9d855d epochNumber=42 {"provingJobId":"1:BASE_PARITY:9863b7033d9d855d"} 21:05:35 [21:05:35.040] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testHM6kkB/42 with map size 1073741824 21:05:35 [21:05:35.044] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.070] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:02ad6c9e876960e0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:02ad6c9e876960e0"} 21:05:35 [21:05:35.074] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.075] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0fa89936de497f6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0fa89936de497f6"} 21:05:35 [21:05:35.075] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test95htMV/1 with map size 1073741824 21:05:35 [21:05:35.077] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.098] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0eb2610c33a862fe epochNumber=1 {"provingJobId":"1:BASE_PARITY:0eb2610c33a862fe"} 21:05:35 [21:05:35.106] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ae50686d02fd8066 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ae50686d02fd8066"} 21:05:35 [21:05:35.108] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzB5mGx/1 with map size 1073741824 21:05:35 [21:05:35.110] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.120] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ae50686d02fd8066 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ae50686d02fd8066"} 21:05:35 [21:05:35.123] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.123] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:734ce7a31f34e14c epochNumber=1 {"provingJobId":"1:BASE_PARITY:734ce7a31f34e14c"} 21:05:35 [21:05:35.124] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQVveGn/1 with map size 1073741824 21:05:35 [21:05:35.126] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.134] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:734ce7a31f34e14c epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:734ce7a31f34e14c"} 21:05:35 [21:05:35.135] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:734ce7a31f34e14c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:734ce7a31f34e14c"} 21:05:35 [21:05:35.138] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:734ce7a31f34e14c epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:734ce7a31f34e14c"} 21:05:35 [21:05:35.142] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.143] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ddb59f7553e0c850 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ddb59f7553e0c850"} 21:05:35 [21:05:35.144] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRSkiln/1 with map size 1073741824 21:05:35 [21:05:35.145] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.157] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.161] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.161] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:81723dfe668a4917 epochNumber=1 {"provingJobId":"1:BASE_PARITY:81723dfe668a4917"} 21:05:35 [21:05:35.162] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyIkTzN/1 with map size 1073741824 21:05:35 [21:05:35.163] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.171] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:81723dfe668a4917 {"provingJobId":"1:BASE_PARITY:81723dfe668a4917"} 21:05:35 [21:05:35.171] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:81723dfe668a4917 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:81723dfe668a4917"} 21:05:35 [21:05:35.171] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:81723dfe668a4917 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:81723dfe668a4917"} 21:05:35 [21:05:35.177] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.177] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d6bc1cedaa5f94b9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d6bc1cedaa5f94b9"} 21:05:35 [21:05:35.178] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRcLQFh/1 with map size 1073741824 21:05:35 [21:05:35.179] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.192] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d6bc1cedaa5f94b9 {"provingJobId":"1:BASE_PARITY:d6bc1cedaa5f94b9"} 21:05:35 [21:05:35.192] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d6bc1cedaa5f94b9 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:d6bc1cedaa5f94b9"} 21:05:35 [21:05:35.201] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.202] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:581a493eb834fe6d epochNumber=1 {"provingJobId":"1:BASE_PARITY:581a493eb834fe6d"} 21:05:35 [21:05:35.202] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOCHQX8/1 with map size 1073741824 21:05:35 [21:05:35.204] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.218] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:581a493eb834fe6d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:581a493eb834fe6d"} 21:05:35 [21:05:35.219] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:581a493eb834fe6d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:581a493eb834fe6d"} 21:05:35 [21:05:35.227] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.228] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f36c6999c153703 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f36c6999c153703"} 21:05:35 [21:05:35.229] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnlqmqt/1 with map size 1073741824 21:05:35 [21:05:35.231] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.250] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3f36c6999c153703 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3f36c6999c153703"} 21:05:35 [21:05:35.251] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3f36c6999c153703 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3f36c6999c153703"} 21:05:35 [21:05:35.260] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.261] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f590a4efbade70a0 epochNumber=0 {"provingJobId":"1:BASE_PARITY:f590a4efbade70a0"} 21:05:35 [21:05:35.262] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testYBqeR6/0 with map size 1073741824 21:05:35 [21:05:35.264] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.280] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f590a4efbade70a0 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:f590a4efbade70a0"} 21:05:35 [21:05:35.281] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ad7371f5636c2a68 epochNumber=0 {"provingJobId":"1:BASE_PARITY:ad7371f5636c2a68"} 21:05:35 [21:05:35.282] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f590a4efbade70a0 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:f590a4efbade70a0"} 21:05:35 [21:05:35.282] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67de135648f390a8 epochNumber=0 {"provingJobId":"1:BASE_PARITY:67de135648f390a8"} 21:05:35 [21:05:35.295] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.298] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.298] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:921ede0eddc1b320 epochNumber=2 {"provingJobId":"1:BASE_PARITY:921ede0eddc1b320"} 21:05:35 [21:05:35.299] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test6y3yA0/2 with map size 1073741824 21:05:35 [21:05:35.300] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.316] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6dc5211200e058ab epochNumber=1 {"provingJobId":"1:BASE_PARITY:6dc5211200e058ab"} 21:05:35 [21:05:35.317] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6y3yA0/1 with map size 1073741824 21:05:35 [21:05:35.319] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.338] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.339] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34d20d4c7c501900 epochNumber=1 {"provingJobId":"1:BASE_PARITY:34d20d4c7c501900"} 21:05:35 [21:05:35.340] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testL62Ukw/1 with map size 1073741824 21:05:35 [21:05:35.341] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.365] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.366] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7026308f81ee0891 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7026308f81ee0891"} 21:05:35 [21:05:35.367] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtjDymq/1 with map size 1073741824 21:05:35 [21:05:35.369] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.386] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:84b71c0151d5cf94 epochNumber=1 {"provingJobId":"1:BASE_PARITY:84b71c0151d5cf94"} 21:05:35 [21:05:35.391] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2bc2f698deef39cb epochNumber=2 {"provingJobId":"1:BASE_PARITY:2bc2f698deef39cb"} 21:05:35 [21:05:35.392] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtjDymq/2 with map size 1073741824 21:05:35 [21:05:35.393] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.410] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:25945e034d1e7385 epochNumber=1 {"provingJobId":"1:BASE_PARITY:25945e034d1e7385"} 21:05:35 [21:05:35.419] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:251410c885b36e44 epochNumber=1 {"provingJobId":"1:BASE_PARITY:251410c885b36e44"} 21:05:35 [21:05:35.420] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBhYArx/1 with map size 1073741824 21:05:35 [21:05:35.422] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.440] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:45af48feec166291 epochNumber=1 {"provingJobId":"1:BASE_PARITY:45af48feec166291"} 21:05:35 [21:05:35.448] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9f353105d2252157 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9f353105d2252157"} 21:05:35 [21:05:35.449] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBhYArx/2 with map size 1073741824 21:05:35 [21:05:35.450] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.473] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fdeb6d67fa2f4740 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fdeb6d67fa2f4740"} 21:05:35 [21:05:35.483] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.484] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a001cc84d79fd856 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a001cc84d79fd856"} 21:05:35 [21:05:35.485] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfnzFmc/1 with map size 1073741824 21:05:35 [21:05:35.486] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.503] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:56006c352fdc10c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:56006c352fdc10c4"} 21:05:35 [21:05:35.509] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7c95ff3bd6ebbe2b epochNumber=2 {"provingJobId":"1:BASE_PARITY:7c95ff3bd6ebbe2b"} 21:05:35 [21:05:35.509] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfnzFmc/2 with map size 1073741824 21:05:35 [21:05:35.511] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.528] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8e51ded5fb5ed84d epochNumber=1 {"provingJobId":"1:BASE_PARITY:8e51ded5fb5ed84d"} 21:05:35 [21:05:35.544] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.545] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d186ead3ac843ae2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d186ead3ac843ae2"} 21:05:35 [21:05:35.547] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2WsIud/1 with map size 1073741824 21:05:35 [21:05:35.549] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.567] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed9b6328d17d3178 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed9b6328d17d3178"} 21:05:35 [21:05:35.573] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d186ead3ac843ae2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d186ead3ac843ae2"} 21:05:35 [21:05:35.587] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.588] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7297244195369348 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7297244195369348"} 21:05:35 [21:05:35.589] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzl4phL/1 with map size 1073741824 21:05:35 [21:05:35.593] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.612] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:497d763187ce10bb epochNumber=1 {"provingJobId":"1:BASE_PARITY:497d763187ce10bb"} 21:05:35 [21:05:35.617] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7297244195369348 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:7297244195369348"} 21:05:35 [21:05:35.627] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.628] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:14528de1ee28f60b epochNumber=1 {"provingJobId":"1:BASE_PARITY:14528de1ee28f60b"} 21:05:35 [21:05:35.628] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQMTxwK/1 with map size 1073741824 21:05:35 [21:05:35.632] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.650] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f775b0c7fa009985 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f775b0c7fa009985"} 21:05:35 [21:05:35.656] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:14528de1ee28f60b type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:14528de1ee28f60b"} 21:05:35 [21:05:35.660] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.661] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:920542eb5e62557b epochNumber=1 {"provingJobId":"1:BASE_PARITY:920542eb5e62557b"} 21:05:35 [21:05:35.662] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test20aaEh/1 with map size 1073741824 21:05:35 [21:05:35.663] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.694] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:920542eb5e62557b {"provingJobId":"1:BASE_PARITY:920542eb5e62557b"} 21:05:35 [21:05:35.695] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:920542eb5e62557b type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:920542eb5e62557b"} 21:05:35 [21:05:35.702] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ce711886663ec20e epochNumber=1 {"provingJobId":"1:BASE_PARITY:ce711886663ec20e"} 21:05:35 [21:05:35.707] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:920542eb5e62557b has already been completed {"provingJobId":"1:BASE_PARITY:920542eb5e62557b"} 21:05:35 [21:05:35.710] INFO: prover-client:proving-broker Proving Broker started 21:05:35 [21:05:35.711] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dbd7955d7fb9ba53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dbd7955d7fb9ba53"} 21:05:35 [21:05:35.712] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhGGl4w/1 with map size 1073741824 21:05:35 [21:05:35.714] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:35 [21:05:35.732] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:12164c5877e1bc7a epochNumber=2 {"provingJobId":"1:BASE_PARITY:12164c5877e1bc7a"} 21:05:35 [21:05:35.733] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhGGl4w/2 with map size 1073741824 21:05:35 [21:05:35.735] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:36 [21:05:36.804] INFO: prover-client:proving-broker Proving Broker started 21:05:36 [21:05:36.811] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dbd7955d7fb9ba53 settled=false {"provingJobId":"1:BASE_PARITY:dbd7955d7fb9ba53","status":"pending"} 21:05:36 [21:05:36.812] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:12164c5877e1bc7a settled=false {"provingJobId":"1:BASE_PARITY:12164c5877e1bc7a","status":"pending"} 21:05:36 [21:05:36.813] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:dbd7955d7fb9ba53 type=BASE_PARITY startedAt=1750107935751 older agent has taken job {"provingJobId":"1:BASE_PARITY:dbd7955d7fb9ba53"} 21:05:36 [21:05:36.813] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:dbd7955d7fb9ba53 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:dbd7955d7fb9ba53"} 21:05:36 [21:05:36.817] INFO: prover-client:proving-broker Proving Broker started 21:05:36 [21:05:36.817] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2eb1a0d7ec1d883f epochNumber=1 {"provingJobId":"1:BASE_PARITY:2eb1a0d7ec1d883f"} 21:05:36 [21:05:36.818] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaJnmHP/1 with map size 1073741824 21:05:36 [21:05:36.819] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:36 [21:05:36.837] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:85c63ba594c4b5ec epochNumber=2 {"provingJobId":"1:BASE_PARITY:85c63ba594c4b5ec"} 21:05:36 [21:05:36.840] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testaJnmHP/2 with map size 1073741824 21:05:36 [21:05:36.842] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:37 [21:05:37.866] INFO: prover-client:proving-broker Proving Broker started 21:05:37 [21:05:37.868] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2eb1a0d7ec1d883f settled=false {"provingJobId":"1:BASE_PARITY:2eb1a0d7ec1d883f","status":"pending"} 21:05:37 [21:05:37.870] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:85c63ba594c4b5ec settled=false {"provingJobId":"1:BASE_PARITY:85c63ba594c4b5ec","status":"pending"} 21:05:37 [21:05:37.871] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2eb1a0d7ec1d883f type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:2eb1a0d7ec1d883f"} 21:05:37 [21:05:37.879] INFO: prover-client:proving-broker Proving Broker started 21:05:37 [21:05:37.880] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:97ad48533868f6ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:97ad48533868f6ea"} 21:05:37 [21:05:37.881] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbPLKz0/1 with map size 1073741824 21:05:37 [21:05:37.885] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:37 [21:05:37.905] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e1a91564b0d86e0d epochNumber=2 {"provingJobId":"1:BASE_PARITY:e1a91564b0d86e0d"} 21:05:37 [21:05:37.906] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testbPLKz0/2 with map size 1073741824 21:05:37 [21:05:37.908] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:38 [21:05:38.930] INFO: prover-client:proving-broker Proving Broker started 21:05:38 [21:05:38.932] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:97ad48533868f6ea settled=false {"provingJobId":"1:BASE_PARITY:97ad48533868f6ea","status":"pending"} 21:05:38 [21:05:38.935] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e1a91564b0d86e0d settled=false {"provingJobId":"1:BASE_PARITY:e1a91564b0d86e0d","status":"pending"} 21:05:38 [21:05:38.936] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:97ad48533868f6ea type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:97ad48533868f6ea"} 21:05:38 [21:05:38.937] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:97ad48533868f6ea type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:97ad48533868f6ea"} 21:05:38 [21:05:38.948] INFO: prover-client:proving-broker Proving Broker started 21:05:38 [21:05:38.949] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0e63bd63787311a7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0e63bd63787311a7"} 21:05:38 [21:05:38.949] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJSTzgt/1 with map size 1073741824 21:05:38 [21:05:38.952] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:38 [21:05:38.973] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b9bdd71896482c6e epochNumber=2 {"provingJobId":"1:BASE_PARITY:b9bdd71896482c6e"} 21:05:38 [21:05:38.974] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testJSTzgt/2 with map size 1073741824 21:05:38 [21:05:38.980] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.000] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0e63bd63787311a7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:0e63bd63787311a7"} 21:05:39 [21:05:39.006] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b9bdd71896482c6e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b9bdd71896482c6e"} 21:05:39 [21:05:39.016] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.017] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d119e600f68089a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d119e600f68089a4"} 21:05:39 [21:05:39.018] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyNMvUy/1 with map size 1073741824 21:05:39 [21:05:39.021] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.044] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec6fe3623d66a960 epochNumber=2 {"provingJobId":"1:BASE_PARITY:ec6fe3623d66a960"} 21:05:39 [21:05:39.045] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyNMvUy/2 with map size 1073741824 21:05:39 [21:05:39.047] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.065] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d119e600f68089a4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d119e600f68089a4"} 21:05:39 [21:05:39.066] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d119e600f68089a4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d119e600f68089a4"} 21:05:39 [21:05:39.072] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ec6fe3623d66a960 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ec6fe3623d66a960"} 21:05:39 [21:05:39.090] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.091] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d4a9040390117a0d err=test error {"provingJoId":"1:BASE_PARITY:d4a9040390117a0d"} 21:05:39 [21:05:39.101] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.103] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2c1eec285f700f0b not found {"provingJobId":"1:BASE_PARITY:2c1eec285f700f0b"} 21:05:39 [21:05:39.107] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7921dfe60147818f epochNumber=1 {"provingJobId":"1:BASE_PARITY:7921dfe60147818f"} 21:05:39 [21:05:39.109] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDP4hdT/1 with map size 1073741824 21:05:39 [21:05:39.112] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.141] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.142] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8344bb208387f2ad epochNumber=1 {"provingJobId":"1:BASE_PARITY:8344bb208387f2ad"} 21:05:39 [21:05:39.143] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdzaH1n/1 with map size 1073741824 21:05:39 [21:05:39.145] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.269] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8344bb208387f2ad timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:8344bb208387f2ad"} 21:05:39 [21:05:39.274] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.275] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:96ec4a14a973cfa0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:96ec4a14a973cfa0"} 21:05:39 [21:05:39.277] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjxawdm/1 with map size 1073741824 21:05:39 [21:05:39.281] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.403] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:96ec4a14a973cfa0 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:96ec4a14a973cfa0"} 21:05:39 [21:05:39.408] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:378792c34d72c540 epochNumber=10 {"provingJobId":"1:BASE_PARITY:378792c34d72c540"} 21:05:39 [21:05:39.410] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testjxawdm/10 with map size 1073741824 21:05:39 [21:05:39.412] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.428] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:39 [21:05:39.429] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 21:05:39 [21:05:39.558] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.559] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eec34e21001e74f7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:eec34e21001e74f7"} 21:05:39 [21:05:39.561] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxhHbFC/1 with map size 1073741824 21:05:39 [21:05:39.562] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.735] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:eec34e21001e74f7 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:eec34e21001e74f7"} 21:05:39 [21:05:39.786] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.789] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:603a896fdaf7b637 epochNumber=1 {"provingJobId":"1:BASE_PARITY:603a896fdaf7b637"} 21:05:39 [21:05:39.791] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgC6Jzq/1 with map size 1073741824 21:05:39 [21:05:39.794] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.814] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:603a896fdaf7b637 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:603a896fdaf7b637"} 21:05:39 [21:05:39.820] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.822] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a170dc0c8f264007 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a170dc0c8f264007"} 21:05:39 [21:05:39.823] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyJw5gs/1 with map size 1073741824 21:05:39 [21:05:39.825] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.843] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a170dc0c8f264007 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:a170dc0c8f264007"} 21:05:39 [21:05:39.844] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a170dc0c8f264007 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:a170dc0c8f264007"} 21:05:39 [21:05:39.855] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.856] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:008369c2d45db25e epochNumber=1 {"provingJobId":"1:BASE_PARITY:008369c2d45db25e"} 21:05:39 [21:05:39.859] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqtlng7/1 with map size 1073741824 21:05:39 [21:05:39.862] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.880] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:008369c2d45db25e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:008369c2d45db25e"} 21:05:39 [21:05:39.890] INFO: prover-client:proving-broker Proving Broker started 21:05:39 [21:05:39.891] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6087396699eb2661 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6087396699eb2661"} 21:05:39 [21:05:39.892] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4jmWQa/1 with map size 1073741824 21:05:39 [21:05:39.895] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.916] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6087396699eb2661 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:6087396699eb2661"} 21:05:39 [21:05:39.918] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7ecc0e394910836e epochNumber=3 {"provingJobId":"1:BASE_PARITY:7ecc0e394910836e"} 21:05:39 [21:05:39.919] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test4jmWQa/3 with map size 1073741824 21:05:39 [21:05:39.920] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:39 [21:05:39.944] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:39 [21:05:39.945] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 21:05:39 [21:05:39.964] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:6087396699eb2661 err=test error {"provingJoId":"1:BASE_PARITY:6087396699eb2661"} 21:05:39 [21:05:39.971] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvouU8y/1 with map size 1073741824 21:05:39 [21:05:39.977] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.003] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testvouU8y/2 with map size 1073741824 21:05:40 [21:05:40.006] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.027] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.029] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:692dec48c2cd3577 settled=false {"provingJobId":"1:BASE_PARITY:692dec48c2cd3577","status":"pending"} 21:05:40 [21:05:40.030] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7dd701e8884edf54 settled=false {"provingJobId":"1:BASE_PARITY:7dd701e8884edf54","status":"pending"} 21:05:40 [21:05:40.037] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testh686Fd/1 with map size 1073741824 21:05:40 [21:05:40.039] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.057] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testh686Fd/2 with map size 1073741824 21:05:40 [21:05:40.060] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.089] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.091] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f4f926bd40abf240 settled=true {"provingJobId":"1:BASE_PARITY:f4f926bd40abf240","status":"fulfilled"} 21:05:40 [21:05:40.093] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:ef0cd0ff8bc88fa8 settled=true {"provingJobId":"2:BASE_PARITY:ef0cd0ff8bc88fa8","status":"fulfilled"} 21:05:40 [21:05:40.098] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwEpC6K/1 with map size 1073741824 21:05:40 [21:05:40.101] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.126] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwEpC6K/2 with map size 1073741824 21:05:40 [21:05:40.130] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.147] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.149] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d5841881d11e8bf5 settled=true {"provingJobId":"1:BASE_PARITY:d5841881d11e8bf5","status":"fulfilled"} 21:05:40 [21:05:40.150] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b27bb53a22196ebf settled=false {"provingJobId":"1:BASE_PARITY:b27bb53a22196ebf","status":"pending"} 21:05:40 [21:05:40.154] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.155] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1793fc0460cbc69f epochNumber=1 {"provingJobId":"1:BASE_PARITY:1793fc0460cbc69f"} 21:05:40 [21:05:40.157] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testn7zxxc/1 with map size 1073741824 21:05:40 [21:05:40.159] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.181] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.182] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20e5defc8b8d85c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:20e5defc8b8d85c1"} 21:05:40 [21:05:40.183] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:20e5defc8b8d85c1: Error: db error: Error: db error 21:05:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:20e5defc8b8d85c1"} 21:05:40 [21:05:40.186] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:268a3752367911e2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:268a3752367911e2"} 21:05:40 [21:05:40.187] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8o7jHC/1 with map size 1073741824 21:05:40 [21:05:40.190] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.209] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:268a3752367911e2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:268a3752367911e2"} 21:05:40 [21:05:40.209] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:268a3752367911e2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:268a3752367911e2"} 21:05:40 [21:05:40.224] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.225] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bc0b7efa524eb2a2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bc0b7efa524eb2a2"} 21:05:40 [21:05:40.225] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWrR38l/1 with map size 1073741824 21:05:40 [21:05:40.227] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.246] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bc0b7efa524eb2a2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bc0b7efa524eb2a2"} 21:05:40 [21:05:40.247] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bc0b7efa524eb2a2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:bc0b7efa524eb2a2"} 21:05:40 [21:05:40.248] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:bc0b7efa524eb2a2: Error: db error 21:05:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:bc0b7efa524eb2a2"} 21:05:40 [21:05:40.252] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.253] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:beea5f414f1d43fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:beea5f414f1d43fb"} 21:05:40 [21:05:40.254] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvkwfJk/1 with map size 1073741824 21:05:40 [21:05:40.255] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.273] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:beea5f414f1d43fb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:beea5f414f1d43fb"} 21:05:40 [21:05:40.273] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:beea5f414f1d43fb type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:beea5f414f1d43fb"} 21:05:40 [21:05:40.285] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.286] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34f923af5bcbe02f epochNumber=1 {"provingJobId":"1:BASE_PARITY:34f923af5bcbe02f"} 21:05:40 [21:05:40.287] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbWWhHp/1 with map size 1073741824 21:05:40 [21:05:40.290] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.310] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34f923af5bcbe02f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:34f923af5bcbe02f"} 21:05:40 [21:05:40.310] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:34f923af5bcbe02f type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:34f923af5bcbe02f"} 21:05:40 [21:05:40.311] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:34f923af5bcbe02f jobErr=test error: Error: db error 21:05:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:34f923af5bcbe02f"} 21:05:40 [21:05:40.315] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.315] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e2d974a86876eace not found {"provingJobId":"1:BASE_PARITY:e2d974a86876eace"} 21:05:40 [21:05:40.320] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.321] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:6f4f16c952402233 err=test error {"provingJoId":"1:BASE_PARITY:6f4f16c952402233"} 21:05:40 [21:05:40.327] INFO: prover-client:proving-broker Proving Broker started 21:05:40 [21:05:40.337] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da7e8b5628d6939e epochNumber=1 {"provingJobId":"1:BASE_PARITY:da7e8b5628d6939e"} 21:05:40 [21:05:40.338] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5GHY08/1 with map size 1073741824 21:05:40 [21:05:40.342] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.363] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:da7e8b5628d6939e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:da7e8b5628d6939e"} 21:05:40 [21:05:40.364] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:da7e8b5628d6939e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:da7e8b5628d6939e"} 21:05:40 [21:05:40.380] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:899806c68b24b92b epochNumber=2 {"provingJobId":"2:BASE_PARITY:899806c68b24b92b"} 21:05:40 [21:05:40.381] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5GHY08/2 with map size 1073741824 21:05:40 [21:05:40.384] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.401] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:899806c68b24b92b type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:899806c68b24b92b"} 21:05:40 [21:05:40.402] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:899806c68b24b92b type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:899806c68b24b92b"} 21:05:40 [21:05:40.468] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:328328d69641410a epochNumber=3 {"provingJobId":"3:BASE_PARITY:328328d69641410a"} 21:05:40 [21:05:40.469] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test5GHY08/3 with map size 1073741824 21:05:40 [21:05:40.473] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.504] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:40 [21:05:40.504] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 21:05:40 [21:05:40.559] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:87672465573a2372 epochNumber=4 {"provingJobId":"4:BASE_PARITY:87672465573a2372"} 21:05:40 [21:05:40.562] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test5GHY08/4 with map size 1073741824 21:05:40 [21:05:40.571] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.616] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:40 [21:05:40.616] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 21:05:40 [21:05:40.660] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:19cd463a436ca881 epochNumber=5 {"provingJobId":"5:BASE_PARITY:19cd463a436ca881"} 21:05:40 [21:05:40.661] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test5GHY08/5 with map size 1073741824 21:05:40 [21:05:40.666] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 21:05:40 [21:05:40.694] INFO: prover-client:proving-broker Cleaned up jobs=1 21:05:40 [21:05:40.695] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 21:05:40 PASS src/proving_broker/proving_broker.test.ts (12.893 s) 21:05:40 ProvingBroker 21:05:40 Producer API 21:05:40 ✓ refuses stale jobs (55 ms) 21:05:40 ✓ enqueues jobs (4 ms) 21:05:40 ✓ ignores duplicate jobs (4 ms) 21:05:40 ✓ reports correct status when enqueuing repeat jobs (4 ms) 21:05:40 ✓ throws an error in case of duplicate job IDs (25 ms) 21:05:40 ✓ returns not-found status for non-existing jobs (1 ms) 21:05:40 ✓ cancels jobs in queue (3 ms) 21:05:40 ✓ cancels jobs in-progress (2 ms) 21:05:40 ✓ returns job result if successful (3 ms) 21:05:40 ✓ returns job error if failed (4 ms) 21:05:40 ✓ correctly returns job status for concurrent writes (3 ms) 21:05:40 ✓ refuses stale jobs (36 ms) 21:05:40 ✓ enqueues jobs (31 ms) 21:05:40 ✓ ignores duplicate jobs (17 ms) 21:05:40 ✓ reports correct status when enqueuing repeat jobs (19 ms) 21:05:40 ✓ throws an error in case of duplicate job IDs (16 ms) 21:05:40 ✓ returns not-found status for non-existing jobs (3 ms) 21:05:40 ✓ cancels jobs in queue (16 ms) 21:05:40 ✓ cancels jobs in-progress (23 ms) 21:05:40 ✓ returns job result if successful (26 ms) 21:05:40 ✓ returns job error if failed (33 ms) 21:05:40 ✓ correctly returns job status for concurrent writes (34 ms) 21:05:40 Consumer API 21:05:40 ✓ returns undefined if no jobs are available (1 ms) 21:05:40 ✓ returns jobs in priority order (1 ms) 21:05:40 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 21:05:40 ✓ returns a job if it is in the allowList (2 ms) 21:05:40 ✓ returns the most important job if it is in the allowList (2 ms) 21:05:40 ✓ returns any job if filter is empty (2 ms) 21:05:40 ✓ returns a new job when reporting job success (2 ms) 21:05:40 ✓ returns a new job when reporting permanent error (2 ms) 21:05:40 ✓ returns a new job when reporting retry-able error (3 ms) 21:05:40 ✓ returns a new job when reporting progress if current one is cancelled (16 ms) 21:05:40 ✓ returns a new job if job is already in progress elsewhere (1057 ms) 21:05:40 ✓ avoids sending the same job to a new agent after a restart (1005 ms) 21:05:40 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 21:05:40 ✓ tracks job result if in progress (2 ms) 21:05:40 ✓ tracks job result even if job is in queue (2 ms) 21:05:40 ✓ ignores reported job error if unknown job (1 ms) 21:05:40 ✓ ignores job result if unknown job (1 ms) 21:05:40 ✓ returns undefined if no jobs are available (2 ms) 21:05:40 ✓ returns jobs in priority order (40 ms) 21:05:40 ✓ returns undefined if no jobs are available for the given allowList (26 ms) 21:05:40 ✓ returns a job if it is in the allowList (53 ms) 21:05:40 ✓ returns the most important job if it is in the allowList (63 ms) 21:05:40 ✓ returns any job if filter is empty (61 ms) 21:05:40 ✓ returns a new job when reporting job success (42 ms) 21:05:40 ✓ returns a new job when reporting permanent error (41 ms) 21:05:40 ✓ returns a new job when reporting retry-able error (32 ms) 21:05:40 ✓ returns a new job when reporting progress if current one is cancelled (51 ms) 21:05:40 ✓ returns a new job if job is already in progress elsewhere (1106 ms) 21:05:40 ✓ avoids sending the same job to a new agent after a restart (1057 ms) 21:05:40 ✓ avoids sending a completed job to a new agent after a restart (1071 ms) 21:05:40 ✓ tracks job result if in progress (70 ms) 21:05:40 ✓ tracks job result even if job is in queue (71 ms) 21:05:40 ✓ ignores reported job error if unknown job (6 ms) 21:05:40 ✓ ignores job result if unknown job (9 ms) 21:05:40 Timeouts 21:05:40 ✓ tracks in progress jobs (2 ms) 21:05:40 ✓ re-enqueues jobs that time out (108 ms) 21:05:40 ✓ cancel stale jobs that time out (240 ms) 21:05:40 ✓ keeps the jobs in progress while it is alive (203 ms) 21:05:40 ✓ tracks in progress jobs (35 ms) 21:05:40 ✓ re-enqueues jobs that time out (132 ms) 21:05:40 ✓ cancel stale jobs that time out (284 ms) 21:05:40 ✓ keeps the jobs in progress while it is alive (227 ms) 21:05:40 Retries 21:05:40 ✓ retries jobs (2 ms) 21:05:40 ✓ retries up to a maximum number of times (2 ms) 21:05:40 ✓ passing retry=false does not retry (1 ms) 21:05:40 ✓ does not retry if job is stale (28 ms) 21:05:40 ✓ retries jobs (31 ms) 21:05:40 ✓ retries up to a maximum number of times (36 ms) 21:05:40 ✓ passing retry=false does not retry (34 ms) 21:05:40 ✓ does not retry if job is stale (78 ms) 21:05:40 Database management 21:05:40 ✓ re-enqueues proof requests on start (3 ms) 21:05:40 ✓ restores proof results on start (1 ms) 21:05:40 ✓ only re-enqueues unfinished jobs (1 ms) 21:05:40 ✓ saves job when enqueued (3 ms) 21:05:40 ✓ does not retain job if database fails to save (3 ms) 21:05:40 ✓ saves job result (2 ms) 21:05:40 ✓ saves result even if database fails to save (2 ms) 21:05:40 ✓ saves job error (1 ms) 21:05:40 ✓ saves job error even if database fails to save (2 ms) 21:05:40 ✓ does not save job result if job is unknown (1 ms) 21:05:40 ✓ does not save job error if job is unknown (2 ms) 21:05:40 ✓ cleans up old jobs periodically (261 ms) 21:05:40 ✓ re-enqueues proof requests on start (66 ms) 21:05:40 ✓ restores proof results on start (60 ms) 21:05:40 ✓ only re-enqueues unfinished jobs (57 ms) 21:05:40 ✓ saves job when enqueued (25 ms) 21:05:40 ✓ does not retain job if database fails to save (6 ms) 21:05:40 ✓ saves job result (36 ms) 21:05:40 ✓ saves result even if database fails to save (29 ms) 21:05:40 ✓ saves job error (31 ms) 21:05:40 ✓ saves job error even if database fails to save (32 ms) 21:05:40 ✓ does not save job result if job is unknown (3 ms) 21:05:40 ✓ does not save job error if job is unknown (5 ms) 21:05:40 ✓ cleans up old jobs periodically (424 ms) 21:05:40 21:05:40 Test Suites: 1 passed, 1 total 21:05:40 Tests: 96 passed, 96 total 21:05:40 Snapshots: 0 total 21:05:40 Time: 13.016 s 21:05:40 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 21:05:40 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?