ACI ·

Parent Log: http://ci.aztec-labs.com/e0d14622d89aaee4 Command: 55a5c1e5cd3f3ca6 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/28bf32bfc2566751e8baa0a9125a9db20d474bee Env: REF_NAME=gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Fri Jun 20 15:25:17 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15154_amd64_x3-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 15:25:22 [15:25:22.476] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.482] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b498e47c20133b2a epochNumber=42 {"provingJobId":"1:BASE_PARITY:b498e47c20133b2a"} 15:25:22 [15:25:22.486] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:1b88d1a960527658 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1b88d1a960527658"} 15:25:22 [15:25:22.513] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.514] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1ec673c390b7aa1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a1ec673c390b7aa1"} 15:25:22 [15:25:22.515] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5f3e040c2eaf3158 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5f3e040c2eaf3158"} 15:25:22 [15:25:22.517] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.518] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b2474854ef7d6691 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b2474854ef7d6691"} 15:25:22 [15:25:22.519] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b2474854ef7d6691 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b2474854ef7d6691"} 15:25:22 [15:25:22.521] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.522] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f5034edcd213a23 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1f5034edcd213a23"} 15:25:22 [15:25:22.523] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f5034edcd213a23 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1f5034edcd213a23"} 15:25:22 [15:25:22.523] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1f5034edcd213a23 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1f5034edcd213a23"} 15:25:22 [15:25:22.524] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f5034edcd213a23 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1f5034edcd213a23"} 15:25:22 [15:25:22.525] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.525] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77e758b3955c926e epochNumber=1 {"provingJobId":"1:BASE_PARITY:77e758b3955c926e"} 15:25:22 [15:25:22.549] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.551] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.551] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64811b172ac4b76d epochNumber=1 {"provingJobId":"1:BASE_PARITY:64811b172ac4b76d"} 15:25:22 [15:25:22.552] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:64811b172ac4b76d {"provingJobId":"1:BASE_PARITY:64811b172ac4b76d"} 15:25:22 [15:25:22.552] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:64811b172ac4b76d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:64811b172ac4b76d"} 15:25:22 [15:25:22.553] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:64811b172ac4b76d type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:64811b172ac4b76d"} 15:25:22 [15:25:22.554] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.555] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9a083da4ae1357a9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9a083da4ae1357a9"} 15:25:22 [15:25:22.555] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:9a083da4ae1357a9 {"provingJobId":"1:BASE_PARITY:9a083da4ae1357a9"} 15:25:22 [15:25:22.555] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9a083da4ae1357a9 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:9a083da4ae1357a9"} 15:25:22 [15:25:22.556] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.557] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4e57f575130beff1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4e57f575130beff1"} 15:25:22 [15:25:22.557] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4e57f575130beff1 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4e57f575130beff1"} 15:25:22 [15:25:22.558] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4e57f575130beff1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4e57f575130beff1"} 15:25:22 [15:25:22.559] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.559] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:71ad40195f2c85c9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:71ad40195f2c85c9"} 15:25:22 [15:25:22.559] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:71ad40195f2c85c9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:71ad40195f2c85c9"} 15:25:22 [15:25:22.560] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:71ad40195f2c85c9 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:71ad40195f2c85c9"} 15:25:22 [15:25:22.560] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.561] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf4606016e4990e7 epochNumber=0 {"provingJobId":"1:BASE_PARITY:bf4606016e4990e7"} 15:25:22 [15:25:22.561] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:bf4606016e4990e7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:bf4606016e4990e7"} 15:25:22 [15:25:22.561] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a9a22a4f7cf10582 epochNumber=0 {"provingJobId":"1:BASE_PARITY:a9a22a4f7cf10582"} 15:25:22 [15:25:22.561] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:bf4606016e4990e7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:bf4606016e4990e7"} 15:25:22 [15:25:22.562] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:763948e67e76a485 epochNumber=0 {"provingJobId":"1:BASE_PARITY:763948e67e76a485"} 15:25:22 [15:25:22.569] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.570] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.570] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:074ab7603b43cb79 epochNumber=2 {"provingJobId":"1:BASE_PARITY:074ab7603b43cb79"} 15:25:22 [15:25:22.571] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:58d0d2f78fa8a184 epochNumber=1 {"provingJobId":"1:BASE_PARITY:58d0d2f78fa8a184"} 15:25:22 [15:25:22.572] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.572] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4bd923b926ab5a25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4bd923b926ab5a25"} 15:25:22 [15:25:22.573] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.573] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0882668d9e6f65fd epochNumber=1 {"provingJobId":"1:BASE_PARITY:0882668d9e6f65fd"} 15:25:22 [15:25:22.574] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dfee776cee13eb5e epochNumber=1 {"provingJobId":"1:BASE_PARITY:dfee776cee13eb5e"} 15:25:22 [15:25:22.574] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:40956d24ba466019 epochNumber=2 {"provingJobId":"1:BASE_PARITY:40956d24ba466019"} 15:25:22 [15:25:22.574] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afba1899d8d4b3a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:afba1899d8d4b3a4"} 15:25:22 [15:25:22.575] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.575] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc71805ddcbdec7f epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc71805ddcbdec7f"} 15:25:22 [15:25:22.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e76337a6e94806cd epochNumber=1 {"provingJobId":"1:BASE_PARITY:e76337a6e94806cd"} 15:25:22 [15:25:22.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:37564f4154c5482b epochNumber=2 {"provingJobId":"1:BASE_PARITY:37564f4154c5482b"} 15:25:22 [15:25:22.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9eeedf2c68d173bc epochNumber=1 {"provingJobId":"1:BASE_PARITY:9eeedf2c68d173bc"} 15:25:22 [15:25:22.577] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.578] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:89d3a7a091b7e340 epochNumber=1 {"provingJobId":"1:BASE_PARITY:89d3a7a091b7e340"} 15:25:22 [15:25:22.578] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8cd4778b6d536e73 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8cd4778b6d536e73"} 15:25:22 [15:25:22.578] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:09ba048436ab925d epochNumber=2 {"provingJobId":"1:BASE_PARITY:09ba048436ab925d"} 15:25:22 [15:25:22.578] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:150991af4122fdca epochNumber=1 {"provingJobId":"1:BASE_PARITY:150991af4122fdca"} 15:25:22 [15:25:22.579] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.580] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da219992f6d216bf epochNumber=1 {"provingJobId":"1:BASE_PARITY:da219992f6d216bf"} 15:25:22 [15:25:22.580] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aee3ef3e551e83eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:aee3ef3e551e83eb"} 15:25:22 [15:25:22.580] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:da219992f6d216bf type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:da219992f6d216bf"} 15:25:22 [15:25:22.581] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.582] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08c7792289a71119 epochNumber=1 {"provingJobId":"1:BASE_PARITY:08c7792289a71119"} 15:25:22 [15:25:22.582] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f2607fcbd8e6b931 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f2607fcbd8e6b931"} 15:25:22 [15:25:22.582] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:08c7792289a71119 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:08c7792289a71119"} 15:25:22 [15:25:22.583] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.583] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2fc53bccdd813d6d epochNumber=1 {"provingJobId":"1:BASE_PARITY:2fc53bccdd813d6d"} 15:25:22 [15:25:22.584] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9a5f576c27079f61 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9a5f576c27079f61"} 15:25:22 [15:25:22.584] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2fc53bccdd813d6d type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:2fc53bccdd813d6d"} 15:25:22 [15:25:22.585] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.585] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a876576bb637887a epochNumber=1 {"provingJobId":"1:BASE_PARITY:a876576bb637887a"} 15:25:22 [15:25:22.585] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:a876576bb637887a {"provingJobId":"1:BASE_PARITY:a876576bb637887a"} 15:25:22 [15:25:22.586] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a876576bb637887a type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:a876576bb637887a"} 15:25:22 [15:25:22.586] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9d8ea237fef0fd18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9d8ea237fef0fd18"} 15:25:22 [15:25:22.586] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a876576bb637887a has already been completed {"provingJobId":"1:BASE_PARITY:a876576bb637887a"} 15:25:22 [15:25:22.587] INFO: prover-client:proving-broker Proving Broker started 15:25:22 [15:25:22.587] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1ac2a13a3cc4e8fa epochNumber=1 {"provingJobId":"1:BASE_PARITY:1ac2a13a3cc4e8fa"} 15:25:22 [15:25:22.588] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:73e28d8defbf184f epochNumber=2 {"provingJobId":"1:BASE_PARITY:73e28d8defbf184f"} 15:25:23 [15:25:23.640] INFO: prover-client:proving-broker Proving Broker started 15:25:23 [15:25:23.641] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1ac2a13a3cc4e8fa settled=false {"provingJobId":"1:BASE_PARITY:1ac2a13a3cc4e8fa","status":"pending"} 15:25:23 [15:25:23.641] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:73e28d8defbf184f settled=false {"provingJobId":"1:BASE_PARITY:73e28d8defbf184f","status":"pending"} 15:25:23 [15:25:23.642] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:1ac2a13a3cc4e8fa type=BASE_PARITY startedAt=1750433122588 older agent has taken job {"provingJobId":"1:BASE_PARITY:1ac2a13a3cc4e8fa"} 15:25:23 [15:25:23.642] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1ac2a13a3cc4e8fa type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:1ac2a13a3cc4e8fa"} 15:25:23 [15:25:23.644] INFO: prover-client:proving-broker Proving Broker started 15:25:23 [15:25:23.644] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:48ab605d3c6ff625 epochNumber=1 {"provingJobId":"1:BASE_PARITY:48ab605d3c6ff625"} 15:25:23 [15:25:23.645] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3463563affac097 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e3463563affac097"} 15:25:24 [15:25:24.646] INFO: prover-client:proving-broker Proving Broker started 15:25:24 [15:25:24.647] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:48ab605d3c6ff625 settled=false {"provingJobId":"1:BASE_PARITY:48ab605d3c6ff625","status":"pending"} 15:25:24 [15:25:24.647] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e3463563affac097 settled=false {"provingJobId":"1:BASE_PARITY:e3463563affac097","status":"pending"} 15:25:24 [15:25:24.648] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:48ab605d3c6ff625 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:48ab605d3c6ff625"} 15:25:24 [15:25:24.650] INFO: prover-client:proving-broker Proving Broker started 15:25:24 [15:25:24.650] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:68d7e96e72707481 epochNumber=1 {"provingJobId":"1:BASE_PARITY:68d7e96e72707481"} 15:25:24 [15:25:24.651] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fbf00e8ddd202260 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fbf00e8ddd202260"} 15:25:25 [15:25:25.651] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.652] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:68d7e96e72707481 settled=false {"provingJobId":"1:BASE_PARITY:68d7e96e72707481","status":"pending"} 15:25:25 [15:25:25.653] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fbf00e8ddd202260 settled=false {"provingJobId":"1:BASE_PARITY:fbf00e8ddd202260","status":"pending"} 15:25:25 [15:25:25.655] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:68d7e96e72707481 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:68d7e96e72707481"} 15:25:25 [15:25:25.656] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:68d7e96e72707481 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:68d7e96e72707481"} 15:25:25 [15:25:25.658] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.660] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a7a0f511863ddd57 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a7a0f511863ddd57"} 15:25:25 [15:25:25.662] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b7bb96eaa1e615bd epochNumber=2 {"provingJobId":"1:BASE_PARITY:b7bb96eaa1e615bd"} 15:25:25 [15:25:25.664] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a7a0f511863ddd57 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a7a0f511863ddd57"} 15:25:25 [15:25:25.665] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b7bb96eaa1e615bd type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b7bb96eaa1e615bd"} 15:25:25 [15:25:25.666] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04f9366cfe5a84e4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:04f9366cfe5a84e4"} 15:25:25 [15:25:25.668] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a173efb5f63aacaf epochNumber=2 {"provingJobId":"1:BASE_PARITY:a173efb5f63aacaf"} 15:25:25 [15:25:25.668] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:04f9366cfe5a84e4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:04f9366cfe5a84e4"} 15:25:25 [15:25:25.668] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:04f9366cfe5a84e4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:04f9366cfe5a84e4"} 15:25:25 [15:25:25.669] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a173efb5f63aacaf type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a173efb5f63aacaf"} 15:25:25 [15:25:25.670] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.671] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:dfd8838192772322 err=test error {"provingJoId":"1:BASE_PARITY:dfd8838192772322"} 15:25:25 [15:25:25.672] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.673] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:da34cdb4693db148 not found {"provingJobId":"1:BASE_PARITY:da34cdb4693db148"} 15:25:25 [15:25:25.674] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.675] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0441a389b296dc1c epochNumber=1 {"provingJobId":"1:BASE_PARITY:0441a389b296dc1c"} 15:25:25 [15:25:25.676] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.677] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e81ce2eb55000230 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e81ce2eb55000230"} 15:25:25 [15:25:25.779] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e81ce2eb55000230 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:e81ce2eb55000230"} 15:25:25 [15:25:25.786] INFO: prover-client:proving-broker Proving Broker started 15:25:25 [15:25:25.788] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6c6d7f75017211e2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6c6d7f75017211e2"} 15:25:25 [15:25:25.891] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6c6d7f75017211e2 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:6c6d7f75017211e2"} 15:25:25 [15:25:25.896] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2530a135ae13c80b epochNumber=10 {"provingJobId":"1:BASE_PARITY:2530a135ae13c80b"} 15:25:25 [15:25:25.917] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:26 [15:25:26.024] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.025] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:750c18f40038471a epochNumber=1 {"provingJobId":"1:BASE_PARITY:750c18f40038471a"} 15:25:26 [15:25:26.177] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:750c18f40038471a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:750c18f40038471a"} 15:25:26 [15:25:26.227] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.228] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6da8e6988b6b9761 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6da8e6988b6b9761"} 15:25:26 [15:25:26.229] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6da8e6988b6b9761 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:6da8e6988b6b9761"} 15:25:26 [15:25:26.230] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.230] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c9bcf8d79f7cf10f epochNumber=1 {"provingJobId":"1:BASE_PARITY:c9bcf8d79f7cf10f"} 15:25:26 [15:25:26.231] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c9bcf8d79f7cf10f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:c9bcf8d79f7cf10f"} 15:25:26 [15:25:26.231] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c9bcf8d79f7cf10f type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:c9bcf8d79f7cf10f"} 15:25:26 [15:25:26.232] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.232] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04df09c3441f39e4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:04df09c3441f39e4"} 15:25:26 [15:25:26.233] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:04df09c3441f39e4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:04df09c3441f39e4"} 15:25:26 [15:25:26.234] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.234] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2867ecae65079295 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2867ecae65079295"} 15:25:26 [15:25:26.235] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2867ecae65079295 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:2867ecae65079295"} 15:25:26 [15:25:26.235] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2dc96cd48e82928 epochNumber=3 {"provingJobId":"1:BASE_PARITY:e2dc96cd48e82928"} 15:25:26 [15:25:26.259] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:26 [15:25:26.261] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2867ecae65079295 err=test error {"provingJoId":"1:BASE_PARITY:2867ecae65079295"} 15:25:26 [15:25:26.263] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.263] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5d71beb84159bce8 settled=false {"provingJobId":"1:BASE_PARITY:5d71beb84159bce8","status":"pending"} 15:25:26 [15:25:26.264] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d7d3712e48ea6b04 settled=false {"provingJobId":"1:BASE_PARITY:d7d3712e48ea6b04","status":"pending"} 15:25:26 [15:25:26.266] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.266] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b8f3efcd3d6aaccd settled=true {"provingJobId":"1:BASE_PARITY:b8f3efcd3d6aaccd","status":"fulfilled"} 15:25:26 [15:25:26.267] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:c24b34ba107df305 settled=true {"provingJobId":"2:BASE_PARITY:c24b34ba107df305","status":"fulfilled"} 15:25:26 [15:25:26.268] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.268] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5911c1a3afae75e6 settled=true {"provingJobId":"1:BASE_PARITY:5911c1a3afae75e6","status":"fulfilled"} 15:25:26 [15:25:26.268] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f32ac3287ccf0d3c settled=false {"provingJobId":"1:BASE_PARITY:f32ac3287ccf0d3c","status":"pending"} 15:25:26 [15:25:26.269] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.271] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a02c37f31e634865 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a02c37f31e634865"} 15:25:26 [15:25:26.273] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.273] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ebdc6f476e06f2f epochNumber=1 {"provingJobId":"1:BASE_PARITY:4ebdc6f476e06f2f"} 15:25:26 [15:25:26.274] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:4ebdc6f476e06f2f: Error: db error: Error: db error 15:25:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:4ebdc6f476e06f2f"} 15:25:26 [15:25:26.275] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.276] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34dec02281d17dcd epochNumber=1 {"provingJobId":"1:BASE_PARITY:34dec02281d17dcd"} 15:25:26 [15:25:26.276] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34dec02281d17dcd type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:34dec02281d17dcd"} 15:25:26 [15:25:26.276] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:34dec02281d17dcd type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:34dec02281d17dcd"} 15:25:26 [15:25:26.278] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.278] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:73a71b07a554a096 epochNumber=1 {"provingJobId":"1:BASE_PARITY:73a71b07a554a096"} 15:25:26 [15:25:26.278] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:73a71b07a554a096 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:73a71b07a554a096"} 15:25:26 [15:25:26.279] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:73a71b07a554a096 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:73a71b07a554a096"} 15:25:26 [15:25:26.279] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:73a71b07a554a096: Error: db error 15:25:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:73a71b07a554a096"} 15:25:26 [15:25:26.280] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.281] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:845c080b46b86740 epochNumber=1 {"provingJobId":"1:BASE_PARITY:845c080b46b86740"} 15:25:26 [15:25:26.282] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:845c080b46b86740 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:845c080b46b86740"} 15:25:26 [15:25:26.282] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:845c080b46b86740 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:845c080b46b86740"} 15:25:26 [15:25:26.284] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.285] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0db309885c71a6d epochNumber=1 {"provingJobId":"1:BASE_PARITY:c0db309885c71a6d"} 15:25:26 [15:25:26.285] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c0db309885c71a6d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c0db309885c71a6d"} 15:25:26 [15:25:26.286] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c0db309885c71a6d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c0db309885c71a6d"} 15:25:26 [15:25:26.286] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:c0db309885c71a6d jobErr=test error: Error: db error 15:25:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:c0db309885c71a6d"} 15:25:26 [15:25:26.288] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.289] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7a66b61728e571da not found {"provingJobId":"1:BASE_PARITY:7a66b61728e571da"} 15:25:26 [15:25:26.290] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.291] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1edd784831b1f3d9 err=test error {"provingJoId":"1:BASE_PARITY:1edd784831b1f3d9"} 15:25:26 [15:25:26.292] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.303] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:814713e7df9161e4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:814713e7df9161e4"} 15:25:26 [15:25:26.303] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:814713e7df9161e4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:814713e7df9161e4"} 15:25:26 [15:25:26.304] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:814713e7df9161e4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:814713e7df9161e4"} 15:25:26 [15:25:26.314] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:5f29b24e742eac00 epochNumber=2 {"provingJobId":"2:BASE_PARITY:5f29b24e742eac00"} 15:25:26 [15:25:26.314] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:5f29b24e742eac00 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:5f29b24e742eac00"} 15:25:26 [15:25:26.315] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:5f29b24e742eac00 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:5f29b24e742eac00"} 15:25:26 [15:25:26.377] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:160e4261c4fd792e epochNumber=3 {"provingJobId":"3:BASE_PARITY:160e4261c4fd792e"} 15:25:26 [15:25:26.395] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:26 [15:25:26.438] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:2da6a99d1caf6cf3 epochNumber=4 {"provingJobId":"4:BASE_PARITY:2da6a99d1caf6cf3"} 15:25:26 [15:25:26.448] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:26 [15:25:26.501] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:6809c571f2080e14 epochNumber=5 {"provingJobId":"5:BASE_PARITY:6809c571f2080e14"} 15:25:26 [15:25:26.525] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:26 [15:25:26.557] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.557] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:838be025dfb569fb epochNumber=42 {"provingJobId":"1:BASE_PARITY:838be025dfb569fb"} 15:25:26 [15:25:26.559] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testJpfkc9/42 with map size 1073741824 15:25:26 [15:25:26.563] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.587] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:de3d5ef7ad922fe0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:de3d5ef7ad922fe0"} 15:25:26 [15:25:26.592] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.592] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bfd35ff1e2e96fb7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bfd35ff1e2e96fb7"} 15:25:26 [15:25:26.593] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0VbCW5/1 with map size 1073741824 15:25:26 [15:25:26.595] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.613] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4eae425b86977da8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4eae425b86977da8"} 15:25:26 [15:25:26.623] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.624] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a873dfaa121e93e epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a873dfaa121e93e"} 15:25:26 [15:25:26.624] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJvtlHI/1 with map size 1073741824 15:25:26 [15:25:26.626] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.643] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1a873dfaa121e93e epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1a873dfaa121e93e"} 15:25:26 [15:25:26.646] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.646] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:79f8047e34861b53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:79f8047e34861b53"} 15:25:26 [15:25:26.647] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPUVdiC/1 with map size 1073741824 15:25:26 [15:25:26.648] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.664] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:79f8047e34861b53 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:79f8047e34861b53"} 15:25:26 [15:25:26.665] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:79f8047e34861b53 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:79f8047e34861b53"} 15:25:26 [15:25:26.671] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:79f8047e34861b53 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:79f8047e34861b53"} 15:25:26 [15:25:26.674] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.674] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4023d0786d58c7f epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4023d0786d58c7f"} 15:25:26 [15:25:26.675] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testObDy9t/1 with map size 1073741824 15:25:26 [15:25:26.677] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.698] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.700] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.700] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4b01f0c8480b8e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4b01f0c8480b8e8"} 15:25:26 [15:25:26.701] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsv8OEe/1 with map size 1073741824 15:25:26 [15:25:26.703] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.721] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c4b01f0c8480b8e8 {"provingJobId":"1:BASE_PARITY:c4b01f0c8480b8e8"} 15:25:26 [15:25:26.722] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c4b01f0c8480b8e8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c4b01f0c8480b8e8"} 15:25:26 [15:25:26.722] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c4b01f0c8480b8e8 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c4b01f0c8480b8e8"} 15:25:26 [15:25:26.732] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.733] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7664e2cf58f5a228 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7664e2cf58f5a228"} 15:25:26 [15:25:26.734] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbWlS4P/1 with map size 1073741824 15:25:26 [15:25:26.736] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.752] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7664e2cf58f5a228 {"provingJobId":"1:BASE_PARITY:7664e2cf58f5a228"} 15:25:26 [15:25:26.753] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7664e2cf58f5a228 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:7664e2cf58f5a228"} 15:25:26 [15:25:26.762] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.762] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a710d321b95593ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:a710d321b95593ea"} 15:25:26 [15:25:26.763] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRDhjOx/1 with map size 1073741824 15:25:26 [15:25:26.768] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.785] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a710d321b95593ea type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a710d321b95593ea"} 15:25:26 [15:25:26.786] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a710d321b95593ea type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a710d321b95593ea"} 15:25:26 [15:25:26.793] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.793] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2278d49e9e932a9c epochNumber=1 {"provingJobId":"1:BASE_PARITY:2278d49e9e932a9c"} 15:25:26 [15:25:26.794] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwzSv25/1 with map size 1073741824 15:25:26 [15:25:26.796] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.812] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2278d49e9e932a9c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2278d49e9e932a9c"} 15:25:26 [15:25:26.813] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2278d49e9e932a9c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2278d49e9e932a9c"} 15:25:26 [15:25:26.821] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.821] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:47c101e01b3121e9 epochNumber=0 {"provingJobId":"1:BASE_PARITY:47c101e01b3121e9"} 15:25:26 [15:25:26.822] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testRB3EWV/0 with map size 1073741824 15:25:26 [15:25:26.824] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.840] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:47c101e01b3121e9 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:47c101e01b3121e9"} 15:25:26 [15:25:26.841] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1952f20a4a4f96d epochNumber=0 {"provingJobId":"1:BASE_PARITY:a1952f20a4a4f96d"} 15:25:26 [15:25:26.841] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:47c101e01b3121e9 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:47c101e01b3121e9"} 15:25:26 [15:25:26.842] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ae8d23ef43b993e3 epochNumber=0 {"provingJobId":"1:BASE_PARITY:ae8d23ef43b993e3"} 15:25:26 [15:25:26.855] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.856] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.857] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33dd95410be868ca epochNumber=2 {"provingJobId":"1:BASE_PARITY:33dd95410be868ca"} 15:25:26 [15:25:26.858] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testMcWfIU/2 with map size 1073741824 15:25:26 [15:25:26.859] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.876] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cadd115bcb0706b9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cadd115bcb0706b9"} 15:25:26 [15:25:26.877] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMcWfIU/1 with map size 1073741824 15:25:26 [15:25:26.878] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.898] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.899] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ea260a46f8788f46 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ea260a46f8788f46"} 15:25:26 [15:25:26.901] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnWwa85/1 with map size 1073741824 15:25:26 [15:25:26.906] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.926] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.927] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cb9061dba0d9822c epochNumber=1 {"provingJobId":"1:BASE_PARITY:cb9061dba0d9822c"} 15:25:26 [15:25:26.928] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxtcWfu/1 with map size 1073741824 15:25:26 [15:25:26.930] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.947] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:41ed8154363b1b8d epochNumber=1 {"provingJobId":"1:BASE_PARITY:41ed8154363b1b8d"} 15:25:26 [15:25:26.953] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8eba29778dc56b60 epochNumber=2 {"provingJobId":"1:BASE_PARITY:8eba29778dc56b60"} 15:25:26 [15:25:26.954] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testxtcWfu/2 with map size 1073741824 15:25:26 [15:25:26.956] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:26 [15:25:26.972] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:efa7cb4b25ea91e9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:efa7cb4b25ea91e9"} 15:25:26 [15:25:26.981] INFO: prover-client:proving-broker Proving Broker started 15:25:26 [15:25:26.982] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:145091e2bd0a182a epochNumber=1 {"provingJobId":"1:BASE_PARITY:145091e2bd0a182a"} 15:25:26 [15:25:26.983] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqNCMsb/1 with map size 1073741824 15:25:26 [15:25:26.985] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.002] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4517a9d0ad9dec0d epochNumber=1 {"provingJobId":"1:BASE_PARITY:4517a9d0ad9dec0d"} 15:25:27 [15:25:27.008] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:87b99f3bfbb35d15 epochNumber=2 {"provingJobId":"1:BASE_PARITY:87b99f3bfbb35d15"} 15:25:27 [15:25:27.009] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqNCMsb/2 with map size 1073741824 15:25:27 [15:25:27.010] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.027] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1350332e876fde9d epochNumber=1 {"provingJobId":"1:BASE_PARITY:1350332e876fde9d"} 15:25:27 [15:25:27.039] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.040] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0cb5f2cebd1730c7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0cb5f2cebd1730c7"} 15:25:27 [15:25:27.041] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXZbjlx/1 with map size 1073741824 15:25:27 [15:25:27.043] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.064] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:df13d070559de14c epochNumber=1 {"provingJobId":"1:BASE_PARITY:df13d070559de14c"} 15:25:27 [15:25:27.070] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3adc15a9b84103a5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3adc15a9b84103a5"} 15:25:27 [15:25:27.071] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXZbjlx/2 with map size 1073741824 15:25:27 [15:25:27.074] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.091] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dca7e08700302705 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dca7e08700302705"} 15:25:27 [15:25:27.104] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7ce6b0fae4b0cde7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7ce6b0fae4b0cde7"} 15:25:27 [15:25:27.105] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBVQUPy/1 with map size 1073741824 15:25:27 [15:25:27.108] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.125] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:55caa3398bc24960 epochNumber=1 {"provingJobId":"1:BASE_PARITY:55caa3398bc24960"} 15:25:27 [15:25:27.132] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7ce6b0fae4b0cde7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7ce6b0fae4b0cde7"} 15:25:27 [15:25:27.144] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.145] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bc84c36766d17058 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bc84c36766d17058"} 15:25:27 [15:25:27.146] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSm5Ol4/1 with map size 1073741824 15:25:27 [15:25:27.149] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.172] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ef68094e428d39a8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ef68094e428d39a8"} 15:25:27 [15:25:27.178] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bc84c36766d17058 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:bc84c36766d17058"} 15:25:27 [15:25:27.189] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.190] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:97e255dd8576e50f epochNumber=1 {"provingJobId":"1:BASE_PARITY:97e255dd8576e50f"} 15:25:27 [15:25:27.191] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6MREAw/1 with map size 1073741824 15:25:27 [15:25:27.192] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.209] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c081a4aba91de485 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c081a4aba91de485"} 15:25:27 [15:25:27.215] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:97e255dd8576e50f type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:97e255dd8576e50f"} 15:25:27 [15:25:27.219] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.219] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7a7b610c200598a2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7a7b610c200598a2"} 15:25:27 [15:25:27.220] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQAVmdr/1 with map size 1073741824 15:25:27 [15:25:27.222] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.238] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7a7b610c200598a2 {"provingJobId":"1:BASE_PARITY:7a7b610c200598a2"} 15:25:27 [15:25:27.238] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a7b610c200598a2 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:7a7b610c200598a2"} 15:25:27 [15:25:27.245] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a083cc33b6d1971 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a083cc33b6d1971"} 15:25:27 [15:25:27.252] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7a7b610c200598a2 has already been completed {"provingJobId":"1:BASE_PARITY:7a7b610c200598a2"} 15:25:27 [15:25:27.258] INFO: prover-client:proving-broker Proving Broker started 15:25:27 [15:25:27.258] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:74fb0431785ac872 epochNumber=1 {"provingJobId":"1:BASE_PARITY:74fb0431785ac872"} 15:25:27 [15:25:27.259] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqlWscI/1 with map size 1073741824 15:25:27 [15:25:27.261] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:27 [15:25:27.280] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2da55581f2000270 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2da55581f2000270"} 15:25:27 [15:25:27.281] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqlWscI/2 with map size 1073741824 15:25:27 [15:25:27.285] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:28 [15:25:28.356] INFO: prover-client:proving-broker Proving Broker started 15:25:28 [15:25:28.365] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:74fb0431785ac872 settled=false {"provingJobId":"1:BASE_PARITY:74fb0431785ac872","status":"pending"} 15:25:28 [15:25:28.367] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2da55581f2000270 settled=false {"provingJobId":"1:BASE_PARITY:2da55581f2000270","status":"pending"} 15:25:28 [15:25:28.369] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:74fb0431785ac872 type=BASE_PARITY startedAt=1750433127304 older agent has taken job {"provingJobId":"1:BASE_PARITY:74fb0431785ac872"} 15:25:28 [15:25:28.369] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:74fb0431785ac872 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:74fb0431785ac872"} 15:25:28 [15:25:28.376] INFO: prover-client:proving-broker Proving Broker started 15:25:28 [15:25:28.377] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:db81906ca0fa49fd epochNumber=1 {"provingJobId":"1:BASE_PARITY:db81906ca0fa49fd"} 15:25:28 [15:25:28.379] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeHmpWg/1 with map size 1073741824 15:25:28 [15:25:28.383] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:28 [15:25:28.404] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4baf71f6116130d epochNumber=2 {"provingJobId":"1:BASE_PARITY:c4baf71f6116130d"} 15:25:28 [15:25:28.406] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testeHmpWg/2 with map size 1073741824 15:25:28 [15:25:28.409] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:29 [15:25:29.435] INFO: prover-client:proving-broker Proving Broker started 15:25:29 [15:25:29.437] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:db81906ca0fa49fd settled=false {"provingJobId":"1:BASE_PARITY:db81906ca0fa49fd","status":"pending"} 15:25:29 [15:25:29.438] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c4baf71f6116130d settled=false {"provingJobId":"1:BASE_PARITY:c4baf71f6116130d","status":"pending"} 15:25:29 [15:25:29.439] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:db81906ca0fa49fd type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:db81906ca0fa49fd"} 15:25:29 [15:25:29.442] INFO: prover-client:proving-broker Proving Broker started 15:25:29 [15:25:29.443] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2099dc8e278f0acd epochNumber=1 {"provingJobId":"1:BASE_PARITY:2099dc8e278f0acd"} 15:25:29 [15:25:29.443] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7OtKq1/1 with map size 1073741824 15:25:29 [15:25:29.445] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:29 [15:25:29.465] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a9b81e69ff17500 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3a9b81e69ff17500"} 15:25:29 [15:25:29.466] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7OtKq1/2 with map size 1073741824 15:25:29 [15:25:29.468] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.486] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.488] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2099dc8e278f0acd settled=false {"provingJobId":"1:BASE_PARITY:2099dc8e278f0acd","status":"pending"} 15:25:30 [15:25:30.490] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3a9b81e69ff17500 settled=false {"provingJobId":"1:BASE_PARITY:3a9b81e69ff17500","status":"pending"} 15:25:30 [15:25:30.491] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2099dc8e278f0acd type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2099dc8e278f0acd"} 15:25:30 [15:25:30.491] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2099dc8e278f0acd type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2099dc8e278f0acd"} 15:25:30 [15:25:30.500] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.501] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:31f66afb2faa37b1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:31f66afb2faa37b1"} 15:25:30 [15:25:30.501] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6aya32/1 with map size 1073741824 15:25:30 [15:25:30.504] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.519] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2189cc15640456a4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2189cc15640456a4"} 15:25:30 [15:25:30.520] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test6aya32/2 with map size 1073741824 15:25:30 [15:25:30.521] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.539] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:31f66afb2faa37b1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:31f66afb2faa37b1"} 15:25:30 [15:25:30.546] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2189cc15640456a4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2189cc15640456a4"} 15:25:30 [15:25:30.555] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.557] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:63ed2aaf6605e5a2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:63ed2aaf6605e5a2"} 15:25:30 [15:25:30.557] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiRqeuW/1 with map size 1073741824 15:25:30 [15:25:30.559] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.575] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ffb69bdd0b545ec epochNumber=2 {"provingJobId":"1:BASE_PARITY:8ffb69bdd0b545ec"} 15:25:30 [15:25:30.576] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiRqeuW/2 with map size 1073741824 15:25:30 [15:25:30.578] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.594] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:63ed2aaf6605e5a2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:63ed2aaf6605e5a2"} 15:25:30 [15:25:30.594] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:63ed2aaf6605e5a2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:63ed2aaf6605e5a2"} 15:25:30 [15:25:30.600] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8ffb69bdd0b545ec type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8ffb69bdd0b545ec"} 15:25:30 [15:25:30.611] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.612] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:89f23060ad68053b err=test error {"provingJoId":"1:BASE_PARITY:89f23060ad68053b"} 15:25:30 [15:25:30.616] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.617] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:71e484143cfffb76 not found {"provingJobId":"1:BASE_PARITY:71e484143cfffb76"} 15:25:30 [15:25:30.621] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.621] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b2d8a251abe8c0e epochNumber=1 {"provingJobId":"1:BASE_PARITY:6b2d8a251abe8c0e"} 15:25:30 [15:25:30.622] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRdODTi/1 with map size 1073741824 15:25:30 [15:25:30.624] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.644] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.645] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b24671be60ec4c07 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b24671be60ec4c07"} 15:25:30 [15:25:30.646] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHlXZJv/1 with map size 1073741824 15:25:30 [15:25:30.648] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.772] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b24671be60ec4c07 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:b24671be60ec4c07"} 15:25:30 [15:25:30.778] INFO: prover-client:proving-broker Proving Broker started 15:25:30 [15:25:30.779] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f885c99821113f0c epochNumber=1 {"provingJobId":"1:BASE_PARITY:f885c99821113f0c"} 15:25:30 [15:25:30.780] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testf5UKzK/1 with map size 1073741824 15:25:30 [15:25:30.783] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.904] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f885c99821113f0c timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f885c99821113f0c"} 15:25:30 [15:25:30.909] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:552c3d24de52c338 epochNumber=10 {"provingJobId":"1:BASE_PARITY:552c3d24de52c338"} 15:25:30 [15:25:30.910] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testf5UKzK/10 with map size 1073741824 15:25:30 [15:25:30.912] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:30 [15:25:30.929] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:30 [15:25:30.930] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 15:25:31 [15:25:31.063] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dca6a42277e5e77e epochNumber=1 {"provingJobId":"1:BASE_PARITY:dca6a42277e5e77e"} 15:25:31 [15:25:31.067] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testo34ESs/1 with map size 1073741824 15:25:31 [15:25:31.072] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.245] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:dca6a42277e5e77e timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:dca6a42277e5e77e"} 15:25:31 [15:25:31.300] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.301] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ce06690e6d4d862e epochNumber=1 {"provingJobId":"1:BASE_PARITY:ce06690e6d4d862e"} 15:25:31 [15:25:31.303] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testP54tPt/1 with map size 1073741824 15:25:31 [15:25:31.305] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.324] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ce06690e6d4d862e type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:ce06690e6d4d862e"} 15:25:31 [15:25:31.333] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.334] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7786632966d8ecb8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7786632966d8ecb8"} 15:25:31 [15:25:31.335] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcP9mpx/1 with map size 1073741824 15:25:31 [15:25:31.337] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.371] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7786632966d8ecb8 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:7786632966d8ecb8"} 15:25:31 [15:25:31.372] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7786632966d8ecb8 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:7786632966d8ecb8"} 15:25:31 [15:25:31.382] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.383] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:041a6a1bd61a40e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:041a6a1bd61a40e5"} 15:25:31 [15:25:31.383] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test04diIn/1 with map size 1073741824 15:25:31 [15:25:31.387] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.408] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:041a6a1bd61a40e5 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:041a6a1bd61a40e5"} 15:25:31 [15:25:31.419] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:22488c6940818380 epochNumber=1 {"provingJobId":"1:BASE_PARITY:22488c6940818380"} 15:25:31 [15:25:31.421] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBjlVxy/1 with map size 1073741824 15:25:31 [15:25:31.423] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.448] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:22488c6940818380 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:22488c6940818380"} 15:25:31 [15:25:31.451] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b20e9675fb65dc54 epochNumber=3 {"provingJobId":"1:BASE_PARITY:b20e9675fb65dc54"} 15:25:31 [15:25:31.457] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testBjlVxy/3 with map size 1073741824 15:25:31 [15:25:31.462] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.498] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:31 [15:25:31.498] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 15:25:31 [15:25:31.505] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:22488c6940818380 err=test error {"provingJoId":"1:BASE_PARITY:22488c6940818380"} 15:25:31 [15:25:31.511] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpX2SUS/1 with map size 1073741824 15:25:31 [15:25:31.513] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.534] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpX2SUS/2 with map size 1073741824 15:25:31 [15:25:31.539] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.564] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.566] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:209e6ad70f789904 settled=false {"provingJobId":"1:BASE_PARITY:209e6ad70f789904","status":"pending"} 15:25:31 [15:25:31.569] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c263f2c1bb11f5f2 settled=false {"provingJobId":"1:BASE_PARITY:c263f2c1bb11f5f2","status":"pending"} 15:25:31 [15:25:31.576] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM0Vuge/1 with map size 1073741824 15:25:31 [15:25:31.578] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.601] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testM0Vuge/2 with map size 1073741824 15:25:31 [15:25:31.604] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.639] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.641] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1e80177c681523d6 settled=true {"provingJobId":"1:BASE_PARITY:1e80177c681523d6","status":"fulfilled"} 15:25:31 [15:25:31.650] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:5ee55a38e45d58f4 settled=true {"provingJobId":"2:BASE_PARITY:5ee55a38e45d58f4","status":"fulfilled"} 15:25:31 [15:25:31.661] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsyPuRP/1 with map size 1073741824 15:25:31 [15:25:31.665] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.691] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testsyPuRP/2 with map size 1073741824 15:25:31 [15:25:31.695] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.714] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.716] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3d25bf9e2e28b72d settled=true {"provingJobId":"1:BASE_PARITY:3d25bf9e2e28b72d","status":"fulfilled"} 15:25:31 [15:25:31.718] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e6e3be13a523d596 settled=false {"provingJobId":"1:BASE_PARITY:e6e3be13a523d596","status":"pending"} 15:25:31 [15:25:31.723] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.724] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8c241d44d1eea0fd epochNumber=1 {"provingJobId":"1:BASE_PARITY:8c241d44d1eea0fd"} 15:25:31 [15:25:31.726] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpWmuc3/1 with map size 1073741824 15:25:31 [15:25:31.732] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.771] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.772] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:91adc79c24b356b7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:91adc79c24b356b7"} 15:25:31 [15:25:31.773] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:91adc79c24b356b7: Error: db error: Error: db error 15:25:31 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:91adc79c24b356b7"} 15:25:31 [15:25:31.777] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.777] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2faf52ab06549e88 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2faf52ab06549e88"} 15:25:31 [15:25:31.778] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnLpabU/1 with map size 1073741824 15:25:31 [15:25:31.782] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.813] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2faf52ab06549e88 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2faf52ab06549e88"} 15:25:31 [15:25:31.813] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2faf52ab06549e88 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2faf52ab06549e88"} 15:25:31 [15:25:31.827] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.828] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f3e733fbd812594b epochNumber=1 {"provingJobId":"1:BASE_PARITY:f3e733fbd812594b"} 15:25:31 [15:25:31.829] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9aBiO5/1 with map size 1073741824 15:25:31 [15:25:31.832] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.859] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f3e733fbd812594b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f3e733fbd812594b"} 15:25:31 [15:25:31.859] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f3e733fbd812594b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f3e733fbd812594b"} 15:25:31 [15:25:31.860] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:f3e733fbd812594b: Error: db error 15:25:31 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:f3e733fbd812594b"} 15:25:31 [15:25:31.867] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.868] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3cc82b4cb709404f epochNumber=1 {"provingJobId":"1:BASE_PARITY:3cc82b4cb709404f"} 15:25:31 [15:25:31.869] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAT5777/1 with map size 1073741824 15:25:31 [15:25:31.871] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.891] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3cc82b4cb709404f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3cc82b4cb709404f"} 15:25:31 [15:25:31.891] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3cc82b4cb709404f type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3cc82b4cb709404f"} 15:25:31 [15:25:31.901] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.902] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:822e078537d6271e epochNumber=1 {"provingJobId":"1:BASE_PARITY:822e078537d6271e"} 15:25:31 [15:25:31.903] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8h6ouj/1 with map size 1073741824 15:25:31 [15:25:31.908] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.933] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:822e078537d6271e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:822e078537d6271e"} 15:25:31 [15:25:31.934] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:822e078537d6271e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:822e078537d6271e"} 15:25:31 [15:25:31.935] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:822e078537d6271e jobErr=test error: Error: db error 15:25:31 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:822e078537d6271e"} 15:25:31 [15:25:31.941] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.942] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bf4b038d24d986c5 not found {"provingJobId":"1:BASE_PARITY:bf4b038d24d986c5"} 15:25:31 [15:25:31.955] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.956] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:5ac31289612a02c1 err=test error {"provingJoId":"1:BASE_PARITY:5ac31289612a02c1"} 15:25:31 [15:25:31.963] INFO: prover-client:proving-broker Proving Broker started 15:25:31 [15:25:31.973] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ccb25ee9ab5af7ec epochNumber=1 {"provingJobId":"1:BASE_PARITY:ccb25ee9ab5af7ec"} 15:25:31 [15:25:31.976] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlaDnps/1 with map size 1073741824 15:25:31 [15:25:31.980] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:31 [15:25:31.999] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ccb25ee9ab5af7ec type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ccb25ee9ab5af7ec"} 15:25:31 [15:25:31.999] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ccb25ee9ab5af7ec type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ccb25ee9ab5af7ec"} 15:25:32 [15:25:32.017] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:b08303d4e7f684d2 epochNumber=2 {"provingJobId":"2:BASE_PARITY:b08303d4e7f684d2"} 15:25:32 [15:25:32.018] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testlaDnps/2 with map size 1073741824 15:25:32 [15:25:32.022] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:32 [15:25:32.041] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:b08303d4e7f684d2 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:b08303d4e7f684d2"} 15:25:32 [15:25:32.041] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:b08303d4e7f684d2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:b08303d4e7f684d2"} 15:25:32 [15:25:32.113] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:2c10fffd687993f5 epochNumber=3 {"provingJobId":"3:BASE_PARITY:2c10fffd687993f5"} 15:25:32 [15:25:32.115] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testlaDnps/3 with map size 1073741824 15:25:32 [15:25:32.118] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:32 [15:25:32.139] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:32 [15:25:32.140] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 15:25:32 [15:25:32.197] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:c8e857763a8f2fb7 epochNumber=4 {"provingJobId":"4:BASE_PARITY:c8e857763a8f2fb7"} 15:25:32 [15:25:32.198] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testlaDnps/4 with map size 1073741824 15:25:32 [15:25:32.201] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:32 [15:25:32.223] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:32 [15:25:32.223] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 15:25:32 [15:25:32.279] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:ae9088c4151ec30e epochNumber=5 {"provingJobId":"5:BASE_PARITY:ae9088c4151ec30e"} 15:25:32 [15:25:32.280] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testlaDnps/5 with map size 1073741824 15:25:32 [15:25:32.283] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 15:25:32 [15:25:32.302] INFO: prover-client:proving-broker Cleaned up jobs=1 15:25:32 [15:25:32.302] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 15:25:32 PASS src/proving_broker/proving_broker.test.ts (13.41 s) 15:25:32 ProvingBroker 15:25:32 Producer API 15:25:32 ✓ refuses stale jobs (39 ms) 15:25:32 ✓ enqueues jobs (4 ms) 15:25:32 ✓ ignores duplicate jobs (3 ms) 15:25:32 ✓ reports correct status when enqueuing repeat jobs (3 ms) 15:25:32 ✓ throws an error in case of duplicate job IDs (23 ms) 15:25:32 ✓ returns not-found status for non-existing jobs (1 ms) 15:25:32 ✓ cancels jobs in queue (3 ms) 15:25:32 ✓ cancels jobs in-progress (2 ms) 15:25:32 ✓ returns job result if successful (2 ms) 15:25:32 ✓ returns job error if failed (2 ms) 15:25:32 ✓ correctly returns job status for concurrent writes (2 ms) 15:25:32 ✓ refuses stale jobs (36 ms) 15:25:32 ✓ enqueues jobs (31 ms) 15:25:32 ✓ ignores duplicate jobs (23 ms) 15:25:32 ✓ reports correct status when enqueuing repeat jobs (27 ms) 15:25:32 ✓ throws an error in case of duplicate job IDs (24 ms) 15:25:32 ✓ returns not-found status for non-existing jobs (3 ms) 15:25:32 ✓ cancels jobs in queue (31 ms) 15:25:32 ✓ cancels jobs in-progress (28 ms) 15:25:32 ✓ returns job result if successful (31 ms) 15:25:32 ✓ returns job error if failed (27 ms) 15:25:32 ✓ correctly returns job status for concurrent writes (33 ms) 15:25:32 Consumer API 15:25:32 ✓ returns undefined if no jobs are available (2 ms) 15:25:32 ✓ returns jobs in priority order (2 ms) 15:25:32 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 15:25:32 ✓ returns a job if it is in the allowList (2 ms) 15:25:32 ✓ returns the most important job if it is in the allowList (2 ms) 15:25:32 ✓ returns any job if filter is empty (2 ms) 15:25:32 ✓ returns a new job when reporting job success (2 ms) 15:25:32 ✓ returns a new job when reporting permanent error (2 ms) 15:25:32 ✓ returns a new job when reporting retry-able error (1 ms) 15:25:32 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 15:25:32 ✓ returns a new job if job is already in progress elsewhere (1056 ms) 15:25:32 ✓ avoids sending the same job to a new agent after a restart (1006 ms) 15:25:32 ✓ avoids sending a completed job to a new agent after a restart (1008 ms) 15:25:32 ✓ tracks job result if in progress (8 ms) 15:25:32 ✓ tracks job result even if job is in queue (3 ms) 15:25:32 ✓ ignores reported job error if unknown job (1 ms) 15:25:32 ✓ ignores job result if unknown job (1 ms) 15:25:32 ✓ returns undefined if no jobs are available (2 ms) 15:25:32 ✓ returns jobs in priority order (40 ms) 15:25:32 ✓ returns undefined if no jobs are available for the given allowList (28 ms) 15:25:32 ✓ returns a job if it is in the allowList (54 ms) 15:25:32 ✓ returns the most important job if it is in the allowList (56 ms) 15:25:32 ✓ returns any job if filter is empty (63 ms) 15:25:32 ✓ returns a new job when reporting job success (38 ms) 15:25:32 ✓ returns a new job when reporting permanent error (46 ms) 15:25:32 ✓ returns a new job when reporting retry-able error (29 ms) 15:25:32 ✓ returns a new job when reporting progress if current one is cancelled (38 ms) 15:25:32 ✓ returns a new job if job is already in progress elsewhere (1117 ms) 15:25:32 ✓ avoids sending the same job to a new agent after a restart (1067 ms) 15:25:32 ✓ avoids sending a completed job to a new agent after a restart (1057 ms) 15:25:32 ✓ tracks job result if in progress (54 ms) 15:25:32 ✓ tracks job result even if job is in queue (54 ms) 15:25:32 ✓ ignores reported job error if unknown job (5 ms) 15:25:32 ✓ ignores job result if unknown job (5 ms) 15:25:32 Timeouts 15:25:32 ✓ tracks in progress jobs (2 ms) 15:25:32 ✓ re-enqueues jobs that time out (109 ms) 15:25:32 ✓ cancel stale jobs that time out (237 ms) 15:25:32 ✓ keeps the jobs in progress while it is alive (202 ms) 15:25:32 ✓ tracks in progress jobs (22 ms) 15:25:32 ✓ re-enqueues jobs that time out (134 ms) 15:25:32 ✓ cancel stale jobs that time out (284 ms) 15:25:32 ✓ keeps the jobs in progress while it is alive (236 ms) 15:25:32 Retries 15:25:32 ✓ retries jobs (3 ms) 15:25:32 ✓ retries up to a maximum number of times (2 ms) 15:25:32 ✓ passing retry=false does not retry (1 ms) 15:25:32 ✓ does not retry if job is stale (27 ms) 15:25:32 ✓ retries jobs (29 ms) 15:25:32 ✓ retries up to a maximum number of times (53 ms) 15:25:32 ✓ passing retry=false does not retry (36 ms) 15:25:32 ✓ does not retry if job is stale (89 ms) 15:25:32 Database management 15:25:32 ✓ re-enqueues proof requests on start (3 ms) 15:25:32 ✓ restores proof results on start (1 ms) 15:25:32 ✓ only re-enqueues unfinished jobs (2 ms) 15:25:32 ✓ saves job when enqueued (3 ms) 15:25:32 ✓ does not retain job if database fails to save (3 ms) 15:25:32 ✓ saves job result (2 ms) 15:25:32 ✓ saves result even if database fails to save (3 ms) 15:25:32 ✓ saves job error (3 ms) 15:25:32 ✓ saves job error even if database fails to save (3 ms) 15:25:32 ✓ does not save job result if job is unknown (2 ms) 15:25:32 ✓ does not save job error if job is unknown (2 ms) 15:25:32 ✓ cleans up old jobs periodically (262 ms) 15:25:32 ✓ re-enqueues proof requests on start (66 ms) 15:25:32 ✓ restores proof results on start (81 ms) 15:25:32 ✓ only re-enqueues unfinished jobs (66 ms) 15:25:32 ✓ saves job when enqueued (46 ms) 15:25:32 ✓ does not retain job if database fails to save (7 ms) 15:25:32 ✓ saves job result (48 ms) 15:25:32 ✓ saves result even if database fails to save (39 ms) 15:25:32 ✓ saves job error (36 ms) 15:25:32 ✓ saves job error even if database fails to save (38 ms) 15:25:32 ✓ does not save job result if job is unknown (10 ms) 15:25:32 ✓ does not save job error if job is unknown (10 ms) 15:25:32 ✓ cleans up old jobs periodically (395 ms) 15:25:32 15:25:32 Test Suites: 1 passed, 1 total 15:25:32 Tests: 96 passed, 96 total 15:25:32 Snapshots: 0 total 15:25:32 Time: 13.519 s 15:25:32 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 15:25:32 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?