ACI ·

Parent Log: http://ci.aztec-labs.com/ab63b2ef1085df0f Command: 68e9475192d7aeac yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/fe25e8151d865a744220512c782713e46482165a Env: REF_NAME=next CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Sat Jun 21 10:22:18 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=next_amd64 Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_staging 10:22:21 [10:22:21.596] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.602] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2935a302a50933a8 epochNumber=42 {"provingJobId":"1:BASE_PARITY:2935a302a50933a8"} 10:22:21 [10:22:21.605] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:1f6020cdd8a1743d epochNumber=1 {"provingJobId":"1:BASE_PARITY:1f6020cdd8a1743d"} 10:22:21 [10:22:21.633] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.634] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f49db6b46bfbf8fd epochNumber=1 {"provingJobId":"1:BASE_PARITY:f49db6b46bfbf8fd"} 10:22:21 [10:22:21.635] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7b92161f9a1d0ae9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7b92161f9a1d0ae9"} 10:22:21 [10:22:21.637] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.637] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a3ef818749710d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a3ef818749710d5"} 10:22:21 [10:22:21.639] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2a3ef818749710d5 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2a3ef818749710d5"} 10:22:21 [10:22:21.640] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.641] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d6fb90b2f90630c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d6fb90b2f90630c4"} 10:22:21 [10:22:21.647] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d6fb90b2f90630c4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d6fb90b2f90630c4"} 10:22:21 [10:22:21.648] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d6fb90b2f90630c4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d6fb90b2f90630c4"} 10:22:21 [10:22:21.648] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d6fb90b2f90630c4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d6fb90b2f90630c4"} 10:22:21 [10:22:21.650] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.650] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:03f240a50e0f3d73 epochNumber=1 {"provingJobId":"1:BASE_PARITY:03f240a50e0f3d73"} 10:22:21 [10:22:21.672] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.673] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.673] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0d8f36fa77fb0123 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0d8f36fa77fb0123"} 10:22:21 [10:22:21.674] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0d8f36fa77fb0123 {"provingJobId":"1:BASE_PARITY:0d8f36fa77fb0123"} 10:22:21 [10:22:21.674] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0d8f36fa77fb0123 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0d8f36fa77fb0123"} 10:22:21 [10:22:21.675] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0d8f36fa77fb0123 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0d8f36fa77fb0123"} 10:22:21 [10:22:21.676] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.677] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:528fd9fa3569bbb6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:528fd9fa3569bbb6"} 10:22:21 [10:22:21.677] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:528fd9fa3569bbb6 {"provingJobId":"1:BASE_PARITY:528fd9fa3569bbb6"} 10:22:21 [10:22:21.677] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:528fd9fa3569bbb6 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:528fd9fa3569bbb6"} 10:22:21 [10:22:21.679] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.679] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b03265e056ce31e6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b03265e056ce31e6"} 10:22:21 [10:22:21.680] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b03265e056ce31e6 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b03265e056ce31e6"} 10:22:21 [10:22:21.680] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b03265e056ce31e6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b03265e056ce31e6"} 10:22:21 [10:22:21.681] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.682] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:48a53ddd8559701e epochNumber=1 {"provingJobId":"1:BASE_PARITY:48a53ddd8559701e"} 10:22:21 [10:22:21.682] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:48a53ddd8559701e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:48a53ddd8559701e"} 10:22:21 [10:22:21.682] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:48a53ddd8559701e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:48a53ddd8559701e"} 10:22:21 [10:22:21.683] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.683] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:18bc42e9ab7fffcc epochNumber=0 {"provingJobId":"1:BASE_PARITY:18bc42e9ab7fffcc"} 10:22:21 [10:22:21.684] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:18bc42e9ab7fffcc epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:18bc42e9ab7fffcc"} 10:22:21 [10:22:21.684] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2adf1a0e1c429879 epochNumber=0 {"provingJobId":"1:BASE_PARITY:2adf1a0e1c429879"} 10:22:21 [10:22:21.684] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:18bc42e9ab7fffcc epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:18bc42e9ab7fffcc"} 10:22:21 [10:22:21.685] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4a6a26388478fe0 epochNumber=0 {"provingJobId":"1:BASE_PARITY:c4a6a26388478fe0"} 10:22:21 [10:22:21.686] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.686] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.687] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d005155573be8700 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d005155573be8700"} 10:22:21 [10:22:21.687] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9bd0c736d4fce2b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9bd0c736d4fce2b6"} 10:22:21 [10:22:21.688] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.688] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9619899f129b1ad9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9619899f129b1ad9"} 10:22:21 [10:22:21.689] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33fce9d71ebbe1c5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:33fce9d71ebbe1c5"} 10:22:21 [10:22:21.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d0bc6db2687efd02 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d0bc6db2687efd02"} 10:22:21 [10:22:21.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8800471d8c700f99 epochNumber=2 {"provingJobId":"1:BASE_PARITY:8800471d8c700f99"} 10:22:21 [10:22:21.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5bb96347ce983c7c epochNumber=1 {"provingJobId":"1:BASE_PARITY:5bb96347ce983c7c"} 10:22:21 [10:22:21.691] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.691] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dad76d485b689ebc epochNumber=1 {"provingJobId":"1:BASE_PARITY:dad76d485b689ebc"} 10:22:21 [10:22:21.692] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:36758cd1fccb10ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:36758cd1fccb10ee"} 10:22:21 [10:22:21.692] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c43d9fe63bb74e3d epochNumber=2 {"provingJobId":"1:BASE_PARITY:c43d9fe63bb74e3d"} 10:22:21 [10:22:21.692] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7595cab1103597f1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7595cab1103597f1"} 10:22:21 [10:22:21.693] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.694] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:37d02fc484d1cb80 epochNumber=1 {"provingJobId":"1:BASE_PARITY:37d02fc484d1cb80"} 10:22:21 [10:22:21.694] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:083042e9d63a3d76 epochNumber=1 {"provingJobId":"1:BASE_PARITY:083042e9d63a3d76"} 10:22:21 [10:22:21.694] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d2dfa3cbc776fc1b epochNumber=2 {"provingJobId":"1:BASE_PARITY:d2dfa3cbc776fc1b"} 10:22:21 [10:22:21.694] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5af5e2dcec104f4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5af5e2dcec104f4"} 10:22:21 [10:22:21.695] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.696] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:38a9bdd5554aab50 epochNumber=1 {"provingJobId":"1:BASE_PARITY:38a9bdd5554aab50"} 10:22:21 [10:22:21.696] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c9969ad1ebe2ce7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c9969ad1ebe2ce7"} 10:22:21 [10:22:21.696] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:38a9bdd5554aab50 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:38a9bdd5554aab50"} 10:22:21 [10:22:21.697] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.698] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:85a1e1ec5825adcc epochNumber=1 {"provingJobId":"1:BASE_PARITY:85a1e1ec5825adcc"} 10:22:21 [10:22:21.698] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67902f47f02ad9ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:67902f47f02ad9ee"} 10:22:21 [10:22:21.698] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:85a1e1ec5825adcc type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:85a1e1ec5825adcc"} 10:22:21 [10:22:21.699] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.699] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2b8c3a4ea4cad07 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e2b8c3a4ea4cad07"} 10:22:21 [10:22:21.700] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6534fa98ed6e4c70 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6534fa98ed6e4c70"} 10:22:21 [10:22:21.700] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e2b8c3a4ea4cad07 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:e2b8c3a4ea4cad07"} 10:22:21 [10:22:21.701] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.701] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b3f850d016631891 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b3f850d016631891"} 10:22:21 [10:22:21.701] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:b3f850d016631891 {"provingJobId":"1:BASE_PARITY:b3f850d016631891"} 10:22:21 [10:22:21.702] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b3f850d016631891 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:b3f850d016631891"} 10:22:21 [10:22:21.702] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:357948862ea739e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:357948862ea739e7"} 10:22:21 [10:22:21.703] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b3f850d016631891 has already been completed {"provingJobId":"1:BASE_PARITY:b3f850d016631891"} 10:22:21 [10:22:21.703] INFO: prover-client:proving-broker Proving Broker started 10:22:21 [10:22:21.704] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:398823d9a41a0270 epochNumber=1 {"provingJobId":"1:BASE_PARITY:398823d9a41a0270"} 10:22:21 [10:22:21.704] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:884e7cc4aabe5d0d epochNumber=2 {"provingJobId":"1:BASE_PARITY:884e7cc4aabe5d0d"} 10:22:22 [10:22:22.755] INFO: prover-client:proving-broker Proving Broker started 10:22:22 [10:22:22.757] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:398823d9a41a0270 settled=false {"provingJobId":"1:BASE_PARITY:398823d9a41a0270","status":"pending"} 10:22:22 [10:22:22.757] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:884e7cc4aabe5d0d settled=false {"provingJobId":"1:BASE_PARITY:884e7cc4aabe5d0d","status":"pending"} 10:22:22 [10:22:22.758] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:398823d9a41a0270 type=BASE_PARITY startedAt=1750501341704 older agent has taken job {"provingJobId":"1:BASE_PARITY:398823d9a41a0270"} 10:22:22 [10:22:22.759] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:398823d9a41a0270 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:398823d9a41a0270"} 10:22:22 [10:22:22.760] INFO: prover-client:proving-broker Proving Broker started 10:22:22 [10:22:22.761] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9ad073cabda1cd16 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ad073cabda1cd16"} 10:22:22 [10:22:22.761] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f77d08e5e030fab8 epochNumber=2 {"provingJobId":"1:BASE_PARITY:f77d08e5e030fab8"} 10:22:23 [10:22:23.762] INFO: prover-client:proving-broker Proving Broker started 10:22:23 [10:22:23.762] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9ad073cabda1cd16 settled=false {"provingJobId":"1:BASE_PARITY:9ad073cabda1cd16","status":"pending"} 10:22:23 [10:22:23.762] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f77d08e5e030fab8 settled=false {"provingJobId":"1:BASE_PARITY:f77d08e5e030fab8","status":"pending"} 10:22:23 [10:22:23.763] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9ad073cabda1cd16 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:9ad073cabda1cd16"} 10:22:23 [10:22:23.765] INFO: prover-client:proving-broker Proving Broker started 10:22:23 [10:22:23.765] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5ca2822e74fae6f5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5ca2822e74fae6f5"} 10:22:23 [10:22:23.766] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9fc2887b45279b9e epochNumber=2 {"provingJobId":"1:BASE_PARITY:9fc2887b45279b9e"} 10:22:24 [10:22:24.767] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.767] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5ca2822e74fae6f5 settled=false {"provingJobId":"1:BASE_PARITY:5ca2822e74fae6f5","status":"pending"} 10:22:24 [10:22:24.767] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9fc2887b45279b9e settled=false {"provingJobId":"1:BASE_PARITY:9fc2887b45279b9e","status":"pending"} 10:22:24 [10:22:24.768] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5ca2822e74fae6f5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5ca2822e74fae6f5"} 10:22:24 [10:22:24.768] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5ca2822e74fae6f5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5ca2822e74fae6f5"} 10:22:24 [10:22:24.770] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.770] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67c6ceca3e1bd04d epochNumber=1 {"provingJobId":"1:BASE_PARITY:67c6ceca3e1bd04d"} 10:22:24 [10:22:24.771] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ce3139a4e3e9618e epochNumber=2 {"provingJobId":"1:BASE_PARITY:ce3139a4e3e9618e"} 10:22:24 [10:22:24.771] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:67c6ceca3e1bd04d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:67c6ceca3e1bd04d"} 10:22:24 [10:22:24.772] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ce3139a4e3e9618e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ce3139a4e3e9618e"} 10:22:24 [10:22:24.773] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.773] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2b9f7a0f2ed8901f epochNumber=1 {"provingJobId":"1:BASE_PARITY:2b9f7a0f2ed8901f"} 10:22:24 [10:22:24.774] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1437cd2cfcc0f9a3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1437cd2cfcc0f9a3"} 10:22:24 [10:22:24.774] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2b9f7a0f2ed8901f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2b9f7a0f2ed8901f"} 10:22:24 [10:22:24.774] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2b9f7a0f2ed8901f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2b9f7a0f2ed8901f"} 10:22:24 [10:22:24.775] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1437cd2cfcc0f9a3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:1437cd2cfcc0f9a3"} 10:22:24 [10:22:24.776] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.776] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:17164d11043084c4 err=test error {"provingJoId":"1:BASE_PARITY:17164d11043084c4"} 10:22:24 [10:22:24.777] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.778] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b267af960d6ddbdd not found {"provingJobId":"1:BASE_PARITY:b267af960d6ddbdd"} 10:22:24 [10:22:24.779] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.779] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b402a1f0684fedf2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b402a1f0684fedf2"} 10:22:24 [10:22:24.780] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.780] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0828f22301e2641d epochNumber=1 {"provingJobId":"1:BASE_PARITY:0828f22301e2641d"} 10:22:24 [10:22:24.881] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0828f22301e2641d timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:0828f22301e2641d"} 10:22:24 [10:22:24.888] INFO: prover-client:proving-broker Proving Broker started 10:22:24 [10:22:24.889] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ede5b45e8c0264c9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ede5b45e8c0264c9"} 10:22:24 [10:22:24.990] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ede5b45e8c0264c9 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ede5b45e8c0264c9"} 10:22:24 [10:22:24.991] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:14e4ae460027369d epochNumber=10 {"provingJobId":"1:BASE_PARITY:14e4ae460027369d"} 10:22:25 [10:22:25.015] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:25 [10:22:25.118] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f739836368f922a7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f739836368f922a7"} 10:22:25 [10:22:25.272] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f739836368f922a7 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f739836368f922a7"} 10:22:25 [10:22:25.321] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.321] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:901116d468eb802c epochNumber=1 {"provingJobId":"1:BASE_PARITY:901116d468eb802c"} 10:22:25 [10:22:25.322] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:901116d468eb802c type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:901116d468eb802c"} 10:22:25 [10:22:25.323] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.323] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c3eb5d1b8b323628 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c3eb5d1b8b323628"} 10:22:25 [10:22:25.324] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c3eb5d1b8b323628 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:c3eb5d1b8b323628"} 10:22:25 [10:22:25.324] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c3eb5d1b8b323628 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:c3eb5d1b8b323628"} 10:22:25 [10:22:25.326] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.326] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2b2faadc39c26160 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2b2faadc39c26160"} 10:22:25 [10:22:25.327] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2b2faadc39c26160 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2b2faadc39c26160"} 10:22:25 [10:22:25.328] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.328] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf2a19c81a879d0b epochNumber=1 {"provingJobId":"1:BASE_PARITY:bf2a19c81a879d0b"} 10:22:25 [10:22:25.329] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bf2a19c81a879d0b type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:bf2a19c81a879d0b"} 10:22:25 [10:22:25.329] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:76ed5163ca834ce1 epochNumber=3 {"provingJobId":"1:BASE_PARITY:76ed5163ca834ce1"} 10:22:25 [10:22:25.352] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:25 [10:22:25.354] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:bf2a19c81a879d0b err=test error {"provingJoId":"1:BASE_PARITY:bf2a19c81a879d0b"} 10:22:25 [10:22:25.356] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.356] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1564cbf9a89b103e settled=false {"provingJobId":"1:BASE_PARITY:1564cbf9a89b103e","status":"pending"} 10:22:25 [10:22:25.356] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:66f704bc86d401a5 settled=false {"provingJobId":"1:BASE_PARITY:66f704bc86d401a5","status":"pending"} 10:22:25 [10:22:25.358] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.358] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:78f86f55edf1e559 settled=true {"provingJobId":"1:BASE_PARITY:78f86f55edf1e559","status":"fulfilled"} 10:22:25 [10:22:25.359] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8de2e29171f45557 settled=true {"provingJobId":"2:BASE_PARITY:8de2e29171f45557","status":"fulfilled"} 10:22:25 [10:22:25.360] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.360] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b3321eaa03ab27c4 settled=true {"provingJobId":"1:BASE_PARITY:b3321eaa03ab27c4","status":"fulfilled"} 10:22:25 [10:22:25.360] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d838d113a0b42c29 settled=false {"provingJobId":"1:BASE_PARITY:d838d113a0b42c29","status":"pending"} 10:22:25 [10:22:25.362] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.363] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:128035919b3a6fd7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:128035919b3a6fd7"} 10:22:25 [10:22:25.365] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.365] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2770a7c27b58c5aa epochNumber=1 {"provingJobId":"1:BASE_PARITY:2770a7c27b58c5aa"} 10:22:25 [10:22:25.366] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:2770a7c27b58c5aa: Error: db error: Error: db error 10:22:25 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:2770a7c27b58c5aa"} 10:22:25 [10:22:25.367] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.368] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64e525a66695c341 epochNumber=1 {"provingJobId":"1:BASE_PARITY:64e525a66695c341"} 10:22:25 [10:22:25.368] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:64e525a66695c341 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:64e525a66695c341"} 10:22:25 [10:22:25.368] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:64e525a66695c341 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:64e525a66695c341"} 10:22:25 [10:22:25.370] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.370] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:808fd648bdd087e2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:808fd648bdd087e2"} 10:22:25 [10:22:25.371] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:808fd648bdd087e2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:808fd648bdd087e2"} 10:22:25 [10:22:25.371] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:808fd648bdd087e2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:808fd648bdd087e2"} 10:22:25 [10:22:25.371] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:808fd648bdd087e2: Error: db error 10:22:25 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:808fd648bdd087e2"} 10:22:25 [10:22:25.373] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.373] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7a0bf9134f7cee20 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7a0bf9134f7cee20"} 10:22:25 [10:22:25.374] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7a0bf9134f7cee20 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7a0bf9134f7cee20"} 10:22:25 [10:22:25.374] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a0bf9134f7cee20 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:7a0bf9134f7cee20"} 10:22:25 [10:22:25.375] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.376] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:246e741f3977f00d epochNumber=1 {"provingJobId":"1:BASE_PARITY:246e741f3977f00d"} 10:22:25 [10:22:25.376] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:246e741f3977f00d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:246e741f3977f00d"} 10:22:25 [10:22:25.376] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:246e741f3977f00d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:246e741f3977f00d"} 10:22:25 [10:22:25.377] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:246e741f3977f00d jobErr=test error: Error: db error 10:22:25 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:246e741f3977f00d"} 10:22:25 [10:22:25.378] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.379] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:939f66be6f8c690d not found {"provingJobId":"1:BASE_PARITY:939f66be6f8c690d"} 10:22:25 [10:22:25.380] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.381] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:508bb77e5e5de9d3 err=test error {"provingJoId":"1:BASE_PARITY:508bb77e5e5de9d3"} 10:22:25 [10:22:25.382] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.393] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:773a966fc478f566 epochNumber=1 {"provingJobId":"1:BASE_PARITY:773a966fc478f566"} 10:22:25 [10:22:25.393] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:773a966fc478f566 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:773a966fc478f566"} 10:22:25 [10:22:25.394] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:773a966fc478f566 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:773a966fc478f566"} 10:22:25 [10:22:25.405] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:915f0d7432e63a87 epochNumber=2 {"provingJobId":"2:BASE_PARITY:915f0d7432e63a87"} 10:22:25 [10:22:25.406] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:915f0d7432e63a87 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:915f0d7432e63a87"} 10:22:25 [10:22:25.406] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:915f0d7432e63a87 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:915f0d7432e63a87"} 10:22:25 [10:22:25.468] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:ddae8d95eb603c61 epochNumber=3 {"provingJobId":"3:BASE_PARITY:ddae8d95eb603c61"} 10:22:25 [10:22:25.484] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:25 [10:22:25.530] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:8158d806a45756f1 epochNumber=4 {"provingJobId":"4:BASE_PARITY:8158d806a45756f1"} 10:22:25 [10:22:25.535] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:25 [10:22:25.591] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:79d2e1fdb89a758a epochNumber=5 {"provingJobId":"5:BASE_PARITY:79d2e1fdb89a758a"} 10:22:25 [10:22:25.613] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:25 [10:22:25.645] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.645] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4590a156ad09d97f epochNumber=42 {"provingJobId":"1:BASE_PARITY:4590a156ad09d97f"} 10:22:25 [10:22:25.646] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testAKKCD8/42 with map size 1073741824 10:22:25 [10:22:25.650] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.676] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:553dca8e9e97d604 epochNumber=1 {"provingJobId":"1:BASE_PARITY:553dca8e9e97d604"} 10:22:25 [10:22:25.680] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.680] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4b78b2b9f7ae1481 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4b78b2b9f7ae1481"} 10:22:25 [10:22:25.681] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9ME1QY/1 with map size 1073741824 10:22:25 [10:22:25.682] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.700] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ea9bcb847480eba9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ea9bcb847480eba9"} 10:22:25 [10:22:25.708] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.709] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a4900ab9189abbef epochNumber=1 {"provingJobId":"1:BASE_PARITY:a4900ab9189abbef"} 10:22:25 [10:22:25.710] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuzAezG/1 with map size 1073741824 10:22:25 [10:22:25.711] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.728] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a4900ab9189abbef epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:a4900ab9189abbef"} 10:22:25 [10:22:25.730] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.731] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f7049e838bbba82 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1f7049e838bbba82"} 10:22:25 [10:22:25.731] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtd48xc/1 with map size 1073741824 10:22:25 [10:22:25.733] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.749] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f7049e838bbba82 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1f7049e838bbba82"} 10:22:25 [10:22:25.749] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1f7049e838bbba82 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1f7049e838bbba82"} 10:22:25 [10:22:25.755] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f7049e838bbba82 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1f7049e838bbba82"} 10:22:25 [10:22:25.758] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.759] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34af0135a4bf4d08 epochNumber=1 {"provingJobId":"1:BASE_PARITY:34af0135a4bf4d08"} 10:22:25 [10:22:25.759] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhTOOr0/1 with map size 1073741824 10:22:25 [10:22:25.761] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.780] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.783] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.783] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:47ed4d95ce5e8d87 epochNumber=1 {"provingJobId":"1:BASE_PARITY:47ed4d95ce5e8d87"} 10:22:25 [10:22:25.784] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test78MZDI/1 with map size 1073741824 10:22:25 [10:22:25.785] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.802] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:47ed4d95ce5e8d87 {"provingJobId":"1:BASE_PARITY:47ed4d95ce5e8d87"} 10:22:25 [10:22:25.802] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:47ed4d95ce5e8d87 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:47ed4d95ce5e8d87"} 10:22:25 [10:22:25.802] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:47ed4d95ce5e8d87 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:47ed4d95ce5e8d87"} 10:22:25 [10:22:25.810] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.811] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59254ed9c9cd3577 epochNumber=1 {"provingJobId":"1:BASE_PARITY:59254ed9c9cd3577"} 10:22:25 [10:22:25.811] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLoU1aj/1 with map size 1073741824 10:22:25 [10:22:25.813] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.829] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:59254ed9c9cd3577 {"provingJobId":"1:BASE_PARITY:59254ed9c9cd3577"} 10:22:25 [10:22:25.830] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:59254ed9c9cd3577 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:59254ed9c9cd3577"} 10:22:25 [10:22:25.842] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.842] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a9c76415b6c3a573 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a9c76415b6c3a573"} 10:22:25 [10:22:25.843] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testr8mcG5/1 with map size 1073741824 10:22:25 [10:22:25.844] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.865] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a9c76415b6c3a573 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a9c76415b6c3a573"} 10:22:25 [10:22:25.865] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a9c76415b6c3a573 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a9c76415b6c3a573"} 10:22:25 [10:22:25.874] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.875] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0b6f9e6b9a06457 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c0b6f9e6b9a06457"} 10:22:25 [10:22:25.875] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTMfwjb/1 with map size 1073741824 10:22:25 [10:22:25.877] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.896] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c0b6f9e6b9a06457 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c0b6f9e6b9a06457"} 10:22:25 [10:22:25.897] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c0b6f9e6b9a06457 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c0b6f9e6b9a06457"} 10:22:25 [10:22:25.904] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.905] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:62939f52605109ef epochNumber=0 {"provingJobId":"1:BASE_PARITY:62939f52605109ef"} 10:22:25 [10:22:25.905] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testt67aIF/0 with map size 1073741824 10:22:25 [10:22:25.907] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.923] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:62939f52605109ef epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:62939f52605109ef"} 10:22:25 [10:22:25.924] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cd1e1add111704ea epochNumber=0 {"provingJobId":"1:BASE_PARITY:cd1e1add111704ea"} 10:22:25 [10:22:25.924] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:62939f52605109ef epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:62939f52605109ef"} 10:22:25 [10:22:25.924] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ea209525b51f3a3 epochNumber=0 {"provingJobId":"1:BASE_PARITY:4ea209525b51f3a3"} 10:22:25 [10:22:25.938] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.939] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.940] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5b448bcf370ffb0c epochNumber=2 {"provingJobId":"1:BASE_PARITY:5b448bcf370ffb0c"} 10:22:25 [10:22:25.940] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtBQzJP/2 with map size 1073741824 10:22:25 [10:22:25.941] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.957] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:492c37c16c617696 epochNumber=1 {"provingJobId":"1:BASE_PARITY:492c37c16c617696"} 10:22:25 [10:22:25.958] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtBQzJP/1 with map size 1073741824 10:22:25 [10:22:25.960] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.977] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.977] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bbc1314f50fa28dd epochNumber=1 {"provingJobId":"1:BASE_PARITY:bbc1314f50fa28dd"} 10:22:25 [10:22:25.978] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEaXZa8/1 with map size 1073741824 10:22:25 [10:22:25.979] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:25 [10:22:25.998] INFO: prover-client:proving-broker Proving Broker started 10:22:25 [10:22:25.999] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06893a244852f5f1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:06893a244852f5f1"} 10:22:25 [10:22:25.999] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVIa5ri/1 with map size 1073741824 10:22:26 [10:22:26.001] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.018] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed45a7b57582a4c0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed45a7b57582a4c0"} 10:22:26 [10:22:26.025] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:440fad760c942237 epochNumber=2 {"provingJobId":"1:BASE_PARITY:440fad760c942237"} 10:22:26 [10:22:26.026] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVIa5ri/2 with map size 1073741824 10:22:26 [10:22:26.027] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.044] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:841ebee592d1bc11 epochNumber=1 {"provingJobId":"1:BASE_PARITY:841ebee592d1bc11"} 10:22:26 [10:22:26.053] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.054] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f0258d29badbbc52 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f0258d29badbbc52"} 10:22:26 [10:22:26.054] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl31G1G/1 with map size 1073741824 10:22:26 [10:22:26.056] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.072] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4a29edab71f42f5a epochNumber=1 {"provingJobId":"1:BASE_PARITY:4a29edab71f42f5a"} 10:22:26 [10:22:26.078] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:23ee18ca2c073dbc epochNumber=2 {"provingJobId":"1:BASE_PARITY:23ee18ca2c073dbc"} 10:22:26 [10:22:26.079] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testl31G1G/2 with map size 1073741824 10:22:26 [10:22:26.081] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.100] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1b35cb16be711c1d epochNumber=1 {"provingJobId":"1:BASE_PARITY:1b35cb16be711c1d"} 10:22:26 [10:22:26.115] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f8303dee8515d095 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f8303dee8515d095"} 10:22:26 [10:22:26.116] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEQ2Kc2/1 with map size 1073741824 10:22:26 [10:22:26.119] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.134] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9ec54f4b2a702e19 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ec54f4b2a702e19"} 10:22:26 [10:22:26.141] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ad2111855086df4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:8ad2111855086df4"} 10:22:26 [10:22:26.142] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testEQ2Kc2/2 with map size 1073741824 10:22:26 [10:22:26.143] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.159] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0cfa9a7343a9df14 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0cfa9a7343a9df14"} 10:22:26 [10:22:26.169] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.170] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:21b513e6551a5e02 epochNumber=1 {"provingJobId":"1:BASE_PARITY:21b513e6551a5e02"} 10:22:26 [10:22:26.171] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiPekqh/1 with map size 1073741824 10:22:26 [10:22:26.173] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.192] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:88968a1540315912 epochNumber=1 {"provingJobId":"1:BASE_PARITY:88968a1540315912"} 10:22:26 [10:22:26.200] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:21b513e6551a5e02 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:21b513e6551a5e02"} 10:22:26 [10:22:26.209] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:088bb593b2e35a8f epochNumber=1 {"provingJobId":"1:BASE_PARITY:088bb593b2e35a8f"} 10:22:26 [10:22:26.210] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testF3jMVl/1 with map size 1073741824 10:22:26 [10:22:26.212] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3fdcfccdb0478689 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3fdcfccdb0478689"} 10:22:26 [10:22:26.235] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:088bb593b2e35a8f type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:088bb593b2e35a8f"} 10:22:26 [10:22:26.245] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.246] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:88f94136d0937246 epochNumber=1 {"provingJobId":"1:BASE_PARITY:88f94136d0937246"} 10:22:26 [10:22:26.246] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYcBoZO/1 with map size 1073741824 10:22:26 [10:22:26.248] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.264] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd8aab875162658c epochNumber=1 {"provingJobId":"1:BASE_PARITY:bd8aab875162658c"} 10:22:26 [10:22:26.271] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:88f94136d0937246 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:88f94136d0937246"} 10:22:26 [10:22:26.275] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.276] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:65c09a7f75a2fd90 epochNumber=1 {"provingJobId":"1:BASE_PARITY:65c09a7f75a2fd90"} 10:22:26 [10:22:26.277] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test993tZh/1 with map size 1073741824 10:22:26 [10:22:26.278] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.295] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:65c09a7f75a2fd90 {"provingJobId":"1:BASE_PARITY:65c09a7f75a2fd90"} 10:22:26 [10:22:26.296] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:65c09a7f75a2fd90 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:65c09a7f75a2fd90"} 10:22:26 [10:22:26.315] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34ecae392a66366c epochNumber=1 {"provingJobId":"1:BASE_PARITY:34ecae392a66366c"} 10:22:26 [10:22:26.319] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:65c09a7f75a2fd90 has already been completed {"provingJobId":"1:BASE_PARITY:65c09a7f75a2fd90"} 10:22:26 [10:22:26.323] INFO: prover-client:proving-broker Proving Broker started 10:22:26 [10:22:26.324] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bbc207944f4c0cf1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bbc207944f4c0cf1"} 10:22:26 [10:22:26.324] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpaccsj/1 with map size 1073741824 10:22:26 [10:22:26.326] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:26 [10:22:26.345] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:813ef441912de8cc epochNumber=2 {"provingJobId":"1:BASE_PARITY:813ef441912de8cc"} 10:22:26 [10:22:26.346] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpaccsj/2 with map size 1073741824 10:22:26 [10:22:26.349] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:27 [10:22:27.415] INFO: prover-client:proving-broker Proving Broker started 10:22:27 [10:22:27.422] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bbc207944f4c0cf1 settled=false {"provingJobId":"1:BASE_PARITY:bbc207944f4c0cf1","status":"pending"} 10:22:27 [10:22:27.424] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:813ef441912de8cc settled=false {"provingJobId":"1:BASE_PARITY:813ef441912de8cc","status":"pending"} 10:22:27 [10:22:27.425] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:bbc207944f4c0cf1 type=BASE_PARITY startedAt=1750501346365 older agent has taken job {"provingJobId":"1:BASE_PARITY:bbc207944f4c0cf1"} 10:22:27 [10:22:27.425] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bbc207944f4c0cf1 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:bbc207944f4c0cf1"} 10:22:27 [10:22:27.428] INFO: prover-client:proving-broker Proving Broker started 10:22:27 [10:22:27.429] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7d75b5602d608e6d epochNumber=1 {"provingJobId":"1:BASE_PARITY:7d75b5602d608e6d"} 10:22:27 [10:22:27.429] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWTCs7X/1 with map size 1073741824 10:22:27 [10:22:27.430] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:27 [10:22:27.448] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fc2faf0db310df84 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fc2faf0db310df84"} 10:22:27 [10:22:27.449] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testWTCs7X/2 with map size 1073741824 10:22:27 [10:22:27.451] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:28 [10:22:28.468] INFO: prover-client:proving-broker Proving Broker started 10:22:28 [10:22:28.470] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7d75b5602d608e6d settled=false {"provingJobId":"1:BASE_PARITY:7d75b5602d608e6d","status":"pending"} 10:22:28 [10:22:28.471] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fc2faf0db310df84 settled=false {"provingJobId":"1:BASE_PARITY:fc2faf0db310df84","status":"pending"} 10:22:28 [10:22:28.472] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7d75b5602d608e6d type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:7d75b5602d608e6d"} 10:22:28 [10:22:28.476] INFO: prover-client:proving-broker Proving Broker started 10:22:28 [10:22:28.477] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2dd4e874fb31a8d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2dd4e874fb31a8d5"} 10:22:28 [10:22:28.478] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqIB0d3/1 with map size 1073741824 10:22:28 [10:22:28.479] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:28 [10:22:28.497] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0a244131e6f02f9 epochNumber=2 {"provingJobId":"1:BASE_PARITY:c0a244131e6f02f9"} 10:22:28 [10:22:28.498] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqIB0d3/2 with map size 1073741824 10:22:28 [10:22:28.499] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.516] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.518] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2dd4e874fb31a8d5 settled=false {"provingJobId":"1:BASE_PARITY:2dd4e874fb31a8d5","status":"pending"} 10:22:29 [10:22:29.520] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c0a244131e6f02f9 settled=false {"provingJobId":"1:BASE_PARITY:c0a244131e6f02f9","status":"pending"} 10:22:29 [10:22:29.521] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2dd4e874fb31a8d5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2dd4e874fb31a8d5"} 10:22:29 [10:22:29.522] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2dd4e874fb31a8d5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2dd4e874fb31a8d5"} 10:22:29 [10:22:29.531] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.532] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b554a909e3c54298 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b554a909e3c54298"} 10:22:29 [10:22:29.533] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTpAKMm/1 with map size 1073741824 10:22:29 [10:22:29.536] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.553] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5b00d3a4ccbf704 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b5b00d3a4ccbf704"} 10:22:29 [10:22:29.555] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTpAKMm/2 with map size 1073741824 10:22:29 [10:22:29.557] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.574] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b554a909e3c54298 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b554a909e3c54298"} 10:22:29 [10:22:29.581] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b5b00d3a4ccbf704 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b5b00d3a4ccbf704"} 10:22:29 [10:22:29.590] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.591] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:16ef199db40aeb65 epochNumber=1 {"provingJobId":"1:BASE_PARITY:16ef199db40aeb65"} 10:22:29 [10:22:29.591] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5B1YqQ/1 with map size 1073741824 10:22:29 [10:22:29.592] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.611] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7c7c84da2d474fad epochNumber=2 {"provingJobId":"1:BASE_PARITY:7c7c84da2d474fad"} 10:22:29 [10:22:29.612] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5B1YqQ/2 with map size 1073741824 10:22:29 [10:22:29.614] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.631] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:16ef199db40aeb65 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:16ef199db40aeb65"} 10:22:29 [10:22:29.632] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:16ef199db40aeb65 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:16ef199db40aeb65"} 10:22:29 [10:22:29.638] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7c7c84da2d474fad type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:7c7c84da2d474fad"} 10:22:29 [10:22:29.650] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.651] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d9a0f6f97ba3a886 err=test error {"provingJoId":"1:BASE_PARITY:d9a0f6f97ba3a886"} 10:22:29 [10:22:29.655] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.656] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e2b7bcfbea80e412 not found {"provingJobId":"1:BASE_PARITY:e2b7bcfbea80e412"} 10:22:29 [10:22:29.661] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.662] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6e7d3cc8e7a5e0c0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6e7d3cc8e7a5e0c0"} 10:22:29 [10:22:29.665] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyRe1iq/1 with map size 1073741824 10:22:29 [10:22:29.666] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.690] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.691] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:927bab226918f7e4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:927bab226918f7e4"} 10:22:29 [10:22:29.692] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testooqgtQ/1 with map size 1073741824 10:22:29 [10:22:29.695] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.817] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:927bab226918f7e4 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:927bab226918f7e4"} 10:22:29 [10:22:29.825] INFO: prover-client:proving-broker Proving Broker started 10:22:29 [10:22:29.826] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f92134103351fc30 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f92134103351fc30"} 10:22:29 [10:22:29.827] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4buMXl/1 with map size 1073741824 10:22:29 [10:22:29.828] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:29 [10:22:29.950] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f92134103351fc30 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f92134103351fc30"} 10:22:29 [10:22:29.957] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:36359e62975a5e99 epochNumber=10 {"provingJobId":"1:BASE_PARITY:36359e62975a5e99"} 10:22:29 [10:22:29.958] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test4buMXl/10 with map size 1073741824 10:22:29 [10:22:29.959] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.002] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:30 [10:22:30.002] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:22:30 [10:22:30.108] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.109] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b8193999ddbc8c40 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b8193999ddbc8c40"} 10:22:30 [10:22:30.110] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testk6LGtW/1 with map size 1073741824 10:22:30 [10:22:30.113] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.286] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b8193999ddbc8c40 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:b8193999ddbc8c40"} 10:22:30 [10:22:30.335] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.336] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe85146c0239e2e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fe85146c0239e2e7"} 10:22:30 [10:22:30.336] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3nZUt9/1 with map size 1073741824 10:22:30 [10:22:30.337] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.355] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fe85146c0239e2e7 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:fe85146c0239e2e7"} 10:22:30 [10:22:30.362] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.363] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:009581bf4ebd589a epochNumber=1 {"provingJobId":"1:BASE_PARITY:009581bf4ebd589a"} 10:22:30 [10:22:30.364] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test55X3v4/1 with map size 1073741824 10:22:30 [10:22:30.366] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.382] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:009581bf4ebd589a type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:009581bf4ebd589a"} 10:22:30 [10:22:30.382] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:009581bf4ebd589a type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:009581bf4ebd589a"} 10:22:30 [10:22:30.391] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.391] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ddca5cc388758728 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ddca5cc388758728"} 10:22:30 [10:22:30.392] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSzM6Xk/1 with map size 1073741824 10:22:30 [10:22:30.393] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.408] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ddca5cc388758728 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ddca5cc388758728"} 10:22:30 [10:22:30.421] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.422] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9fbb6302530b3239 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9fbb6302530b3239"} 10:22:30 [10:22:30.423] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlaESr5/1 with map size 1073741824 10:22:30 [10:22:30.425] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.442] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9fbb6302530b3239 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:9fbb6302530b3239"} 10:22:30 [10:22:30.443] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8912c64130fc62a7 epochNumber=3 {"provingJobId":"1:BASE_PARITY:8912c64130fc62a7"} 10:22:30 [10:22:30.443] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testlaESr5/3 with map size 1073741824 10:22:30 [10:22:30.445] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.473] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:30 [10:22:30.473] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:22:30 [10:22:30.486] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:9fbb6302530b3239 err=test error {"provingJoId":"1:BASE_PARITY:9fbb6302530b3239"} 10:22:30 [10:22:30.490] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXa14bP/1 with map size 1073741824 10:22:30 [10:22:30.492] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.513] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXa14bP/2 with map size 1073741824 10:22:30 [10:22:30.516] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.534] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.536] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a79ddb40cb7e66e5 settled=false {"provingJobId":"1:BASE_PARITY:a79ddb40cb7e66e5","status":"pending"} 10:22:30 [10:22:30.538] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8ec0bafc4615d8f2 settled=false {"provingJobId":"1:BASE_PARITY:8ec0bafc4615d8f2","status":"pending"} 10:22:30 [10:22:30.546] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfzYj9U/1 with map size 1073741824 10:22:30 [10:22:30.548] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.566] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfzYj9U/2 with map size 1073741824 10:22:30 [10:22:30.572] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.601] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.603] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7022bdde7596261b settled=true {"provingJobId":"1:BASE_PARITY:7022bdde7596261b","status":"fulfilled"} 10:22:30 [10:22:30.604] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:2e54782efb854dc6 settled=true {"provingJobId":"2:BASE_PARITY:2e54782efb854dc6","status":"fulfilled"} 10:22:30 [10:22:30.610] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtI4MND/1 with map size 1073741824 10:22:30 [10:22:30.614] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.638] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtI4MND/2 with map size 1073741824 10:22:30 [10:22:30.643] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.662] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.664] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3cc210bbb51e6183 settled=true {"provingJobId":"1:BASE_PARITY:3cc210bbb51e6183","status":"fulfilled"} 10:22:30 [10:22:30.666] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8e194ea1bc092585 settled=false {"provingJobId":"1:BASE_PARITY:8e194ea1bc092585","status":"pending"} 10:22:30 [10:22:30.672] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.673] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2d939311f1d02083 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2d939311f1d02083"} 10:22:30 [10:22:30.674] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqWv0Nb/1 with map size 1073741824 10:22:30 [10:22:30.677] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.700] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.701] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b6c099c3ae355af epochNumber=1 {"provingJobId":"1:BASE_PARITY:6b6c099c3ae355af"} 10:22:30 [10:22:30.702] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:6b6c099c3ae355af: Error: db error: Error: db error 10:22:30 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:6b6c099c3ae355af"} 10:22:30 [10:22:30.707] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.707] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:045df8fe788ac19f epochNumber=1 {"provingJobId":"1:BASE_PARITY:045df8fe788ac19f"} 10:22:30 [10:22:30.709] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFMnzEf/1 with map size 1073741824 10:22:30 [10:22:30.711] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.731] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:045df8fe788ac19f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:045df8fe788ac19f"} 10:22:30 [10:22:30.731] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:045df8fe788ac19f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:045df8fe788ac19f"} 10:22:30 [10:22:30.742] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.742] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8b6aff31f27c3af9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8b6aff31f27c3af9"} 10:22:30 [10:22:30.744] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOuwT7f/1 with map size 1073741824 10:22:30 [10:22:30.746] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.762] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8b6aff31f27c3af9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8b6aff31f27c3af9"} 10:22:30 [10:22:30.762] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8b6aff31f27c3af9 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:8b6aff31f27c3af9"} 10:22:30 [10:22:30.763] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:8b6aff31f27c3af9: Error: db error 10:22:30 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:8b6aff31f27c3af9"} 10:22:30 [10:22:30.767] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.767] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8349d9b9c4dbf666 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8349d9b9c4dbf666"} 10:22:30 [10:22:30.768] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testztBOgp/1 with map size 1073741824 10:22:30 [10:22:30.770] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.790] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8349d9b9c4dbf666 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8349d9b9c4dbf666"} 10:22:30 [10:22:30.791] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8349d9b9c4dbf666 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8349d9b9c4dbf666"} 10:22:30 [10:22:30.801] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.802] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a6496bae98ed48da epochNumber=1 {"provingJobId":"1:BASE_PARITY:a6496bae98ed48da"} 10:22:30 [10:22:30.803] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMemp2C/1 with map size 1073741824 10:22:30 [10:22:30.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.820] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a6496bae98ed48da type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a6496bae98ed48da"} 10:22:30 [10:22:30.821] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a6496bae98ed48da type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a6496bae98ed48da"} 10:22:30 [10:22:30.822] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:a6496bae98ed48da jobErr=test error: Error: db error 10:22:30 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:a6496bae98ed48da"} 10:22:30 [10:22:30.825] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.826] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:20e2f967947ba988 not found {"provingJobId":"1:BASE_PARITY:20e2f967947ba988"} 10:22:30 [10:22:30.829] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.830] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:26e41eb070efa515 err=test error {"provingJoId":"1:BASE_PARITY:26e41eb070efa515"} 10:22:30 [10:22:30.835] INFO: prover-client:proving-broker Proving Broker started 10:22:30 [10:22:30.846] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6ccda5bc97fc8562 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6ccda5bc97fc8562"} 10:22:30 [10:22:30.847] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyn6KR2/1 with map size 1073741824 10:22:30 [10:22:30.848] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.867] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6ccda5bc97fc8562 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6ccda5bc97fc8562"} 10:22:30 [10:22:30.868] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6ccda5bc97fc8562 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6ccda5bc97fc8562"} 10:22:30 [10:22:30.885] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:c36e99a633df4282 epochNumber=2 {"provingJobId":"2:BASE_PARITY:c36e99a633df4282"} 10:22:30 [10:22:30.886] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyn6KR2/2 with map size 1073741824 10:22:30 [10:22:30.889] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:30 [10:22:30.910] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:c36e99a633df4282 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:c36e99a633df4282"} 10:22:30 [10:22:30.910] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:c36e99a633df4282 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:c36e99a633df4282"} 10:22:30 [10:22:30.979] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:e4d7216ad8c0ee48 epochNumber=3 {"provingJobId":"3:BASE_PARITY:e4d7216ad8c0ee48"} 10:22:30 [10:22:30.980] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testyn6KR2/3 with map size 1073741824 10:22:30 [10:22:30.984] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:31 [10:22:31.012] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:31 [10:22:31.013] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 10:22:31 [10:22:31.062] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:9531ceac3a78d52c epochNumber=4 {"provingJobId":"4:BASE_PARITY:9531ceac3a78d52c"} 10:22:31 [10:22:31.064] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testyn6KR2/4 with map size 1073741824 10:22:31 [10:22:31.067] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:31 [10:22:31.091] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:31 [10:22:31.091] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 10:22:31 [10:22:31.146] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:23d714a176bd3d54 epochNumber=5 {"provingJobId":"5:BASE_PARITY:23d714a176bd3d54"} 10:22:31 [10:22:31.147] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testyn6KR2/5 with map size 1073741824 10:22:31 [10:22:31.150] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 10:22:31 [10:22:31.194] INFO: prover-client:proving-broker Cleaned up jobs=1 10:22:31 [10:22:31.195] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 10:22:31 PASS src/proving_broker/proving_broker.test.ts (12.132 s) 10:22:31 ProvingBroker 10:22:31 Producer API 10:22:31 ✓ refuses stale jobs (38 ms) 10:22:31 ✓ enqueues jobs (4 ms) 10:22:31 ✓ ignores duplicate jobs (4 ms) 10:22:31 ✓ reports correct status when enqueuing repeat jobs (9 ms) 10:22:31 ✓ throws an error in case of duplicate job IDs (22 ms) 10:22:31 ✓ returns not-found status for non-existing jobs (1 ms) 10:22:31 ✓ cancels jobs in queue (3 ms) 10:22:31 ✓ cancels jobs in-progress (2 ms) 10:22:31 ✓ returns job result if successful (2 ms) 10:22:31 ✓ returns job error if failed (2 ms) 10:22:31 ✓ correctly returns job status for concurrent writes (2 ms) 10:22:31 ✓ refuses stale jobs (35 ms) 10:22:31 ✓ enqueues jobs (27 ms) 10:22:31 ✓ ignores duplicate jobs (22 ms) 10:22:31 ✓ reports correct status when enqueuing repeat jobs (27 ms) 10:22:31 ✓ throws an error in case of duplicate job IDs (21 ms) 10:22:31 ✓ returns not-found status for non-existing jobs (2 ms) 10:22:31 ✓ cancels jobs in queue (28 ms) 10:22:31 ✓ cancels jobs in-progress (31 ms) 10:22:31 ✓ returns job result if successful (32 ms) 10:22:31 ✓ returns job error if failed (30 ms) 10:22:31 ✓ correctly returns job status for concurrent writes (34 ms) 10:22:31 Consumer API 10:22:31 ✓ returns undefined if no jobs are available (1 ms) 10:22:31 ✓ returns jobs in priority order (2 ms) 10:22:31 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 10:22:31 ✓ returns a job if it is in the allowList (2 ms) 10:22:31 ✓ returns the most important job if it is in the allowList (2 ms) 10:22:31 ✓ returns any job if filter is empty (2 ms) 10:22:31 ✓ returns a new job when reporting job success (2 ms) 10:22:31 ✓ returns a new job when reporting permanent error (2 ms) 10:22:31 ✓ returns a new job when reporting retry-able error (1 ms) 10:22:31 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 10:22:31 ✓ returns a new job if job is already in progress elsewhere (1056 ms) 10:22:31 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 10:22:31 ✓ avoids sending a completed job to a new agent after a restart (1005 ms) 10:22:31 ✓ tracks job result if in progress (2 ms) 10:22:31 ✓ tracks job result even if job is in queue (2 ms) 10:22:31 ✓ ignores reported job error if unknown job (2 ms) 10:22:31 ✓ ignores job result if unknown job (1 ms) 10:22:31 ✓ returns undefined if no jobs are available (2 ms) 10:22:31 ✓ returns jobs in priority order (37 ms) 10:22:31 ✓ returns undefined if no jobs are available for the given allowList (21 ms) 10:22:31 ✓ returns a job if it is in the allowList (54 ms) 10:22:31 ✓ returns the most important job if it is in the allowList (62 ms) 10:22:31 ✓ returns any job if filter is empty (54 ms) 10:22:31 ✓ returns a new job when reporting job success (38 ms) 10:22:31 ✓ returns a new job when reporting permanent error (36 ms) 10:22:31 ✓ returns a new job when reporting retry-able error (30 ms) 10:22:31 ✓ returns a new job when reporting progress if current one is cancelled (46 ms) 10:22:31 ✓ returns a new job if job is already in progress elsewhere (1106 ms) 10:22:31 ✓ avoids sending the same job to a new agent after a restart (1047 ms) 10:22:31 ✓ avoids sending a completed job to a new agent after a restart (1054 ms) 10:22:31 ✓ tracks job result if in progress (58 ms) 10:22:31 ✓ tracks job result even if job is in queue (59 ms) 10:22:31 ✓ ignores reported job error if unknown job (4 ms) 10:22:31 ✓ ignores job result if unknown job (5 ms) 10:22:31 Timeouts 10:22:31 ✓ tracks in progress jobs (1 ms) 10:22:31 ✓ re-enqueues jobs that time out (108 ms) 10:22:31 ✓ cancel stale jobs that time out (229 ms) 10:22:31 ✓ keeps the jobs in progress while it is alive (202 ms) 10:22:31 ✓ tracks in progress jobs (28 ms) 10:22:31 ✓ re-enqueues jobs that time out (135 ms) 10:22:31 ✓ cancel stale jobs that time out (282 ms) 10:22:31 ✓ keeps the jobs in progress while it is alive (227 ms) 10:22:31 Retries 10:22:31 ✓ retries jobs (2 ms) 10:22:31 ✓ retries up to a maximum number of times (2 ms) 10:22:31 ✓ passing retry=false does not retry (2 ms) 10:22:31 ✓ does not retry if job is stale (28 ms) 10:22:31 ✓ retries jobs (26 ms) 10:22:31 ✓ retries up to a maximum number of times (27 ms) 10:22:31 ✓ passing retry=false does not retry (27 ms) 10:22:31 ✓ does not retry if job is stale (72 ms) 10:22:31 Database management 10:22:31 ✓ re-enqueues proof requests on start (2 ms) 10:22:31 ✓ restores proof results on start (1 ms) 10:22:31 ✓ only re-enqueues unfinished jobs (2 ms) 10:22:31 ✓ saves job when enqueued (3 ms) 10:22:31 ✓ does not retain job if database fails to save (3 ms) 10:22:31 ✓ saves job result (2 ms) 10:22:31 ✓ saves result even if database fails to save (3 ms) 10:22:31 ✓ saves job error (3 ms) 10:22:31 ✓ saves job error even if database fails to save (3 ms) 10:22:31 ✓ does not save job result if job is unknown (2 ms) 10:22:31 ✓ does not save job error if job is unknown (1 ms) 10:22:31 ✓ cleans up old jobs periodically (261 ms) 10:22:31 ✓ re-enqueues proof requests on start (55 ms) 10:22:31 ✓ restores proof results on start (63 ms) 10:22:31 ✓ only re-enqueues unfinished jobs (62 ms) 10:22:31 ✓ saves job when enqueued (28 ms) 10:22:31 ✓ does not retain job if database fails to save (5 ms) 10:22:31 ✓ saves job result (37 ms) 10:22:31 ✓ saves result even if database fails to save (25 ms) 10:22:31 ✓ saves job error (33 ms) 10:22:31 ✓ saves job error even if database fails to save (24 ms) 10:22:31 ✓ does not save job result if job is unknown (3 ms) 10:22:31 ✓ does not save job error if job is unknown (4 ms) 10:22:31 ✓ cleans up old jobs periodically (395 ms) 10:22:31 10:22:31 Test Suites: 1 passed, 1 total 10:22:31 Tests: 96 passed, 96 total 10:22:31 Snapshots: 0 total 10:22:31 Time: 12.21 s 10:22:31 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 10:22:31 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?