Parent Log: http://ci.aztec-labs.com/7ea8a95c2d722890 Command: e49a94ea893520f8 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/e9a2301883801edbe36ea8bba05dcca7e714f30a Env: REF_NAME=gh-readonly-queue/master/pr-15131-464e1a9d9b40b5188586643b44bc52bffccbc3e6 CURRENT_VERSION=0.87.8 CI_FULL=0 Date: Wed Jun 18 12:58:01 UTC 2025 System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15131_arm64_a1-fast Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_master 12:58:04 [12:58:04.079] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.085] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:559f7983acf6fb54 epochNumber=42 {"provingJobId":"1:BASE_PARITY:559f7983acf6fb54"} 12:58:04 [12:58:04.087] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:41f79e220640d3eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:41f79e220640d3eb"} 12:58:04 [12:58:04.111] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.111] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:600efbce788b53be epochNumber=1 {"provingJobId":"1:BASE_PARITY:600efbce788b53be"} 12:58:04 [12:58:04.113] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:21f3a809283508cd epochNumber=1 {"provingJobId":"1:BASE_PARITY:21f3a809283508cd"} 12:58:04 [12:58:04.114] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8369f105e40c4c1f epochNumber=1 {"provingJobId":"1:BASE_PARITY:8369f105e40c4c1f"} 12:58:04 [12:58:04.116] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:8369f105e40c4c1f epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:8369f105e40c4c1f"} 12:58:04 [12:58:04.118] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3037ef36fbda7d33 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3037ef36fbda7d33"} 12:58:04 [12:58:04.120] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3037ef36fbda7d33 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3037ef36fbda7d33"} 12:58:04 [12:58:04.121] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3037ef36fbda7d33 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3037ef36fbda7d33"} 12:58:04 [12:58:04.121] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3037ef36fbda7d33 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3037ef36fbda7d33"} 12:58:04 [12:58:04.122] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.123] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:711fd85ca2c455bd epochNumber=1 {"provingJobId":"1:BASE_PARITY:711fd85ca2c455bd"} 12:58:04 [12:58:04.141] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.142] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.143] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9672e301a19587a5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9672e301a19587a5"} 12:58:04 [12:58:04.146] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:9672e301a19587a5 {"provingJobId":"1:BASE_PARITY:9672e301a19587a5"} 12:58:04 [12:58:04.147] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9672e301a19587a5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9672e301a19587a5"} 12:58:04 [12:58:04.147] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9672e301a19587a5 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:9672e301a19587a5"} 12:58:04 [12:58:04.148] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.149] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c984b2eb157b73d1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c984b2eb157b73d1"} 12:58:04 [12:58:04.150] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c984b2eb157b73d1 {"provingJobId":"1:BASE_PARITY:c984b2eb157b73d1"} 12:58:04 [12:58:04.150] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c984b2eb157b73d1 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c984b2eb157b73d1"} 12:58:04 [12:58:04.151] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.152] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d3ed73c3683e5a7a epochNumber=1 {"provingJobId":"1:BASE_PARITY:d3ed73c3683e5a7a"} 12:58:04 [12:58:04.152] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d3ed73c3683e5a7a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d3ed73c3683e5a7a"} 12:58:04 [12:58:04.153] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d3ed73c3683e5a7a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d3ed73c3683e5a7a"} 12:58:04 [12:58:04.154] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.155] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e36fc816b03afe96 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e36fc816b03afe96"} 12:58:04 [12:58:04.155] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e36fc816b03afe96 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e36fc816b03afe96"} 12:58:04 [12:58:04.155] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e36fc816b03afe96 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:e36fc816b03afe96"} 12:58:04 [12:58:04.156] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.156] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04afcde0d0e91c2f epochNumber=0 {"provingJobId":"1:BASE_PARITY:04afcde0d0e91c2f"} 12:58:04 [12:58:04.157] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:04afcde0d0e91c2f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:04afcde0d0e91c2f"} 12:58:04 [12:58:04.157] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:23d5ab34f630b7fa epochNumber=0 {"provingJobId":"1:BASE_PARITY:23d5ab34f630b7fa"} 12:58:04 [12:58:04.157] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:04afcde0d0e91c2f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:04afcde0d0e91c2f"} 12:58:04 [12:58:04.157] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8f77f4b2ff792fbe epochNumber=0 {"provingJobId":"1:BASE_PARITY:8f77f4b2ff792fbe"} 12:58:04 [12:58:04.159] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.159] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.160] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7dd8ff17f14d7dec epochNumber=2 {"provingJobId":"1:BASE_PARITY:7dd8ff17f14d7dec"} 12:58:04 [12:58:04.160] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6af05b61fa247eb5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6af05b61fa247eb5"} 12:58:04 [12:58:04.161] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.162] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:884bd485a491a109 epochNumber=1 {"provingJobId":"1:BASE_PARITY:884bd485a491a109"} 12:58:04 [12:58:04.162] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.163] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:99459649f4c31ff7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:99459649f4c31ff7"} 12:58:04 [12:58:04.163] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1175ab08eb0a03a6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1175ab08eb0a03a6"} 12:58:04 [12:58:04.163] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51370b7ea69646bb epochNumber=2 {"provingJobId":"1:BASE_PARITY:51370b7ea69646bb"} 12:58:04 [12:58:04.164] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:84a14708765f32d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:84a14708765f32d7"} 12:58:04 [12:58:04.165] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f6020b17c9cea0b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f6020b17c9cea0b6"} 12:58:04 [12:58:04.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f816678c6f9bde56 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f816678c6f9bde56"} 12:58:04 [12:58:04.166] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:701f0679e004fadd epochNumber=2 {"provingJobId":"1:BASE_PARITY:701f0679e004fadd"} 12:58:04 [12:58:04.166] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0fa41681fca0c59 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0fa41681fca0c59"} 12:58:04 [12:58:04.167] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.167] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:74f226fbcc1507bb epochNumber=1 {"provingJobId":"1:BASE_PARITY:74f226fbcc1507bb"} 12:58:04 [12:58:04.167] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d3b6ebbfbc3e15d1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d3b6ebbfbc3e15d1"} 12:58:04 [12:58:04.168] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1e318636c0b74a08 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1e318636c0b74a08"} 12:58:04 [12:58:04.168] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7097e8e2be3b3a00 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7097e8e2be3b3a00"} 12:58:04 [12:58:04.169] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.169] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0d16f0c702e63c9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b0d16f0c702e63c9"} 12:58:04 [12:58:04.170] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9482ca4998b820c6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9482ca4998b820c6"} 12:58:04 [12:58:04.170] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b0d16f0c702e63c9 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b0d16f0c702e63c9"} 12:58:04 [12:58:04.171] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.171] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ce53c5b50fed4b03 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ce53c5b50fed4b03"} 12:58:04 [12:58:04.172] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba02a273da8de8ca epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba02a273da8de8ca"} 12:58:04 [12:58:04.172] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ce53c5b50fed4b03 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:ce53c5b50fed4b03"} 12:58:04 [12:58:04.173] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.173] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a7e030e6014ede99 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a7e030e6014ede99"} 12:58:04 [12:58:04.173] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c0dc5fa2008ac86 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1c0dc5fa2008ac86"} 12:58:04 [12:58:04.174] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a7e030e6014ede99 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:a7e030e6014ede99"} 12:58:04 [12:58:04.174] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.175] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61377ccad51cf64d epochNumber=1 {"provingJobId":"1:BASE_PARITY:61377ccad51cf64d"} 12:58:04 [12:58:04.175] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:61377ccad51cf64d {"provingJobId":"1:BASE_PARITY:61377ccad51cf64d"} 12:58:04 [12:58:04.176] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:61377ccad51cf64d type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:61377ccad51cf64d"} 12:58:04 [12:58:04.176] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a264c61418399558 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a264c61418399558"} 12:58:04 [12:58:04.176] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:61377ccad51cf64d has already been completed {"provingJobId":"1:BASE_PARITY:61377ccad51cf64d"} 12:58:04 [12:58:04.177] INFO: prover-client:proving-broker Proving Broker started 12:58:04 [12:58:04.178] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8d4a99114d18171d epochNumber=1 {"provingJobId":"1:BASE_PARITY:8d4a99114d18171d"} 12:58:04 [12:58:04.178] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d150cd3d953dc953 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d150cd3d953dc953"} 12:58:05 [12:58:05.229] INFO: prover-client:proving-broker Proving Broker started 12:58:05 [12:58:05.229] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8d4a99114d18171d settled=false {"provingJobId":"1:BASE_PARITY:8d4a99114d18171d","status":"pending"} 12:58:05 [12:58:05.229] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d150cd3d953dc953 settled=false {"provingJobId":"1:BASE_PARITY:d150cd3d953dc953","status":"pending"} 12:58:05 [12:58:05.230] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:8d4a99114d18171d type=BASE_PARITY startedAt=1750251484178 older agent has taken job {"provingJobId":"1:BASE_PARITY:8d4a99114d18171d"} 12:58:05 [12:58:05.230] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8d4a99114d18171d type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:8d4a99114d18171d"} 12:58:05 [12:58:05.231] INFO: prover-client:proving-broker Proving Broker started 12:58:05 [12:58:05.232] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e4dd864218995d6f epochNumber=1 {"provingJobId":"1:BASE_PARITY:e4dd864218995d6f"} 12:58:05 [12:58:05.232] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:817ea4bf2e02dd06 epochNumber=2 {"provingJobId":"1:BASE_PARITY:817ea4bf2e02dd06"} 12:58:06 [12:58:06.232] INFO: prover-client:proving-broker Proving Broker started 12:58:06 [12:58:06.232] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e4dd864218995d6f settled=false {"provingJobId":"1:BASE_PARITY:e4dd864218995d6f","status":"pending"} 12:58:06 [12:58:06.232] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:817ea4bf2e02dd06 settled=false {"provingJobId":"1:BASE_PARITY:817ea4bf2e02dd06","status":"pending"} 12:58:06 [12:58:06.233] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e4dd864218995d6f type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:e4dd864218995d6f"} 12:58:06 [12:58:06.234] INFO: prover-client:proving-broker Proving Broker started 12:58:06 [12:58:06.234] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:588ce89a5bc1de2d epochNumber=1 {"provingJobId":"1:BASE_PARITY:588ce89a5bc1de2d"} 12:58:06 [12:58:06.235] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b1e17367e135725 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9b1e17367e135725"} 12:58:07 [12:58:07.235] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.236] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:588ce89a5bc1de2d settled=false {"provingJobId":"1:BASE_PARITY:588ce89a5bc1de2d","status":"pending"} 12:58:07 [12:58:07.236] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9b1e17367e135725 settled=false {"provingJobId":"1:BASE_PARITY:9b1e17367e135725","status":"pending"} 12:58:07 [12:58:07.236] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:588ce89a5bc1de2d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:588ce89a5bc1de2d"} 12:58:07 [12:58:07.237] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:588ce89a5bc1de2d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:588ce89a5bc1de2d"} 12:58:07 [12:58:07.238] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.238] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4307cad1bebfd1ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:4307cad1bebfd1ee"} 12:58:07 [12:58:07.238] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d6106db8b4a10e35 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d6106db8b4a10e35"} 12:58:07 [12:58:07.239] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4307cad1bebfd1ee type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4307cad1bebfd1ee"} 12:58:07 [12:58:07.239] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d6106db8b4a10e35 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:d6106db8b4a10e35"} 12:58:07 [12:58:07.240] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.241] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba35db3d5829ac53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba35db3d5829ac53"} 12:58:07 [12:58:07.241] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb4877c06f8fdab1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:eb4877c06f8fdab1"} 12:58:07 [12:58:07.241] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ba35db3d5829ac53 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ba35db3d5829ac53"} 12:58:07 [12:58:07.241] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ba35db3d5829ac53 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ba35db3d5829ac53"} 12:58:07 [12:58:07.242] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:eb4877c06f8fdab1 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:eb4877c06f8fdab1"} 12:58:07 [12:58:07.243] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.243] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d3c0896c6a644c0b err=test error {"provingJoId":"1:BASE_PARITY:d3c0896c6a644c0b"} 12:58:07 [12:58:07.244] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.244] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bd9440fa982bd973 not found {"provingJobId":"1:BASE_PARITY:bd9440fa982bd973"} 12:58:07 [12:58:07.245] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.246] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a55566e6b1bcd00c epochNumber=1 {"provingJobId":"1:BASE_PARITY:a55566e6b1bcd00c"} 12:58:07 [12:58:07.247] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.247] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ae84d2bb7d1a9169 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ae84d2bb7d1a9169"} 12:58:07 [12:58:07.348] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ae84d2bb7d1a9169 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ae84d2bb7d1a9169"} 12:58:07 [12:58:07.355] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.355] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ae46137795d3d773 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ae46137795d3d773"} 12:58:07 [12:58:07.456] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ae46137795d3d773 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ae46137795d3d773"} 12:58:07 [12:58:07.462] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a0d8da9794902463 epochNumber=10 {"provingJobId":"1:BASE_PARITY:a0d8da9794902463"} 12:58:07 [12:58:07.482] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:07 [12:58:07.588] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.589] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:26901074357c9b56 epochNumber=1 {"provingJobId":"1:BASE_PARITY:26901074357c9b56"} 12:58:07 [12:58:07.740] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:26901074357c9b56 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:26901074357c9b56"} 12:58:07 [12:58:07.791] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.792] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c259e7b11eb5764a epochNumber=1 {"provingJobId":"1:BASE_PARITY:c259e7b11eb5764a"} 12:58:07 [12:58:07.793] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c259e7b11eb5764a type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:c259e7b11eb5764a"} 12:58:07 [12:58:07.794] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.794] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dea0462feeb2b7b4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dea0462feeb2b7b4"} 12:58:07 [12:58:07.794] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:dea0462feeb2b7b4 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:dea0462feeb2b7b4"} 12:58:07 [12:58:07.795] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dea0462feeb2b7b4 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:dea0462feeb2b7b4"} 12:58:07 [12:58:07.796] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.796] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:277c8027b1e089c7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:277c8027b1e089c7"} 12:58:07 [12:58:07.797] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:277c8027b1e089c7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:277c8027b1e089c7"} 12:58:07 [12:58:07.797] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.798] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f0ae55a115bf3b50 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f0ae55a115bf3b50"} 12:58:07 [12:58:07.798] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f0ae55a115bf3b50 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:f0ae55a115bf3b50"} 12:58:07 [12:58:07.799] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:511b8adcc80e941c epochNumber=3 {"provingJobId":"1:BASE_PARITY:511b8adcc80e941c"} 12:58:07 [12:58:07.822] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:07 [12:58:07.824] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f0ae55a115bf3b50 err=test error {"provingJoId":"1:BASE_PARITY:f0ae55a115bf3b50"} 12:58:07 [12:58:07.825] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.825] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a70665e45788aca5 settled=false {"provingJobId":"1:BASE_PARITY:a70665e45788aca5","status":"pending"} 12:58:07 [12:58:07.826] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:27a108439bb41ec7 settled=false {"provingJobId":"1:BASE_PARITY:27a108439bb41ec7","status":"pending"} 12:58:07 [12:58:07.827] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.827] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bef6814ecf5f2fcd settled=true {"provingJobId":"1:BASE_PARITY:bef6814ecf5f2fcd","status":"fulfilled"} 12:58:07 [12:58:07.828] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:b3d71bd4f8e271ac settled=true {"provingJobId":"2:BASE_PARITY:b3d71bd4f8e271ac","status":"fulfilled"} 12:58:07 [12:58:07.829] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.829] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:68f655dff7569054 settled=true {"provingJobId":"1:BASE_PARITY:68f655dff7569054","status":"fulfilled"} 12:58:07 [12:58:07.829] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7e7ad058e743268a settled=false {"provingJobId":"1:BASE_PARITY:7e7ad058e743268a","status":"pending"} 12:58:07 [12:58:07.830] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.831] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c6cf32bfa587e38 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c6cf32bfa587e38"} 12:58:07 [12:58:07.833] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.833] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca04fda789421b51 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ca04fda789421b51"} 12:58:07 [12:58:07.834] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:ca04fda789421b51: Error: db error: Error: db error 12:58:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:ca04fda789421b51"} 12:58:07 [12:58:07.835] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.835] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec916efbb6694bf1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ec916efbb6694bf1"} 12:58:07 [12:58:07.836] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ec916efbb6694bf1 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ec916efbb6694bf1"} 12:58:07 [12:58:07.836] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ec916efbb6694bf1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ec916efbb6694bf1"} 12:58:07 [12:58:07.837] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.837] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:05965496786cecae epochNumber=1 {"provingJobId":"1:BASE_PARITY:05965496786cecae"} 12:58:07 [12:58:07.838] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:05965496786cecae type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:05965496786cecae"} 12:58:07 [12:58:07.838] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:05965496786cecae type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:05965496786cecae"} 12:58:07 [12:58:07.838] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:05965496786cecae: Error: db error 12:58:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:05965496786cecae"} 12:58:07 [12:58:07.839] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.840] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1dcbd741c04ff2ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:1dcbd741c04ff2ee"} 12:58:07 [12:58:07.840] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1dcbd741c04ff2ee type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1dcbd741c04ff2ee"} 12:58:07 [12:58:07.840] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1dcbd741c04ff2ee type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:1dcbd741c04ff2ee"} 12:58:07 [12:58:07.841] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.841] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2144b9a1b76b8369 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2144b9a1b76b8369"} 12:58:07 [12:58:07.842] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2144b9a1b76b8369 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2144b9a1b76b8369"} 12:58:07 [12:58:07.842] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2144b9a1b76b8369 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2144b9a1b76b8369"} 12:58:07 [12:58:07.842] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:2144b9a1b76b8369 jobErr=test error: Error: db error 12:58:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:2144b9a1b76b8369"} 12:58:07 [12:58:07.843] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.844] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:77996546898fc8bc not found {"provingJobId":"1:BASE_PARITY:77996546898fc8bc"} 12:58:07 [12:58:07.845] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.845] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8ac7020f1069f166 err=test error {"provingJoId":"1:BASE_PARITY:8ac7020f1069f166"} 12:58:07 [12:58:07.846] INFO: prover-client:proving-broker Proving Broker started 12:58:07 [12:58:07.857] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c3aa9eba1e887e80 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c3aa9eba1e887e80"} 12:58:07 [12:58:07.858] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c3aa9eba1e887e80 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c3aa9eba1e887e80"} 12:58:07 [12:58:07.858] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c3aa9eba1e887e80 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c3aa9eba1e887e80"} 12:58:07 [12:58:07.868] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:67d9eeafc6e82b04 epochNumber=2 {"provingJobId":"2:BASE_PARITY:67d9eeafc6e82b04"} 12:58:07 [12:58:07.868] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:67d9eeafc6e82b04 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:67d9eeafc6e82b04"} 12:58:07 [12:58:07.869] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:67d9eeafc6e82b04 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:67d9eeafc6e82b04"} 12:58:07 [12:58:07.929] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:39929d8212c1e678 epochNumber=3 {"provingJobId":"3:BASE_PARITY:39929d8212c1e678"} 12:58:07 [12:58:07.950] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:07 [12:58:07.989] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:3dbf2c4ab2750ccc epochNumber=4 {"provingJobId":"4:BASE_PARITY:3dbf2c4ab2750ccc"} 12:58:07 [12:58:08.000] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:08 [12:58:08.051] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:11cc34b06a6fd3bf epochNumber=5 {"provingJobId":"5:BASE_PARITY:11cc34b06a6fd3bf"} 12:58:08 [12:58:08.051] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:08 [12:58:08.104] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:54a2db44aed7cb71 epochNumber=42 {"provingJobId":"1:BASE_PARITY:54a2db44aed7cb71"} 12:58:08 [12:58:08.105] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testJ3nQxh/42 with map size 1073741824 12:58:08 [12:58:08.108] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.128] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:68b2bdd689ebf40f epochNumber=1 {"provingJobId":"1:BASE_PARITY:68b2bdd689ebf40f"} 12:58:08 [12:58:08.131] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.132] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca0684cfb178d95e epochNumber=1 {"provingJobId":"1:BASE_PARITY:ca0684cfb178d95e"} 12:58:08 [12:58:08.132] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjWtWXR/1 with map size 1073741824 12:58:08 [12:58:08.133] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.148] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4b04e9dd65766552 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4b04e9dd65766552"} 12:58:08 [12:58:08.156] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.156] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2d261d19f5c9624a epochNumber=1 {"provingJobId":"1:BASE_PARITY:2d261d19f5c9624a"} 12:58:08 [12:58:08.157] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeZbbPA/1 with map size 1073741824 12:58:08 [12:58:08.158] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.174] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2d261d19f5c9624a epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2d261d19f5c9624a"} 12:58:08 [12:58:08.176] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.176] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0e4c7cc15ac638d epochNumber=1 {"provingJobId":"1:BASE_PARITY:c0e4c7cc15ac638d"} 12:58:08 [12:58:08.177] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGXxFA7/1 with map size 1073741824 12:58:08 [12:58:08.178] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.193] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c0e4c7cc15ac638d epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c0e4c7cc15ac638d"} 12:58:08 [12:58:08.193] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c0e4c7cc15ac638d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c0e4c7cc15ac638d"} 12:58:08 [12:58:08.199] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c0e4c7cc15ac638d epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:c0e4c7cc15ac638d"} 12:58:08 [12:58:08.201] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.201] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4fd4e1286c921cbf epochNumber=1 {"provingJobId":"1:BASE_PARITY:4fd4e1286c921cbf"} 12:58:08 [12:58:08.202] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7rBRTg/1 with map size 1073741824 12:58:08 [12:58:08.203] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.220] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.222] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.223] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7d4148eec6476d7d epochNumber=1 {"provingJobId":"1:BASE_PARITY:7d4148eec6476d7d"} 12:58:08 [12:58:08.223] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaVqpYc/1 with map size 1073741824 12:58:08 [12:58:08.224] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.238] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7d4148eec6476d7d {"provingJobId":"1:BASE_PARITY:7d4148eec6476d7d"} 12:58:08 [12:58:08.239] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7d4148eec6476d7d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7d4148eec6476d7d"} 12:58:08 [12:58:08.239] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7d4148eec6476d7d type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:7d4148eec6476d7d"} 12:58:08 [12:58:08.246] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.246] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ee43c34e0eb9e14 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ee43c34e0eb9e14"} 12:58:08 [12:58:08.247] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgbzRCP/1 with map size 1073741824 12:58:08 [12:58:08.248] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.262] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2ee43c34e0eb9e14 {"provingJobId":"1:BASE_PARITY:2ee43c34e0eb9e14"} 12:58:08 [12:58:08.262] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2ee43c34e0eb9e14 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:2ee43c34e0eb9e14"} 12:58:08 [12:58:08.269] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:60feca738f538711 epochNumber=1 {"provingJobId":"1:BASE_PARITY:60feca738f538711"} 12:58:08 [12:58:08.270] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmFVhiW/1 with map size 1073741824 12:58:08 [12:58:08.271] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.285] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:60feca738f538711 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:60feca738f538711"} 12:58:08 [12:58:08.286] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:60feca738f538711 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:60feca738f538711"} 12:58:08 [12:58:08.292] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.293] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e50e054442b8f03b epochNumber=1 {"provingJobId":"1:BASE_PARITY:e50e054442b8f03b"} 12:58:08 [12:58:08.293] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0yygSE/1 with map size 1073741824 12:58:08 [12:58:08.295] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.309] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e50e054442b8f03b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e50e054442b8f03b"} 12:58:08 [12:58:08.310] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e50e054442b8f03b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:e50e054442b8f03b"} 12:58:08 [12:58:08.316] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.317] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca84427673f9bb9c epochNumber=0 {"provingJobId":"1:BASE_PARITY:ca84427673f9bb9c"} 12:58:08 [12:58:08.317] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testAOAYAl/0 with map size 1073741824 12:58:08 [12:58:08.318] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.333] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ca84427673f9bb9c epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ca84427673f9bb9c"} 12:58:08 [12:58:08.334] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b2a3b112e529f2a8 epochNumber=0 {"provingJobId":"1:BASE_PARITY:b2a3b112e529f2a8"} 12:58:08 [12:58:08.334] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ca84427673f9bb9c epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ca84427673f9bb9c"} 12:58:08 [12:58:08.334] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b70410c817207638 epochNumber=0 {"provingJobId":"1:BASE_PARITY:b70410c817207638"} 12:58:08 [12:58:08.345] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.347] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.347] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:523a90518359b439 epochNumber=2 {"provingJobId":"1:BASE_PARITY:523a90518359b439"} 12:58:08 [12:58:08.348] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testvFdHzw/2 with map size 1073741824 12:58:08 [12:58:08.349] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.364] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4cd988a47eee2d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4cd988a47eee2d5"} 12:58:08 [12:58:08.365] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvFdHzw/1 with map size 1073741824 12:58:08 [12:58:08.366] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.386] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.386] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a4fb83afeb809b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a4fb83afeb809b0"} 12:58:08 [12:58:08.388] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwGMnYU/1 with map size 1073741824 12:58:08 [12:58:08.389] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.406] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.407] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06a84a683945a36c epochNumber=1 {"provingJobId":"1:BASE_PARITY:06a84a683945a36c"} 12:58:08 [12:58:08.409] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0ybP2L/1 with map size 1073741824 12:58:08 [12:58:08.411] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.428] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b3fa76fdb0c3e467 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b3fa76fdb0c3e467"} 12:58:08 [12:58:08.433] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ffdc1f9d8d628415 epochNumber=2 {"provingJobId":"1:BASE_PARITY:ffdc1f9d8d628415"} 12:58:08 [12:58:08.434] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0ybP2L/2 with map size 1073741824 12:58:08 [12:58:08.435] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.449] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c1364855656df0be epochNumber=1 {"provingJobId":"1:BASE_PARITY:c1364855656df0be"} 12:58:08 [12:58:08.457] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.457] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c381794db34f966 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c381794db34f966"} 12:58:08 [12:58:08.458] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3V7B6r/1 with map size 1073741824 12:58:08 [12:58:08.459] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.473] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3354748360c3080 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3354748360c3080"} 12:58:08 [12:58:08.479] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15ddc1421fe9829f epochNumber=2 {"provingJobId":"1:BASE_PARITY:15ddc1421fe9829f"} 12:58:08 [12:58:08.479] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3V7B6r/2 with map size 1073741824 12:58:08 [12:58:08.481] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.494] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fc03b687e3f0614c epochNumber=1 {"provingJobId":"1:BASE_PARITY:fc03b687e3f0614c"} 12:58:08 [12:58:08.501] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.502] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0b4d8c5e465d6f14 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0b4d8c5e465d6f14"} 12:58:08 [12:58:08.502] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDYrJGL/1 with map size 1073741824 12:58:08 [12:58:08.503] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.517] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:00fec303244e425d epochNumber=1 {"provingJobId":"1:BASE_PARITY:00fec303244e425d"} 12:58:08 [12:58:08.522] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61673e124fa47752 epochNumber=2 {"provingJobId":"1:BASE_PARITY:61673e124fa47752"} 12:58:08 [12:58:08.523] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testDYrJGL/2 with map size 1073741824 12:58:08 [12:58:08.524] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.538] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9d5d411e1b35f0a3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9d5d411e1b35f0a3"} 12:58:08 [12:58:08.545] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.546] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:65ce419d3461ba4c epochNumber=1 {"provingJobId":"1:BASE_PARITY:65ce419d3461ba4c"} 12:58:08 [12:58:08.546] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testECPRfB/1 with map size 1073741824 12:58:08 [12:58:08.547] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.563] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:982dd06b6c77d9c2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:982dd06b6c77d9c2"} 12:58:08 [12:58:08.570] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:65ce419d3461ba4c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:65ce419d3461ba4c"} 12:58:08 [12:58:08.581] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.581] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:65461b88f03e9039 epochNumber=1 {"provingJobId":"1:BASE_PARITY:65461b88f03e9039"} 12:58:08 [12:58:08.583] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testh0muVW/1 with map size 1073741824 12:58:08 [12:58:08.584] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.603] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b17389f51a7c9b4e epochNumber=1 {"provingJobId":"1:BASE_PARITY:b17389f51a7c9b4e"} 12:58:08 [12:58:08.609] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:65461b88f03e9039 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:65461b88f03e9039"} 12:58:08 [12:58:08.617] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.617] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a336b0b02e1a411 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a336b0b02e1a411"} 12:58:08 [12:58:08.619] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeeAoi9/1 with map size 1073741824 12:58:08 [12:58:08.620] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.635] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a7be51565bf747d epochNumber=1 {"provingJobId":"1:BASE_PARITY:1a7be51565bf747d"} 12:58:08 [12:58:08.640] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3a336b0b02e1a411 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:3a336b0b02e1a411"} 12:58:08 [12:58:08.642] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.642] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6044d9bdfde9fe3f epochNumber=1 {"provingJobId":"1:BASE_PARITY:6044d9bdfde9fe3f"} 12:58:08 [12:58:08.643] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testh78wPk/1 with map size 1073741824 12:58:08 [12:58:08.644] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.660] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:6044d9bdfde9fe3f {"provingJobId":"1:BASE_PARITY:6044d9bdfde9fe3f"} 12:58:08 [12:58:08.660] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6044d9bdfde9fe3f type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:6044d9bdfde9fe3f"} 12:58:08 [12:58:08.665] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:81fe12d7c6749894 epochNumber=1 {"provingJobId":"1:BASE_PARITY:81fe12d7c6749894"} 12:58:08 [12:58:08.670] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6044d9bdfde9fe3f has already been completed {"provingJobId":"1:BASE_PARITY:6044d9bdfde9fe3f"} 12:58:08 [12:58:08.672] INFO: prover-client:proving-broker Proving Broker started 12:58:08 [12:58:08.673] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:223e04bf1c9c6324 epochNumber=1 {"provingJobId":"1:BASE_PARITY:223e04bf1c9c6324"} 12:58:08 [12:58:08.673] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWAGKfJ/1 with map size 1073741824 12:58:08 [12:58:08.674] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:08 [12:58:08.688] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1d1d8efd16550ba1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1d1d8efd16550ba1"} 12:58:08 [12:58:08.689] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testWAGKfJ/2 with map size 1073741824 12:58:08 [12:58:08.690] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:09 [12:58:09.754] INFO: prover-client:proving-broker Proving Broker started 12:58:09 [12:58:09.758] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:223e04bf1c9c6324 settled=false {"provingJobId":"1:BASE_PARITY:223e04bf1c9c6324","status":"pending"} 12:58:09 [12:58:09.759] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1d1d8efd16550ba1 settled=false {"provingJobId":"1:BASE_PARITY:1d1d8efd16550ba1","status":"pending"} 12:58:09 [12:58:09.760] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:223e04bf1c9c6324 type=BASE_PARITY startedAt=1750251488704 older agent has taken job {"provingJobId":"1:BASE_PARITY:223e04bf1c9c6324"} 12:58:09 [12:58:09.760] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:223e04bf1c9c6324 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:223e04bf1c9c6324"} 12:58:09 [12:58:09.764] INFO: prover-client:proving-broker Proving Broker started 12:58:09 [12:58:09.764] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:56c929b3ec0c4941 epochNumber=1 {"provingJobId":"1:BASE_PARITY:56c929b3ec0c4941"} 12:58:09 [12:58:09.766] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testO6WiVM/1 with map size 1073741824 12:58:09 [12:58:09.767] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:09 [12:58:09.787] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:611af44a3de0ee4c epochNumber=2 {"provingJobId":"1:BASE_PARITY:611af44a3de0ee4c"} 12:58:09 [12:58:09.787] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testO6WiVM/2 with map size 1073741824 12:58:09 [12:58:09.788] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:10 [12:58:10.805] INFO: prover-client:proving-broker Proving Broker started 12:58:10 [12:58:10.806] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:56c929b3ec0c4941 settled=false {"provingJobId":"1:BASE_PARITY:56c929b3ec0c4941","status":"pending"} 12:58:10 [12:58:10.807] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:611af44a3de0ee4c settled=false {"provingJobId":"1:BASE_PARITY:611af44a3de0ee4c","status":"pending"} 12:58:10 [12:58:10.808] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:56c929b3ec0c4941 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:56c929b3ec0c4941"} 12:58:10 [12:58:10.811] INFO: prover-client:proving-broker Proving Broker started 12:58:10 [12:58:10.811] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3b4c5b680f9f72d epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3b4c5b680f9f72d"} 12:58:10 [12:58:10.812] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOzyjZT/1 with map size 1073741824 12:58:10 [12:58:10.813] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:10 [12:58:10.829] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51b61a2ae8f816c0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:51b61a2ae8f816c0"} 12:58:10 [12:58:10.830] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOzyjZT/2 with map size 1073741824 12:58:10 [12:58:10.831] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.846] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.847] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a3b4c5b680f9f72d settled=false {"provingJobId":"1:BASE_PARITY:a3b4c5b680f9f72d","status":"pending"} 12:58:11 [12:58:11.849] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:51b61a2ae8f816c0 settled=false {"provingJobId":"1:BASE_PARITY:51b61a2ae8f816c0","status":"pending"} 12:58:11 [12:58:11.850] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a3b4c5b680f9f72d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a3b4c5b680f9f72d"} 12:58:11 [12:58:11.850] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a3b4c5b680f9f72d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a3b4c5b680f9f72d"} 12:58:11 [12:58:11.858] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.858] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:653e34a4c3345f6f epochNumber=1 {"provingJobId":"1:BASE_PARITY:653e34a4c3345f6f"} 12:58:11 [12:58:11.859] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwtoZNP/1 with map size 1073741824 12:58:11 [12:58:11.860] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.873] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:13a21e22130371e8 epochNumber=2 {"provingJobId":"1:BASE_PARITY:13a21e22130371e8"} 12:58:11 [12:58:11.874] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwtoZNP/2 with map size 1073741824 12:58:11 [12:58:11.876] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.889] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:653e34a4c3345f6f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:653e34a4c3345f6f"} 12:58:11 [12:58:11.894] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:13a21e22130371e8 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:13a21e22130371e8"} 12:58:11 [12:58:11.901] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.901] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b90c47cd2e1d6b16 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b90c47cd2e1d6b16"} 12:58:11 [12:58:11.902] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBROnEM/1 with map size 1073741824 12:58:11 [12:58:11.903] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.916] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9c2a93909ad394cd epochNumber=2 {"provingJobId":"1:BASE_PARITY:9c2a93909ad394cd"} 12:58:11 [12:58:11.917] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBROnEM/2 with map size 1073741824 12:58:11 [12:58:11.918] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.931] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b90c47cd2e1d6b16 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b90c47cd2e1d6b16"} 12:58:11 [12:58:11.932] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b90c47cd2e1d6b16 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b90c47cd2e1d6b16"} 12:58:11 [12:58:11.937] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9c2a93909ad394cd type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:9c2a93909ad394cd"} 12:58:11 [12:58:11.943] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.944] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b57ae39a9cb11e71 err=test error {"provingJoId":"1:BASE_PARITY:b57ae39a9cb11e71"} 12:58:11 [12:58:11.946] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.947] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:30cd2dfe5cb2d4eb not found {"provingJobId":"1:BASE_PARITY:30cd2dfe5cb2d4eb"} 12:58:11 [12:58:11.948] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.949] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:57c42cb41af99b5e epochNumber=1 {"provingJobId":"1:BASE_PARITY:57c42cb41af99b5e"} 12:58:11 [12:58:11.949] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoSZmwr/1 with map size 1073741824 12:58:11 [12:58:11.950] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:11 [12:58:11.966] INFO: prover-client:proving-broker Proving Broker started 12:58:11 [12:58:11.967] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ff1c4d8fd3082838 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ff1c4d8fd3082838"} 12:58:11 [12:58:11.967] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testk0q4Kq/1 with map size 1073741824 12:58:11 [12:58:11.968] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.094] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ff1c4d8fd3082838 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ff1c4d8fd3082838"} 12:58:12 [12:58:12.097] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.097] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca91b63405955f4c epochNumber=1 {"provingJobId":"1:BASE_PARITY:ca91b63405955f4c"} 12:58:12 [12:58:12.098] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiD0V8O/1 with map size 1073741824 12:58:12 [12:58:12.100] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.223] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ca91b63405955f4c timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ca91b63405955f4c"} 12:58:12 [12:58:12.225] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3211ca18db983b6f epochNumber=10 {"provingJobId":"1:BASE_PARITY:3211ca18db983b6f"} 12:58:12 [12:58:12.226] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testiD0V8O/10 with map size 1073741824 12:58:12 [12:58:12.227] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.250] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:12 [12:58:12.250] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:58:12 [12:58:12.368] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.369] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:286583a694298dc3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:286583a694298dc3"} 12:58:12 [12:58:12.369] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testx35lc4/1 with map size 1073741824 12:58:12 [12:58:12.370] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.545] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:286583a694298dc3 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:286583a694298dc3"} 12:58:12 [12:58:12.588] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.589] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ced70607898b9d8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ced70607898b9d8"} 12:58:12 [12:58:12.590] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBShg3s/1 with map size 1073741824 12:58:12 [12:58:12.591] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.607] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2ced70607898b9d8 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:2ced70607898b9d8"} 12:58:12 [12:58:12.609] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.610] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6c72f911b7d9bed2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6c72f911b7d9bed2"} 12:58:12 [12:58:12.611] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdQq1qs/1 with map size 1073741824 12:58:12 [12:58:12.612] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.629] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6c72f911b7d9bed2 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:6c72f911b7d9bed2"} 12:58:12 [12:58:12.629] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6c72f911b7d9bed2 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:6c72f911b7d9bed2"} 12:58:12 [12:58:12.636] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.637] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ee306b6f8695980a epochNumber=1 {"provingJobId":"1:BASE_PARITY:ee306b6f8695980a"} 12:58:12 [12:58:12.637] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfY2qQN/1 with map size 1073741824 12:58:12 [12:58:12.639] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.653] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ee306b6f8695980a type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ee306b6f8695980a"} 12:58:12 [12:58:12.660] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.660] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e46829f279d81556 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e46829f279d81556"} 12:58:12 [12:58:12.661] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJ5Dcdq/1 with map size 1073741824 12:58:12 [12:58:12.662] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.677] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e46829f279d81556 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:e46829f279d81556"} 12:58:12 [12:58:12.678] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7180d3af34dbb87a epochNumber=3 {"provingJobId":"1:BASE_PARITY:7180d3af34dbb87a"} 12:58:12 [12:58:12.679] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testJ5Dcdq/3 with map size 1073741824 12:58:12 [12:58:12.680] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.710] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:12 [12:58:12.711] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:58:12 [12:58:12.723] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e46829f279d81556 err=test error {"provingJoId":"1:BASE_PARITY:e46829f279d81556"} 12:58:12 [12:58:12.727] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrxlXVR/1 with map size 1073741824 12:58:12 [12:58:12.728] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.744] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrxlXVR/2 with map size 1073741824 12:58:12 [12:58:12.746] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.761] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.762] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9d219cb2f2a7eb1f settled=false {"provingJobId":"1:BASE_PARITY:9d219cb2f2a7eb1f","status":"pending"} 12:58:12 [12:58:12.763] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e6c95f9f7a7d2fcb settled=false {"provingJobId":"1:BASE_PARITY:e6c95f9f7a7d2fcb","status":"pending"} 12:58:12 [12:58:12.767] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwiS0v5/1 with map size 1073741824 12:58:12 [12:58:12.769] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.784] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwiS0v5/2 with map size 1073741824 12:58:12 [12:58:12.785] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.808] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.810] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:57ee5201a232b0f7 settled=true {"provingJobId":"1:BASE_PARITY:57ee5201a232b0f7","status":"fulfilled"} 12:58:12 [12:58:12.811] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:86a9572edbc0929e settled=true {"provingJobId":"2:BASE_PARITY:86a9572edbc0929e","status":"fulfilled"} 12:58:12 [12:58:12.816] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl9ij3N/1 with map size 1073741824 12:58:12 [12:58:12.817] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.837] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testl9ij3N/2 with map size 1073741824 12:58:12 [12:58:12.839] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.855] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.856] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:429d4fc8ea4ce143 settled=true {"provingJobId":"1:BASE_PARITY:429d4fc8ea4ce143","status":"fulfilled"} 12:58:12 [12:58:12.858] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c44c9d40e7432ba0 settled=false {"provingJobId":"1:BASE_PARITY:c44c9d40e7432ba0","status":"pending"} 12:58:12 [12:58:12.861] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.861] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e53d9e3fd5f1ec89 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e53d9e3fd5f1ec89"} 12:58:12 [12:58:12.862] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testW40CVs/1 with map size 1073741824 12:58:12 [12:58:12.863] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.882] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.882] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:89630ead573e701a epochNumber=1 {"provingJobId":"1:BASE_PARITY:89630ead573e701a"} 12:58:12 [12:58:12.883] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:89630ead573e701a: Error: db error: Error: db error 12:58:12 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:89630ead573e701a"} 12:58:12 [12:58:12.886] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.886] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34ebfae6ab26b66c epochNumber=1 {"provingJobId":"1:BASE_PARITY:34ebfae6ab26b66c"} 12:58:12 [12:58:12.888] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testO0clen/1 with map size 1073741824 12:58:12 [12:58:12.890] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.904] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:34ebfae6ab26b66c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:34ebfae6ab26b66c"} 12:58:12 [12:58:12.905] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:34ebfae6ab26b66c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:34ebfae6ab26b66c"} 12:58:12 [12:58:12.912] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.913] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:483d0336455f97c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:483d0336455f97c1"} 12:58:12 [12:58:12.913] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI4FPgf/1 with map size 1073741824 12:58:12 [12:58:12.915] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.929] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:483d0336455f97c1 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:483d0336455f97c1"} 12:58:12 [12:58:12.929] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:483d0336455f97c1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:483d0336455f97c1"} 12:58:12 [12:58:12.930] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:483d0336455f97c1: Error: db error 12:58:12 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:483d0336455f97c1"} 12:58:12 [12:58:12.932] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.933] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5381e6abc713c3a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5381e6abc713c3a4"} 12:58:12 [12:58:12.933] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkpLTW1/1 with map size 1073741824 12:58:12 [12:58:12.934] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.949] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5381e6abc713c3a4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5381e6abc713c3a4"} 12:58:12 [12:58:12.950] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5381e6abc713c3a4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:5381e6abc713c3a4"} 12:58:12 [12:58:12.958] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.959] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:52ad892a7cee1c51 epochNumber=1 {"provingJobId":"1:BASE_PARITY:52ad892a7cee1c51"} 12:58:12 [12:58:12.959] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testq4NTN3/1 with map size 1073741824 12:58:12 [12:58:12.961] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:12 [12:58:12.977] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:52ad892a7cee1c51 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:52ad892a7cee1c51"} 12:58:12 [12:58:12.978] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:52ad892a7cee1c51 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:52ad892a7cee1c51"} 12:58:12 [12:58:12.978] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:52ad892a7cee1c51 jobErr=test error: Error: db error 12:58:12 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:52ad892a7cee1c51"} 12:58:12 [12:58:12.981] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.982] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c40968fa2d370927 not found {"provingJobId":"1:BASE_PARITY:c40968fa2d370927"} 12:58:12 [12:58:12.984] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.984] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:46c2c585b6d3304e err=test error {"provingJoId":"1:BASE_PARITY:46c2c585b6d3304e"} 12:58:12 [12:58:12.987] INFO: prover-client:proving-broker Proving Broker started 12:58:12 [12:58:12.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:933c2c7f6961c5eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:933c2c7f6961c5eb"} 12:58:12 [12:58:12.998] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCYm4iv/1 with map size 1073741824 12:58:12 [12:58:12.999] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:13 [12:58:13.016] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:933c2c7f6961c5eb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:933c2c7f6961c5eb"} 12:58:13 [12:58:13.016] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:933c2c7f6961c5eb type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:933c2c7f6961c5eb"} 12:58:13 [12:58:13.032] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:f7c2097f73991dd5 epochNumber=2 {"provingJobId":"2:BASE_PARITY:f7c2097f73991dd5"} 12:58:13 [12:58:13.033] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCYm4iv/2 with map size 1073741824 12:58:13 [12:58:13.035] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:13 [12:58:13.050] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:f7c2097f73991dd5 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:f7c2097f73991dd5"} 12:58:13 [12:58:13.051] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:f7c2097f73991dd5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:f7c2097f73991dd5"} 12:58:13 [12:58:13.116] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:9f851c2d1603a288 epochNumber=3 {"provingJobId":"3:BASE_PARITY:9f851c2d1603a288"} 12:58:13 [12:58:13.117] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testCYm4iv/3 with map size 1073741824 12:58:13 [12:58:13.118] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:13 [12:58:13.137] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:13 [12:58:13.137] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:58:13 [12:58:13.194] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:fc0617455c235916 epochNumber=4 {"provingJobId":"4:BASE_PARITY:fc0617455c235916"} 12:58:13 [12:58:13.195] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testCYm4iv/4 with map size 1073741824 12:58:13 [12:58:13.196] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:13 [12:58:13.215] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:13 [12:58:13.215] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 12:58:13 [12:58:13.273] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:bd66dd7b0974cf67 epochNumber=5 {"provingJobId":"5:BASE_PARITY:bd66dd7b0974cf67"} 12:58:13 [12:58:13.274] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testCYm4iv/5 with map size 1073741824 12:58:13 [12:58:13.275] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:58:13 [12:58:13.292] INFO: prover-client:proving-broker Cleaned up jobs=1 12:58:13 [12:58:13.292] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 12:58:13 PASS src/proving_broker/proving_broker.test.ts (10.852 s) 12:58:13 ProvingBroker 12:58:13 Producer API 12:58:13 ✓ refuses stale jobs (33 ms) 12:58:13 ✓ enqueues jobs (4 ms) 12:58:13 ✓ ignores duplicate jobs (4 ms) 12:58:13 ✓ reports correct status when enqueuing repeat jobs (4 ms) 12:58:13 ✓ throws an error in case of duplicate job IDs (18 ms) 12:58:13 ✓ returns not-found status for non-existing jobs (1 ms) 12:58:13 ✓ cancels jobs in queue (6 ms) 12:58:13 ✓ cancels jobs in-progress (2 ms) 12:58:13 ✓ returns job result if successful (3 ms) 12:58:13 ✓ returns job error if failed (2 ms) 12:58:13 ✓ correctly returns job status for concurrent writes (2 ms) 12:58:13 ✓ refuses stale jobs (28 ms) 12:58:13 ✓ enqueues jobs (25 ms) 12:58:13 ✓ ignores duplicate jobs (20 ms) 12:58:13 ✓ reports correct status when enqueuing repeat jobs (24 ms) 12:58:13 ✓ throws an error in case of duplicate job IDs (19 ms) 12:58:13 ✓ returns not-found status for non-existing jobs (2 ms) 12:58:13 ✓ cancels jobs in queue (24 ms) 12:58:13 ✓ cancels jobs in-progress (23 ms) 12:58:13 ✓ returns job result if successful (23 ms) 12:58:13 ✓ returns job error if failed (23 ms) 12:58:13 ✓ correctly returns job status for concurrent writes (29 ms) 12:58:13 Consumer API 12:58:13 ✓ returns undefined if no jobs are available (1 ms) 12:58:13 ✓ returns jobs in priority order (2 ms) 12:58:13 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 12:58:13 ✓ returns a job if it is in the allowList (2 ms) 12:58:13 ✓ returns the most important job if it is in the allowList (2 ms) 12:58:13 ✓ returns any job if filter is empty (2 ms) 12:58:13 ✓ returns a new job when reporting job success (2 ms) 12:58:13 ✓ returns a new job when reporting permanent error (1 ms) 12:58:13 ✓ returns a new job when reporting retry-able error (2 ms) 12:58:13 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 12:58:13 ✓ returns a new job if job is already in progress elsewhere (1054 ms) 12:58:13 ✓ avoids sending the same job to a new agent after a restart (1002 ms) 12:58:13 ✓ avoids sending a completed job to a new agent after a restart (1003 ms) 12:58:13 ✓ tracks job result if in progress (3 ms) 12:58:13 ✓ tracks job result even if job is in queue (2 ms) 12:58:13 ✓ ignores reported job error if unknown job (1 ms) 12:58:13 ✓ ignores job result if unknown job (1 ms) 12:58:13 ✓ returns undefined if no jobs are available (1 ms) 12:58:13 ✓ returns jobs in priority order (39 ms) 12:58:13 ✓ returns undefined if no jobs are available for the given allowList (20 ms) 12:58:13 ✓ returns a job if it is in the allowList (50 ms) 12:58:13 ✓ returns the most important job if it is in the allowList (44 ms) 12:58:13 ✓ returns any job if filter is empty (44 ms) 12:58:13 ✓ returns a new job when reporting job success (34 ms) 12:58:13 ✓ returns a new job when reporting permanent error (36 ms) 12:58:13 ✓ returns a new job when reporting retry-able error (25 ms) 12:58:13 ✓ returns a new job when reporting progress if current one is cancelled (30 ms) 12:58:13 ✓ returns a new job if job is already in progress elsewhere (1091 ms) 12:58:13 ✓ avoids sending the same job to a new agent after a restart (1046 ms) 12:58:13 ✓ avoids sending a completed job to a new agent after a restart (1047 ms) 12:58:13 ✓ tracks job result if in progress (43 ms) 12:58:13 ✓ tracks job result even if job is in queue (42 ms) 12:58:13 ✓ ignores reported job error if unknown job (3 ms) 12:58:13 ✓ ignores job result if unknown job (2 ms) 12:58:13 Timeouts 12:58:13 ✓ tracks in progress jobs (1 ms) 12:58:13 ✓ re-enqueues jobs that time out (107 ms) 12:58:13 ✓ cancel stale jobs that time out (233 ms) 12:58:13 ✓ keeps the jobs in progress while it is alive (203 ms) 12:58:13 ✓ tracks in progress jobs (17 ms) 12:58:13 ✓ re-enqueues jobs that time out (130 ms) 12:58:13 ✓ cancel stale jobs that time out (271 ms) 12:58:13 ✓ keeps the jobs in progress while it is alive (220 ms) 12:58:13 Retries 12:58:13 ✓ retries jobs (2 ms) 12:58:13 ✓ retries up to a maximum number of times (2 ms) 12:58:13 ✓ passing retry=false does not retry (2 ms) 12:58:13 ✓ does not retry if job is stale (27 ms) 12:58:13 ✓ retries jobs (21 ms) 12:58:13 ✓ retries up to a maximum number of times (27 ms) 12:58:13 ✓ passing retry=false does not retry (23 ms) 12:58:13 ✓ does not retry if job is stale (65 ms) 12:58:13 Database management 12:58:13 ✓ re-enqueues proof requests on start (2 ms) 12:58:13 ✓ restores proof results on start (1 ms) 12:58:13 ✓ only re-enqueues unfinished jobs (2 ms) 12:58:13 ✓ saves job when enqueued (3 ms) 12:58:13 ✓ does not retain job if database fails to save (2 ms) 12:58:13 ✓ saves job result (2 ms) 12:58:13 ✓ saves result even if database fails to save (2 ms) 12:58:13 ✓ saves job error (2 ms) 12:58:13 ✓ saves job error even if database fails to save (2 ms) 12:58:13 ✓ does not save job result if job is unknown (1 ms) 12:58:13 ✓ does not save job error if job is unknown (1 ms) 12:58:13 ✓ cleans up old jobs periodically (256 ms) 12:58:13 ✓ re-enqueues proof requests on start (42 ms) 12:58:13 ✓ restores proof results on start (48 ms) 12:58:13 ✓ only re-enqueues unfinished jobs (45 ms) 12:58:13 ✓ saves job when enqueued (20 ms) 12:58:13 ✓ does not retain job if database fails to save (4 ms) 12:58:13 ✓ saves job result (27 ms) 12:58:13 ✓ saves result even if database fails to save (20 ms) 12:58:13 ✓ saves job error (25 ms) 12:58:13 ✓ saves job error even if database fails to save (24 ms) 12:58:13 ✓ does not save job result if job is unknown (3 ms) 12:58:13 ✓ does not save job error if job is unknown (2 ms) 12:58:13 ✓ cleans up old jobs periodically (357 ms) 12:58:13 12:58:13 Test Suites: 1 passed, 1 total 12:58:13 Tests: 96 passed, 96 total 12:58:13 Snapshots: 0 total 12:58:13 Time: 10.912 s 12:58:13 Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i. 12:58:13 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?