Parent Log: http://ci.aztec-labs.com/810a77e866dce8f5 Command: 411627cf9c043d00 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/721a41c449eb0c4538a26180f6061f326128cf78 Env: REF_NAME=gh-readonly-queue/next/pr-14963-60e60f99f2634ad75a76b6e103858ff9ccd041d5 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Mon Jun 16 18:41:36 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-14963_amd64_x3-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 18:41:42 [18:41:42.086] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.094] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0703c3bed4119a70 epochNumber=42 {"provingJobId":"1:BASE_PARITY:0703c3bed4119a70"} 18:41:42 [18:41:42.097] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:2bdf626c879f7600 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2bdf626c879f7600"} 18:41:42 [18:41:42.127] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.129] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:82b42865268694b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:82b42865268694b2"} 18:41:42 [18:41:42.130] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:635aef9f51402966 epochNumber=1 {"provingJobId":"1:BASE_PARITY:635aef9f51402966"} 18:41:42 [18:41:42.132] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.133] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dd7139a95ce952b3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dd7139a95ce952b3"} 18:41:42 [18:41:42.135] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dd7139a95ce952b3 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:dd7139a95ce952b3"} 18:41:42 [18:41:42.138] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.140] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:32897272f6a2a488 epochNumber=1 {"provingJobId":"1:BASE_PARITY:32897272f6a2a488"} 18:41:42 [18:41:42.141] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:32897272f6a2a488 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:32897272f6a2a488"} 18:41:42 [18:41:42.142] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:32897272f6a2a488 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:32897272f6a2a488"} 18:41:42 [18:41:42.142] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:32897272f6a2a488 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:32897272f6a2a488"} 18:41:42 [18:41:42.144] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.144] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5180d5d0bf4effca epochNumber=1 {"provingJobId":"1:BASE_PARITY:5180d5d0bf4effca"} 18:41:42 [18:41:42.184] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.186] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:176d84c2b0aad5cd epochNumber=1 {"provingJobId":"1:BASE_PARITY:176d84c2b0aad5cd"} 18:41:42 [18:41:42.187] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:176d84c2b0aad5cd {"provingJobId":"1:BASE_PARITY:176d84c2b0aad5cd"} 18:41:42 [18:41:42.187] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:176d84c2b0aad5cd type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:176d84c2b0aad5cd"} 18:41:42 [18:41:42.188] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:176d84c2b0aad5cd type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:176d84c2b0aad5cd"} 18:41:42 [18:41:42.189] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.189] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3f1afebc29adef0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3f1afebc29adef0"} 18:41:42 [18:41:42.190] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:a3f1afebc29adef0 {"provingJobId":"1:BASE_PARITY:a3f1afebc29adef0"} 18:41:42 [18:41:42.190] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3f1afebc29adef0 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:a3f1afebc29adef0"} 18:41:42 [18:41:42.197] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5659493d964041ca epochNumber=1 {"provingJobId":"1:BASE_PARITY:5659493d964041ca"} 18:41:42 [18:41:42.199] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5659493d964041ca type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5659493d964041ca"} 18:41:42 [18:41:42.200] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5659493d964041ca type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5659493d964041ca"} 18:41:42 [18:41:42.202] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3776a96a28621e72 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3776a96a28621e72"} 18:41:42 [18:41:42.203] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3776a96a28621e72 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3776a96a28621e72"} 18:41:42 [18:41:42.203] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3776a96a28621e72 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3776a96a28621e72"} 18:41:42 [18:41:42.205] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f4a24da58612458c epochNumber=0 {"provingJobId":"1:BASE_PARITY:f4a24da58612458c"} 18:41:42 [18:41:42.205] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f4a24da58612458c epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:f4a24da58612458c"} 18:41:42 [18:41:42.206] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a6f85e505ec6283f epochNumber=0 {"provingJobId":"1:BASE_PARITY:a6f85e505ec6283f"} 18:41:42 [18:41:42.206] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f4a24da58612458c epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:f4a24da58612458c"} 18:41:42 [18:41:42.206] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a312a0f72223389 epochNumber=0 {"provingJobId":"1:BASE_PARITY:2a312a0f72223389"} 18:41:42 [18:41:42.208] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.209] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1bbd891108f9f5df epochNumber=2 {"provingJobId":"1:BASE_PARITY:1bbd891108f9f5df"} 18:41:42 [18:41:42.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:22e6b15434d7b123 epochNumber=1 {"provingJobId":"1:BASE_PARITY:22e6b15434d7b123"} 18:41:42 [18:41:42.211] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.212] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a747943b8c494ed epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a747943b8c494ed"} 18:41:42 [18:41:42.213] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.213] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4b03b3b89d43a5f epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4b03b3b89d43a5f"} 18:41:42 [18:41:42.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6290a3a151c90760 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6290a3a151c90760"} 18:41:42 [18:41:42.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4132a9f8cb772de5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4132a9f8cb772de5"} 18:41:42 [18:41:42.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0df32fae15b5f86 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b0df32fae15b5f86"} 18:41:42 [18:41:42.216] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.216] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59597ca4f0bc992d epochNumber=1 {"provingJobId":"1:BASE_PARITY:59597ca4f0bc992d"} 18:41:42 [18:41:42.216] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8d8f2e21ddef2e06 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8d8f2e21ddef2e06"} 18:41:42 [18:41:42.217] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:10a6aa76dfc2ea01 epochNumber=2 {"provingJobId":"1:BASE_PARITY:10a6aa76dfc2ea01"} 18:41:42 [18:41:42.217] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:778368aa9b3f1732 epochNumber=1 {"provingJobId":"1:BASE_PARITY:778368aa9b3f1732"} 18:41:42 [18:41:42.218] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.218] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1fc36ff6c6c0fba2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1fc36ff6c6c0fba2"} 18:41:42 [18:41:42.219] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d489d460dad58ed0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d489d460dad58ed0"} 18:41:42 [18:41:42.219] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4604dc26289787d2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4604dc26289787d2"} 18:41:42 [18:41:42.219] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a95804596219ce55 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a95804596219ce55"} 18:41:42 [18:41:42.221] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.221] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5d7c93516c9f585f epochNumber=1 {"provingJobId":"1:BASE_PARITY:5d7c93516c9f585f"} 18:41:42 [18:41:42.222] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a2c8e790c3ec330d epochNumber=1 {"provingJobId":"1:BASE_PARITY:a2c8e790c3ec330d"} 18:41:42 [18:41:42.222] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5d7c93516c9f585f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5d7c93516c9f585f"} 18:41:42 [18:41:42.223] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.224] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3dc5ceec4411ac05 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3dc5ceec4411ac05"} 18:41:42 [18:41:42.224] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64db5c6bec4cb5f8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:64db5c6bec4cb5f8"} 18:41:42 [18:41:42.224] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3dc5ceec4411ac05 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:3dc5ceec4411ac05"} 18:41:42 [18:41:42.226] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.226] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d47db60a8333a6c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d47db60a8333a6c8"} 18:41:42 [18:41:42.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0234c842e6cc6a84 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0234c842e6cc6a84"} 18:41:42 [18:41:42.227] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d47db60a8333a6c8 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:d47db60a8333a6c8"} 18:41:42 [18:41:42.229] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc3c80c08298c617 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc3c80c08298c617"} 18:41:42 [18:41:42.230] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cc3c80c08298c617 {"provingJobId":"1:BASE_PARITY:cc3c80c08298c617"} 18:41:42 [18:41:42.230] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cc3c80c08298c617 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:cc3c80c08298c617"} 18:41:42 [18:41:42.230] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:114a41372bb7369b epochNumber=1 {"provingJobId":"1:BASE_PARITY:114a41372bb7369b"} 18:41:42 [18:41:42.231] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cc3c80c08298c617 has already been completed {"provingJobId":"1:BASE_PARITY:cc3c80c08298c617"} 18:41:42 [18:41:42.232] INFO: prover-client:proving-broker Proving Broker started 18:41:42 [18:41:42.232] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe1b267252695ae8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fe1b267252695ae8"} 18:41:42 [18:41:42.233] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e257c9387af4f14e epochNumber=2 {"provingJobId":"1:BASE_PARITY:e257c9387af4f14e"} 18:41:43 [18:41:43.284] INFO: prover-client:proving-broker Proving Broker started 18:41:43 [18:41:43.285] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fe1b267252695ae8 settled=false {"provingJobId":"1:BASE_PARITY:fe1b267252695ae8","status":"pending"} 18:41:43 [18:41:43.285] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e257c9387af4f14e settled=false {"provingJobId":"1:BASE_PARITY:e257c9387af4f14e","status":"pending"} 18:41:43 [18:41:43.286] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:fe1b267252695ae8 type=BASE_PARITY startedAt=1750099302233 older agent has taken job {"provingJobId":"1:BASE_PARITY:fe1b267252695ae8"} 18:41:43 [18:41:43.287] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:fe1b267252695ae8 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:fe1b267252695ae8"} 18:41:43 [18:41:43.288] INFO: prover-client:proving-broker Proving Broker started 18:41:43 [18:41:43.289] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:93a84560f76728b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:93a84560f76728b2"} 18:41:43 [18:41:43.289] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f0c0a10cbbc218ef epochNumber=2 {"provingJobId":"1:BASE_PARITY:f0c0a10cbbc218ef"} 18:41:44 [18:41:44.290] INFO: prover-client:proving-broker Proving Broker started 18:41:44 [18:41:44.291] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:93a84560f76728b2 settled=false {"provingJobId":"1:BASE_PARITY:93a84560f76728b2","status":"pending"} 18:41:44 [18:41:44.291] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f0c0a10cbbc218ef settled=false {"provingJobId":"1:BASE_PARITY:f0c0a10cbbc218ef","status":"pending"} 18:41:44 [18:41:44.292] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:93a84560f76728b2 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:93a84560f76728b2"} 18:41:44 [18:41:44.293] INFO: prover-client:proving-broker Proving Broker started 18:41:44 [18:41:44.294] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:837a826b37611c3b epochNumber=1 {"provingJobId":"1:BASE_PARITY:837a826b37611c3b"} 18:41:44 [18:41:44.294] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:af56149010652593 epochNumber=2 {"provingJobId":"1:BASE_PARITY:af56149010652593"} 18:41:45 [18:41:45.295] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.295] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:837a826b37611c3b settled=false {"provingJobId":"1:BASE_PARITY:837a826b37611c3b","status":"pending"} 18:41:45 [18:41:45.296] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:af56149010652593 settled=false {"provingJobId":"1:BASE_PARITY:af56149010652593","status":"pending"} 18:41:45 [18:41:45.297] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:837a826b37611c3b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:837a826b37611c3b"} 18:41:45 [18:41:45.297] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:837a826b37611c3b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:837a826b37611c3b"} 18:41:45 [18:41:45.299] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.299] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:57afa236407b80f3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:57afa236407b80f3"} 18:41:45 [18:41:45.300] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2fc9ccc797aa6c64 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2fc9ccc797aa6c64"} 18:41:45 [18:41:45.300] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:57afa236407b80f3 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:57afa236407b80f3"} 18:41:45 [18:41:45.301] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2fc9ccc797aa6c64 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2fc9ccc797aa6c64"} 18:41:45 [18:41:45.302] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.303] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c7a083bd429c572a epochNumber=1 {"provingJobId":"1:BASE_PARITY:c7a083bd429c572a"} 18:41:45 [18:41:45.303] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6568289baf6534cd epochNumber=2 {"provingJobId":"1:BASE_PARITY:6568289baf6534cd"} 18:41:45 [18:41:45.304] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c7a083bd429c572a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c7a083bd429c572a"} 18:41:45 [18:41:45.304] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c7a083bd429c572a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c7a083bd429c572a"} 18:41:45 [18:41:45.305] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6568289baf6534cd type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:6568289baf6534cd"} 18:41:45 [18:41:45.307] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.307] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b0f1718a83bc1418 err=test error {"provingJoId":"1:BASE_PARITY:b0f1718a83bc1418"} 18:41:45 [18:41:45.308] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.309] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:77fb29214379682a not found {"provingJobId":"1:BASE_PARITY:77fb29214379682a"} 18:41:45 [18:41:45.310] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.311] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1bf905de0afbfeb4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1bf905de0afbfeb4"} 18:41:45 [18:41:45.312] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.312] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ce77de5a64c7a112 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ce77de5a64c7a112"} 18:41:45 [18:41:45.413] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ce77de5a64c7a112 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ce77de5a64c7a112"} 18:41:45 [18:41:45.420] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.422] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2fcbea8d89e12d34 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2fcbea8d89e12d34"} 18:41:45 [18:41:45.547] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2fcbea8d89e12d34 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:2fcbea8d89e12d34"} 18:41:45 [18:41:45.550] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2e8a4a628473564 epochNumber=10 {"provingJobId":"1:BASE_PARITY:e2e8a4a628473564"} 18:41:45 [18:41:45.576] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:45 [18:41:45.678] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.679] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:341320184865a5e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:341320184865a5e5"} 18:41:45 [18:41:45.831] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:341320184865a5e5 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:341320184865a5e5"} 18:41:45 [18:41:45.884] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.885] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:30448ab8f5870830 epochNumber=1 {"provingJobId":"1:BASE_PARITY:30448ab8f5870830"} 18:41:45 [18:41:45.885] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:30448ab8f5870830 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:30448ab8f5870830"} 18:41:45 [18:41:45.887] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.887] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:66eba31586066292 epochNumber=1 {"provingJobId":"1:BASE_PARITY:66eba31586066292"} 18:41:45 [18:41:45.888] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:66eba31586066292 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:66eba31586066292"} 18:41:45 [18:41:45.888] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:66eba31586066292 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:66eba31586066292"} 18:41:45 [18:41:45.890] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.890] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8cc958c784f84f76 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8cc958c784f84f76"} 18:41:45 [18:41:45.890] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8cc958c784f84f76 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8cc958c784f84f76"} 18:41:45 [18:41:45.891] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.892] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:942877d8cd00163d epochNumber=1 {"provingJobId":"1:BASE_PARITY:942877d8cd00163d"} 18:41:45 [18:41:45.892] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:942877d8cd00163d type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:942877d8cd00163d"} 18:41:45 [18:41:45.893] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab369d64c1e1d6d5 epochNumber=3 {"provingJobId":"1:BASE_PARITY:ab369d64c1e1d6d5"} 18:41:45 [18:41:45.916] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:45 [18:41:45.918] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:942877d8cd00163d err=test error {"provingJoId":"1:BASE_PARITY:942877d8cd00163d"} 18:41:45 [18:41:45.920] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.921] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9c1fc640d44a3471 settled=false {"provingJobId":"1:BASE_PARITY:9c1fc640d44a3471","status":"pending"} 18:41:45 [18:41:45.921] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e066d8074e4f6488 settled=false {"provingJobId":"1:BASE_PARITY:e066d8074e4f6488","status":"pending"} 18:41:45 [18:41:45.924] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.924] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:88b262c93a83b1fb settled=true {"provingJobId":"1:BASE_PARITY:88b262c93a83b1fb","status":"fulfilled"} 18:41:45 [18:41:45.925] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:e2f3bc29b2a715b9 settled=true {"provingJobId":"2:BASE_PARITY:e2f3bc29b2a715b9","status":"fulfilled"} 18:41:45 [18:41:45.926] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.926] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:baf70bb3b5ec8780 settled=true {"provingJobId":"1:BASE_PARITY:baf70bb3b5ec8780","status":"fulfilled"} 18:41:45 [18:41:45.926] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:242f5b6b23a16248 settled=false {"provingJobId":"1:BASE_PARITY:242f5b6b23a16248","status":"pending"} 18:41:45 [18:41:45.928] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.929] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a503e62126be08b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a503e62126be08b5"} 18:41:45 [18:41:45.932] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.932] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a8fcd75b4c3832b epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a8fcd75b4c3832b"} 18:41:45 [18:41:45.933] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:3a8fcd75b4c3832b: Error: db error: Error: db error 18:41:45 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:3a8fcd75b4c3832b"} 18:41:45 [18:41:45.935] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.935] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:19c4441eb7a62593 epochNumber=1 {"provingJobId":"1:BASE_PARITY:19c4441eb7a62593"} 18:41:45 [18:41:45.935] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:19c4441eb7a62593 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:19c4441eb7a62593"} 18:41:45 [18:41:45.936] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:19c4441eb7a62593 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:19c4441eb7a62593"} 18:41:45 [18:41:45.937] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.937] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d117b0996018b9d6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d117b0996018b9d6"} 18:41:45 [18:41:45.938] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d117b0996018b9d6 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d117b0996018b9d6"} 18:41:45 [18:41:45.938] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d117b0996018b9d6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d117b0996018b9d6"} 18:41:45 [18:41:45.939] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:d117b0996018b9d6: Error: db error 18:41:45 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:d117b0996018b9d6"} 18:41:45 [18:41:45.941] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.942] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3db06d1d0c2d22d9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3db06d1d0c2d22d9"} 18:41:45 [18:41:45.942] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3db06d1d0c2d22d9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3db06d1d0c2d22d9"} 18:41:45 [18:41:45.943] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3db06d1d0c2d22d9 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3db06d1d0c2d22d9"} 18:41:45 [18:41:45.944] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.945] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a543e19f548d91c epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a543e19f548d91c"} 18:41:45 [18:41:45.945] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2a543e19f548d91c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2a543e19f548d91c"} 18:41:45 [18:41:45.945] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2a543e19f548d91c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2a543e19f548d91c"} 18:41:45 [18:41:45.946] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:2a543e19f548d91c jobErr=test error: Error: db error 18:41:45 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:2a543e19f548d91c"} 18:41:45 [18:41:45.947] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.948] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e8df6e9d9b5679f6 not found {"provingJobId":"1:BASE_PARITY:e8df6e9d9b5679f6"} 18:41:45 [18:41:45.950] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.950] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e9731e896f31b082 err=test error {"provingJoId":"1:BASE_PARITY:e9731e896f31b082"} 18:41:45 [18:41:45.952] INFO: prover-client:proving-broker Proving Broker started 18:41:45 [18:41:45.962] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08a546d402a07691 epochNumber=1 {"provingJobId":"1:BASE_PARITY:08a546d402a07691"} 18:41:45 [18:41:45.963] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:08a546d402a07691 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:08a546d402a07691"} 18:41:45 [18:41:45.964] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:08a546d402a07691 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:08a546d402a07691"} 18:41:45 [18:41:45.976] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:17af5907d3c1613b epochNumber=2 {"provingJobId":"2:BASE_PARITY:17af5907d3c1613b"} 18:41:45 [18:41:45.977] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:17af5907d3c1613b type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:17af5907d3c1613b"} 18:41:45 [18:41:45.977] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:17af5907d3c1613b type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:17af5907d3c1613b"} 18:41:46 [18:41:46.038] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:553e134a3aa6c19f epochNumber=3 {"provingJobId":"3:BASE_PARITY:553e134a3aa6c19f"} 18:41:46 [18:41:46.053] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:46 [18:41:46.100] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:34f5be062f081688 epochNumber=4 {"provingJobId":"4:BASE_PARITY:34f5be062f081688"} 18:41:46 [18:41:46.106] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:46 [18:41:46.163] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:fd0611bdb985f347 epochNumber=5 {"provingJobId":"5:BASE_PARITY:fd0611bdb985f347"} 18:41:46 [18:41:46.184] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:46 [18:41:46.219] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.219] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2bc3228067f9bb6a epochNumber=42 {"provingJobId":"1:BASE_PARITY:2bc3228067f9bb6a"} 18:41:46 [18:41:46.224] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-test9rpKbh/42 with map size 1073741824 18:41:46 [18:41:46.229] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.263] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:53fdf29169bbb082 epochNumber=1 {"provingJobId":"1:BASE_PARITY:53fdf29169bbb082"} 18:41:46 [18:41:46.268] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8e72409b86078b7d epochNumber=1 {"provingJobId":"1:BASE_PARITY:8e72409b86078b7d"} 18:41:46 [18:41:46.271] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrX22JP/1 with map size 1073741824 18:41:46 [18:41:46.274] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.294] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:87fc3de23d94a082 epochNumber=1 {"provingJobId":"1:BASE_PARITY:87fc3de23d94a082"} 18:41:46 [18:41:46.304] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.305] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5d245ada7194ff41 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5d245ada7194ff41"} 18:41:46 [18:41:46.309] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVqXGH5/1 with map size 1073741824 18:41:46 [18:41:46.312] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.332] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:5d245ada7194ff41 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:5d245ada7194ff41"} 18:41:46 [18:41:46.337] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.338] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9dff757a785ed96e epochNumber=1 {"provingJobId":"1:BASE_PARITY:9dff757a785ed96e"} 18:41:46 [18:41:46.339] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjxZXPc/1 with map size 1073741824 18:41:46 [18:41:46.342] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.370] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9dff757a785ed96e epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:9dff757a785ed96e"} 18:41:46 [18:41:46.371] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9dff757a785ed96e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9dff757a785ed96e"} 18:41:46 [18:41:46.377] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9dff757a785ed96e epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:9dff757a785ed96e"} 18:41:46 [18:41:46.382] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.383] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:66441707607f7dac epochNumber=1 {"provingJobId":"1:BASE_PARITY:66441707607f7dac"} 18:41:46 [18:41:46.384] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdTZoeO/1 with map size 1073741824 18:41:46 [18:41:46.386] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.413] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.415] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.416] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:485aff2dd29e67cb epochNumber=1 {"provingJobId":"1:BASE_PARITY:485aff2dd29e67cb"} 18:41:46 [18:41:46.417] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfadHf3/1 with map size 1073741824 18:41:46 [18:41:46.419] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.438] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:485aff2dd29e67cb {"provingJobId":"1:BASE_PARITY:485aff2dd29e67cb"} 18:41:46 [18:41:46.439] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:485aff2dd29e67cb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:485aff2dd29e67cb"} 18:41:46 [18:41:46.439] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:485aff2dd29e67cb type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:485aff2dd29e67cb"} 18:41:46 [18:41:46.453] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.454] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3cd3509a88349aa7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3cd3509a88349aa7"} 18:41:46 [18:41:46.455] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJWFYmO/1 with map size 1073741824 18:41:46 [18:41:46.457] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.479] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3cd3509a88349aa7 {"provingJobId":"1:BASE_PARITY:3cd3509a88349aa7"} 18:41:46 [18:41:46.480] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3cd3509a88349aa7 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:3cd3509a88349aa7"} 18:41:46 [18:41:46.489] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.490] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ee6b6a136c89796e epochNumber=1 {"provingJobId":"1:BASE_PARITY:ee6b6a136c89796e"} 18:41:46 [18:41:46.491] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLGheib/1 with map size 1073741824 18:41:46 [18:41:46.494] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.511] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ee6b6a136c89796e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ee6b6a136c89796e"} 18:41:46 [18:41:46.511] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ee6b6a136c89796e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ee6b6a136c89796e"} 18:41:46 [18:41:46.522] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.523] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c1fe3b4531c889c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c1fe3b4531c889c4"} 18:41:46 [18:41:46.524] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHBejNO/1 with map size 1073741824 18:41:46 [18:41:46.528] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.545] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c1fe3b4531c889c4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c1fe3b4531c889c4"} 18:41:46 [18:41:46.546] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c1fe3b4531c889c4 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c1fe3b4531c889c4"} 18:41:46 [18:41:46.562] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.563] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b84b7e569a356325 epochNumber=0 {"provingJobId":"1:BASE_PARITY:b84b7e569a356325"} 18:41:46 [18:41:46.563] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testtesXlL/0 with map size 1073741824 18:41:46 [18:41:46.567] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.585] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b84b7e569a356325 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b84b7e569a356325"} 18:41:46 [18:41:46.586] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d2a263f39a20f3b2 epochNumber=0 {"provingJobId":"1:BASE_PARITY:d2a263f39a20f3b2"} 18:41:46 [18:41:46.587] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b84b7e569a356325 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b84b7e569a356325"} 18:41:46 [18:41:46.587] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9486cd48d60fdd75 epochNumber=0 {"provingJobId":"1:BASE_PARITY:9486cd48d60fdd75"} 18:41:46 [18:41:46.602] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.605] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.606] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bfa7da6a312aa246 epochNumber=2 {"provingJobId":"1:BASE_PARITY:bfa7da6a312aa246"} 18:41:46 [18:41:46.607] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testHbsbI5/2 with map size 1073741824 18:41:46 [18:41:46.609] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.626] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:026f6edb145acf25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:026f6edb145acf25"} 18:41:46 [18:41:46.627] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHbsbI5/1 with map size 1073741824 18:41:46 [18:41:46.630] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.657] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.658] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7508d3cebe7df4ca epochNumber=1 {"provingJobId":"1:BASE_PARITY:7508d3cebe7df4ca"} 18:41:46 [18:41:46.661] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testu9zwBh/1 with map size 1073741824 18:41:46 [18:41:46.666] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.689] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.690] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c376a80f0683b201 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c376a80f0683b201"} 18:41:46 [18:41:46.691] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeUSkvy/1 with map size 1073741824 18:41:46 [18:41:46.692] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.709] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:956f52f84673e73f epochNumber=1 {"provingJobId":"1:BASE_PARITY:956f52f84673e73f"} 18:41:46 [18:41:46.715] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec36cb1905df8272 epochNumber=2 {"provingJobId":"1:BASE_PARITY:ec36cb1905df8272"} 18:41:46 [18:41:46.716] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testeUSkvy/2 with map size 1073741824 18:41:46 [18:41:46.718] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.735] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a59043a5e8e4eb5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a59043a5e8e4eb5"} 18:41:46 [18:41:46.744] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.744] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:669ca965f5d3cf1d epochNumber=1 {"provingJobId":"1:BASE_PARITY:669ca965f5d3cf1d"} 18:41:46 [18:41:46.745] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLFmH3e/1 with map size 1073741824 18:41:46 [18:41:46.746] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.764] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:70592f7c24e7a48c epochNumber=1 {"provingJobId":"1:BASE_PARITY:70592f7c24e7a48c"} 18:41:46 [18:41:46.771] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:676a0adf2a9a7782 epochNumber=2 {"provingJobId":"1:BASE_PARITY:676a0adf2a9a7782"} 18:41:46 [18:41:46.772] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLFmH3e/2 with map size 1073741824 18:41:46 [18:41:46.774] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.790] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:004da594adff2a05 epochNumber=1 {"provingJobId":"1:BASE_PARITY:004da594adff2a05"} 18:41:46 [18:41:46.802] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.803] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3bf709ce25639d20 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3bf709ce25639d20"} 18:41:46 [18:41:46.803] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOj4ODV/1 with map size 1073741824 18:41:46 [18:41:46.805] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.822] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67c40b5b4ae10595 epochNumber=1 {"provingJobId":"1:BASE_PARITY:67c40b5b4ae10595"} 18:41:46 [18:41:46.828] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:00380e63a19e0c8e epochNumber=2 {"provingJobId":"1:BASE_PARITY:00380e63a19e0c8e"} 18:41:46 [18:41:46.829] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOj4ODV/2 with map size 1073741824 18:41:46 [18:41:46.832] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.852] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4d1a263ad03619fa epochNumber=1 {"provingJobId":"1:BASE_PARITY:4d1a263ad03619fa"} 18:41:46 [18:41:46.877] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.878] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8943b1c9b9a98df epochNumber=1 {"provingJobId":"1:BASE_PARITY:a8943b1c9b9a98df"} 18:41:46 [18:41:46.879] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRHR8x4/1 with map size 1073741824 18:41:46 [18:41:46.881] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.898] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:52fa6f60b4af7b57 epochNumber=1 {"provingJobId":"1:BASE_PARITY:52fa6f60b4af7b57"} 18:41:46 [18:41:46.904] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a8943b1c9b9a98df type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a8943b1c9b9a98df"} 18:41:46 [18:41:46.915] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.916] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3d35285909f674b epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3d35285909f674b"} 18:41:46 [18:41:46.917] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1yA5xl/1 with map size 1073741824 18:41:46 [18:41:46.921] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e8a110f134df8ff7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e8a110f134df8ff7"} 18:41:46 [18:41:46.945] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3d35285909f674b type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:a3d35285909f674b"} 18:41:46 [18:41:46.962] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.963] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61470b0914f877c7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:61470b0914f877c7"} 18:41:46 [18:41:46.964] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTn7JPl/1 with map size 1073741824 18:41:46 [18:41:46.965] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:46 [18:41:46.986] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa6ea436be428387 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aa6ea436be428387"} 18:41:46 [18:41:46.993] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:61470b0914f877c7 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:61470b0914f877c7"} 18:41:46 [18:41:46.997] INFO: prover-client:proving-broker Proving Broker started 18:41:46 [18:41:46.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59e210847e33b958 epochNumber=1 {"provingJobId":"1:BASE_PARITY:59e210847e33b958"} 18:41:46 [18:41:46.999] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPfKJW4/1 with map size 1073741824 18:41:47 [18:41:47.002] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:47 [18:41:47.020] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:59e210847e33b958 {"provingJobId":"1:BASE_PARITY:59e210847e33b958"} 18:41:47 [18:41:47.020] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:59e210847e33b958 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:59e210847e33b958"} 18:41:47 [18:41:47.027] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:968e64ee88f9ddff epochNumber=1 {"provingJobId":"1:BASE_PARITY:968e64ee88f9ddff"} 18:41:47 [18:41:47.034] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:59e210847e33b958 has already been completed {"provingJobId":"1:BASE_PARITY:59e210847e33b958"} 18:41:47 [18:41:47.039] INFO: prover-client:proving-broker Proving Broker started 18:41:47 [18:41:47.040] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:964f13247564b87a epochNumber=1 {"provingJobId":"1:BASE_PARITY:964f13247564b87a"} 18:41:47 [18:41:47.041] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDsgUgj/1 with map size 1073741824 18:41:47 [18:41:47.044] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:47 [18:41:47.066] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a06a415ba1a5945a epochNumber=2 {"provingJobId":"1:BASE_PARITY:a06a415ba1a5945a"} 18:41:47 [18:41:47.069] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testDsgUgj/2 with map size 1073741824 18:41:47 [18:41:47.072] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:48 [18:41:48.140] INFO: prover-client:proving-broker Proving Broker started 18:41:48 [18:41:48.146] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:964f13247564b87a settled=false {"provingJobId":"1:BASE_PARITY:964f13247564b87a","status":"pending"} 18:41:48 [18:41:48.150] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a06a415ba1a5945a settled=false {"provingJobId":"1:BASE_PARITY:a06a415ba1a5945a","status":"pending"} 18:41:48 [18:41:48.152] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:964f13247564b87a type=BASE_PARITY startedAt=1750099307088 older agent has taken job {"provingJobId":"1:BASE_PARITY:964f13247564b87a"} 18:41:48 [18:41:48.153] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:964f13247564b87a type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:964f13247564b87a"} 18:41:48 [18:41:48.161] INFO: prover-client:proving-broker Proving Broker started 18:41:48 [18:41:48.162] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6fb6d216d4e92b85 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6fb6d216d4e92b85"} 18:41:48 [18:41:48.163] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqifcUJ/1 with map size 1073741824 18:41:48 [18:41:48.170] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:48 [18:41:48.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:259cda837add1878 epochNumber=2 {"provingJobId":"1:BASE_PARITY:259cda837add1878"} 18:41:48 [18:41:48.189] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqifcUJ/2 with map size 1073741824 18:41:48 [18:41:48.191] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:49 [18:41:49.212] INFO: prover-client:proving-broker Proving Broker started 18:41:49 [18:41:49.214] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6fb6d216d4e92b85 settled=false {"provingJobId":"1:BASE_PARITY:6fb6d216d4e92b85","status":"pending"} 18:41:49 [18:41:49.216] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:259cda837add1878 settled=false {"provingJobId":"1:BASE_PARITY:259cda837add1878","status":"pending"} 18:41:49 [18:41:49.217] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6fb6d216d4e92b85 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:6fb6d216d4e92b85"} 18:41:49 [18:41:49.222] INFO: prover-client:proving-broker Proving Broker started 18:41:49 [18:41:49.223] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b49013fad7b8b075 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b49013fad7b8b075"} 18:41:49 [18:41:49.223] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7xC4OM/1 with map size 1073741824 18:41:49 [18:41:49.226] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:49 [18:41:49.245] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1a1949b74415004e epochNumber=2 {"provingJobId":"1:BASE_PARITY:1a1949b74415004e"} 18:41:49 [18:41:49.246] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7xC4OM/2 with map size 1073741824 18:41:49 [18:41:49.256] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.275] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.277] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b49013fad7b8b075 settled=false {"provingJobId":"1:BASE_PARITY:b49013fad7b8b075","status":"pending"} 18:41:50 [18:41:50.278] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1a1949b74415004e settled=false {"provingJobId":"1:BASE_PARITY:1a1949b74415004e","status":"pending"} 18:41:50 [18:41:50.279] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b49013fad7b8b075 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b49013fad7b8b075"} 18:41:50 [18:41:50.280] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b49013fad7b8b075 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b49013fad7b8b075"} 18:41:50 [18:41:50.289] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.290] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5931752d6f7ac9c epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5931752d6f7ac9c"} 18:41:50 [18:41:50.290] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYzU5op/1 with map size 1073741824 18:41:50 [18:41:50.291] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.307] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8ad6625c8b7f390 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a8ad6625c8b7f390"} 18:41:50 [18:41:50.308] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testYzU5op/2 with map size 1073741824 18:41:50 [18:41:50.311] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.327] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f5931752d6f7ac9c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f5931752d6f7ac9c"} 18:41:50 [18:41:50.332] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a8ad6625c8b7f390 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a8ad6625c8b7f390"} 18:41:50 [18:41:50.342] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.343] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:05894de7538dff28 epochNumber=1 {"provingJobId":"1:BASE_PARITY:05894de7538dff28"} 18:41:50 [18:41:50.343] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLXSdCm/1 with map size 1073741824 18:41:50 [18:41:50.345] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.361] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:516704ecef568176 epochNumber=2 {"provingJobId":"1:BASE_PARITY:516704ecef568176"} 18:41:50 [18:41:50.362] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLXSdCm/2 with map size 1073741824 18:41:50 [18:41:50.364] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.382] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:05894de7538dff28 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:05894de7538dff28"} 18:41:50 [18:41:50.383] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:05894de7538dff28 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:05894de7538dff28"} 18:41:50 [18:41:50.389] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:516704ecef568176 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:516704ecef568176"} 18:41:50 [18:41:50.398] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.399] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:717792fa8659a446 err=test error {"provingJoId":"1:BASE_PARITY:717792fa8659a446"} 18:41:50 [18:41:50.402] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.402] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:dce6a6621bdb7101 not found {"provingJobId":"1:BASE_PARITY:dce6a6621bdb7101"} 18:41:50 [18:41:50.405] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.406] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8b4543811ea1d7fe epochNumber=1 {"provingJobId":"1:BASE_PARITY:8b4543811ea1d7fe"} 18:41:50 [18:41:50.406] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHC1vZ7/1 with map size 1073741824 18:41:50 [18:41:50.407] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.429] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.430] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b6953b0558554288 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b6953b0558554288"} 18:41:50 [18:41:50.432] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmdM5ZZ/1 with map size 1073741824 18:41:50 [18:41:50.435] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.558] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b6953b0558554288 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:b6953b0558554288"} 18:41:50 [18:41:50.565] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.566] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:278cb7bcf3df6d68 epochNumber=1 {"provingJobId":"1:BASE_PARITY:278cb7bcf3df6d68"} 18:41:50 [18:41:50.566] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testucJ3x6/1 with map size 1073741824 18:41:50 [18:41:50.569] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.693] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:278cb7bcf3df6d68 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:278cb7bcf3df6d68"} 18:41:50 [18:41:50.695] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8c8d9cf4c1da8e73 epochNumber=10 {"provingJobId":"1:BASE_PARITY:8c8d9cf4c1da8e73"} 18:41:50 [18:41:50.695] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testucJ3x6/10 with map size 1073741824 18:41:50 [18:41:50.699] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:50 [18:41:50.718] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:50 [18:41:50.719] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:41:50 [18:41:50.849] INFO: prover-client:proving-broker Proving Broker started 18:41:50 [18:41:50.850] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8d0a585fb064187e epochNumber=1 {"provingJobId":"1:BASE_PARITY:8d0a585fb064187e"} 18:41:50 [18:41:50.853] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkKOVDG/1 with map size 1073741824 18:41:50 [18:41:50.859] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.028] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8d0a585fb064187e timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:8d0a585fb064187e"} 18:41:51 [18:41:51.086] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.087] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:718490ce1cc48505 epochNumber=1 {"provingJobId":"1:BASE_PARITY:718490ce1cc48505"} 18:41:51 [18:41:51.088] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testy5EPu4/1 with map size 1073741824 18:41:51 [18:41:51.091] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.110] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:718490ce1cc48505 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:718490ce1cc48505"} 18:41:51 [18:41:51.114] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e3e2d044f37d83a epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e3e2d044f37d83a"} 18:41:51 [18:41:51.117] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYzlcc1/1 with map size 1073741824 18:41:51 [18:41:51.120] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.139] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7e3e2d044f37d83a type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:7e3e2d044f37d83a"} 18:41:51 [18:41:51.140] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7e3e2d044f37d83a type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:7e3e2d044f37d83a"} 18:41:51 [18:41:51.155] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.156] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15ab58784c847985 epochNumber=1 {"provingJobId":"1:BASE_PARITY:15ab58784c847985"} 18:41:51 [18:41:51.157] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQQKZtO/1 with map size 1073741824 18:41:51 [18:41:51.159] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.179] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:15ab58784c847985 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:15ab58784c847985"} 18:41:51 [18:41:51.189] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.190] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:05965491a1435fe7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:05965491a1435fe7"} 18:41:51 [18:41:51.191] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testF7cbBx/1 with map size 1073741824 18:41:51 [18:41:51.194] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.215] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:05965491a1435fe7 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:05965491a1435fe7"} 18:41:51 [18:41:51.217] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:771dec04c63ef538 epochNumber=3 {"provingJobId":"1:BASE_PARITY:771dec04c63ef538"} 18:41:51 [18:41:51.217] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testF7cbBx/3 with map size 1073741824 18:41:51 [18:41:51.221] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.241] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:51 [18:41:51.241] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:41:51 [18:41:51.264] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:05965491a1435fe7 err=test error {"provingJoId":"1:BASE_PARITY:05965491a1435fe7"} 18:41:51 [18:41:51.273] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0qYTEN/1 with map size 1073741824 18:41:51 [18:41:51.277] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.300] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0qYTEN/2 with map size 1073741824 18:41:51 [18:41:51.304] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.320] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.322] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5165736287642b91 settled=false {"provingJobId":"1:BASE_PARITY:5165736287642b91","status":"pending"} 18:41:51 [18:41:51.324] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2b6708be99760e88 settled=false {"provingJobId":"1:BASE_PARITY:2b6708be99760e88","status":"pending"} 18:41:51 [18:41:51.334] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD14G8e/1 with map size 1073741824 18:41:51 [18:41:51.338] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.370] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testD14G8e/2 with map size 1073741824 18:41:51 [18:41:51.373] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.411] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.414] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3883e6c9bb8508d0 settled=true {"provingJobId":"1:BASE_PARITY:3883e6c9bb8508d0","status":"fulfilled"} 18:41:51 [18:41:51.416] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:51f1fc1b8ec7fc3c settled=true {"provingJobId":"2:BASE_PARITY:51f1fc1b8ec7fc3c","status":"fulfilled"} 18:41:51 [18:41:51.426] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9ZM4kf/1 with map size 1073741824 18:41:51 [18:41:51.432] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.468] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test9ZM4kf/2 with map size 1073741824 18:41:51 [18:41:51.475] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.500] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.502] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:06d2ebcd67ba35a0 settled=true {"provingJobId":"1:BASE_PARITY:06d2ebcd67ba35a0","status":"fulfilled"} 18:41:51 [18:41:51.504] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6c34713be91268af settled=false {"provingJobId":"1:BASE_PARITY:6c34713be91268af","status":"pending"} 18:41:51 [18:41:51.511] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.512] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8aabf37da0e699ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:8aabf37da0e699ee"} 18:41:51 [18:41:51.513] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbspdW5/1 with map size 1073741824 18:41:51 [18:41:51.517] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.553] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.554] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:000a0896c0f32792 epochNumber=1 {"provingJobId":"1:BASE_PARITY:000a0896c0f32792"} 18:41:51 [18:41:51.555] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:000a0896c0f32792: Error: db error: Error: db error 18:41:51 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:000a0896c0f32792"} 18:41:51 [18:41:51.568] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.569] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac990b8a969684d0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac990b8a969684d0"} 18:41:51 [18:41:51.570] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUJ4dk5/1 with map size 1073741824 18:41:51 [18:41:51.574] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.599] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ac990b8a969684d0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ac990b8a969684d0"} 18:41:51 [18:41:51.600] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ac990b8a969684d0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ac990b8a969684d0"} 18:41:51 [18:41:51.614] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.615] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e97181ced4537668 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e97181ced4537668"} 18:41:51 [18:41:51.617] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtvYKmU/1 with map size 1073741824 18:41:51 [18:41:51.622] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.647] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e97181ced4537668 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e97181ced4537668"} 18:41:51 [18:41:51.647] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e97181ced4537668 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e97181ced4537668"} 18:41:51 [18:41:51.648] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:e97181ced4537668: Error: db error 18:41:51 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:e97181ced4537668"} 18:41:51 [18:41:51.655] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.656] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c70a78737826407b epochNumber=1 {"provingJobId":"1:BASE_PARITY:c70a78737826407b"} 18:41:51 [18:41:51.657] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUkxW1h/1 with map size 1073741824 18:41:51 [18:41:51.659] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.683] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c70a78737826407b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c70a78737826407b"} 18:41:51 [18:41:51.684] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c70a78737826407b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c70a78737826407b"} 18:41:51 [18:41:51.700] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.701] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06217eabbe69d619 epochNumber=1 {"provingJobId":"1:BASE_PARITY:06217eabbe69d619"} 18:41:51 [18:41:51.703] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEzmpHU/1 with map size 1073741824 18:41:51 [18:41:51.708] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.730] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:06217eabbe69d619 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:06217eabbe69d619"} 18:41:51 [18:41:51.731] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:06217eabbe69d619 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:06217eabbe69d619"} 18:41:51 [18:41:51.732] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:06217eabbe69d619 jobErr=test error: Error: db error 18:41:51 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:06217eabbe69d619"} 18:41:51 [18:41:51.738] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.739] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:04bffdff7837dcc7 not found {"provingJobId":"1:BASE_PARITY:04bffdff7837dcc7"} 18:41:51 [18:41:51.744] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.745] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:03735c9d7e61afd7 err=test error {"provingJoId":"1:BASE_PARITY:03735c9d7e61afd7"} 18:41:51 [18:41:51.755] INFO: prover-client:proving-broker Proving Broker started 18:41:51 [18:41:51.767] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:797a677f4744ce64 epochNumber=1 {"provingJobId":"1:BASE_PARITY:797a677f4744ce64"} 18:41:51 [18:41:51.769] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAFCcVw/1 with map size 1073741824 18:41:51 [18:41:51.772] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.792] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:797a677f4744ce64 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:797a677f4744ce64"} 18:41:51 [18:41:51.793] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:797a677f4744ce64 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:797a677f4744ce64"} 18:41:51 [18:41:51.810] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:4fae2b2541a60629 epochNumber=2 {"provingJobId":"2:BASE_PARITY:4fae2b2541a60629"} 18:41:51 [18:41:51.812] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAFCcVw/2 with map size 1073741824 18:41:51 [18:41:51.816] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.841] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:4fae2b2541a60629 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:4fae2b2541a60629"} 18:41:51 [18:41:51.841] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:4fae2b2541a60629 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:4fae2b2541a60629"} 18:41:51 [18:41:51.911] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:024c3b9483562199 epochNumber=3 {"provingJobId":"3:BASE_PARITY:024c3b9483562199"} 18:41:51 [18:41:51.913] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testAFCcVw/3 with map size 1073741824 18:41:51 [18:41:51.916] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:51 [18:41:51.935] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:51 [18:41:51.936] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:41:51 [18:41:51.995] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:e2a2d5768a12dc20 epochNumber=4 {"provingJobId":"4:BASE_PARITY:e2a2d5768a12dc20"} 18:41:51 [18:41:51.997] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testAFCcVw/4 with map size 1073741824 18:41:52 [18:41:52.001] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:52 [18:41:52.041] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:52 [18:41:52.043] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 18:41:52 [18:41:52.087] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:ff0d5023949c66ec epochNumber=5 {"provingJobId":"5:BASE_PARITY:ff0d5023949c66ec"} 18:41:52 [18:41:52.090] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testAFCcVw/5 with map size 1073741824 18:41:52 [18:41:52.092] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:41:52 [18:41:52.122] INFO: prover-client:proving-broker Cleaned up jobs=1 18:41:52 [18:41:52.122] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 18:41:52 PASS src/proving_broker/proving_broker.test.ts (14.036 s) 18:41:52 ProvingBroker 18:41:52 Producer API 18:41:52 ✓ refuses stale jobs (43 ms) 18:41:52 ✓ enqueues jobs (5 ms) 18:41:52 ✓ ignores duplicate jobs (5 ms) 18:41:52 ✓ reports correct status when enqueuing repeat jobs (6 ms) 18:41:52 ✓ throws an error in case of duplicate job IDs (40 ms) 18:41:52 ✓ returns not-found status for non-existing jobs (1 ms) 18:41:52 ✓ cancels jobs in queue (2 ms) 18:41:52 ✓ cancels jobs in-progress (1 ms) 18:41:52 ✓ returns job result if successful (10 ms) 18:41:52 ✓ returns job error if failed (3 ms) 18:41:52 ✓ correctly returns job status for concurrent writes (3 ms) 18:41:52 ✓ refuses stale jobs (50 ms) 18:41:52 ✓ enqueues jobs (35 ms) 18:41:52 ✓ ignores duplicate jobs (33 ms) 18:41:52 ✓ reports correct status when enqueuing repeat jobs (44 ms) 18:41:52 ✓ throws an error in case of duplicate job IDs (32 ms) 18:41:52 ✓ returns not-found status for non-existing jobs (2 ms) 18:41:52 ✓ cancels jobs in queue (32 ms) 18:41:52 ✓ cancels jobs in-progress (39 ms) 18:41:52 ✓ returns job result if successful (32 ms) 18:41:52 ✓ returns job error if failed (41 ms) 18:41:52 ✓ correctly returns job status for concurrent writes (40 ms) 18:41:52 Consumer API 18:41:52 ✓ returns undefined if no jobs are available (1 ms) 18:41:52 ✓ returns jobs in priority order (2 ms) 18:41:52 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 18:41:52 ✓ returns a job if it is in the allowList (2 ms) 18:41:52 ✓ returns the most important job if it is in the allowList (3 ms) 18:41:52 ✓ returns any job if filter is empty (2 ms) 18:41:52 ✓ returns a new job when reporting job success (3 ms) 18:41:52 ✓ returns a new job when reporting permanent error (2 ms) 18:41:52 ✓ returns a new job when reporting retry-able error (3 ms) 18:41:52 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 18:41:52 ✓ returns a new job if job is already in progress elsewhere (1055 ms) 18:41:52 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 18:41:52 ✓ avoids sending a completed job to a new agent after a restart (1005 ms) 18:41:52 ✓ tracks job result if in progress (4 ms) 18:41:52 ✓ tracks job result even if job is in queue (4 ms) 18:41:52 ✓ ignores reported job error if unknown job (2 ms) 18:41:52 ✓ ignores job result if unknown job (1 ms) 18:41:52 ✓ returns undefined if no jobs are available (3 ms) 18:41:52 ✓ returns jobs in priority order (50 ms) 18:41:52 ✓ returns undefined if no jobs are available for the given allowList (33 ms) 18:41:52 ✓ returns a job if it is in the allowList (55 ms) 18:41:52 ✓ returns the most important job if it is in the allowList (56 ms) 18:41:52 ✓ returns any job if filter is empty (74 ms) 18:41:52 ✓ returns a new job when reporting job success (38 ms) 18:41:52 ✓ returns a new job when reporting permanent error (46 ms) 18:41:52 ✓ returns a new job when reporting retry-able error (35 ms) 18:41:52 ✓ returns a new job when reporting progress if current one is cancelled (41 ms) 18:41:52 ✓ returns a new job if job is already in progress elsewhere (1120 ms) 18:41:52 ✓ avoids sending the same job to a new agent after a restart (1062 ms) 18:41:52 ✓ avoids sending a completed job to a new agent after a restart (1068 ms) 18:41:52 ✓ tracks job result if in progress (53 ms) 18:41:52 ✓ tracks job result even if job is in queue (56 ms) 18:41:52 ✓ ignores reported job error if unknown job (4 ms) 18:41:52 ✓ ignores job result if unknown job (3 ms) 18:41:52 Timeouts 18:41:52 ✓ tracks in progress jobs (1 ms) 18:41:52 ✓ re-enqueues jobs that time out (109 ms) 18:41:52 ✓ cancel stale jobs that time out (257 ms) 18:41:52 ✓ keeps the jobs in progress while it is alive (206 ms) 18:41:52 ✓ tracks in progress jobs (21 ms) 18:41:52 ✓ re-enqueues jobs that time out (136 ms) 18:41:52 ✓ cancel stale jobs that time out (283 ms) 18:41:52 ✓ keeps the jobs in progress while it is alive (236 ms) 18:41:52 Retries 18:41:52 ✓ retries jobs (3 ms) 18:41:52 ✓ retries up to a maximum number of times (3 ms) 18:41:52 ✓ passing retry=false does not retry (2 ms) 18:41:52 ✓ does not retry if job is stale (28 ms) 18:41:52 ✓ retries jobs (29 ms) 18:41:52 ✓ retries up to a maximum number of times (37 ms) 18:41:52 ✓ passing retry=false does not retry (37 ms) 18:41:52 ✓ does not retry if job is stale (79 ms) 18:41:52 Database management 18:41:52 ✓ re-enqueues proof requests on start (3 ms) 18:41:52 ✓ restores proof results on start (2 ms) 18:41:52 ✓ only re-enqueues unfinished jobs (2 ms) 18:41:52 ✓ saves job when enqueued (3 ms) 18:41:52 ✓ does not retain job if database fails to save (3 ms) 18:41:52 ✓ saves job result (2 ms) 18:41:52 ✓ saves result even if database fails to save (3 ms) 18:41:52 ✓ saves job error (3 ms) 18:41:52 ✓ saves job error even if database fails to save (2 ms) 18:41:52 ✓ does not save job result if job is unknown (2 ms) 18:41:52 ✓ does not save job error if job is unknown (2 ms) 18:41:52 ✓ cleans up old jobs periodically (265 ms) 18:41:52 ✓ re-enqueues proof requests on start (61 ms) 18:41:52 ✓ restores proof results on start (90 ms) 18:41:52 ✓ only re-enqueues unfinished jobs (86 ms) 18:41:52 ✓ saves job when enqueued (39 ms) 18:41:52 ✓ does not retain job if database fails to save (8 ms) 18:41:52 ✓ saves job result (52 ms) 18:41:52 ✓ saves result even if database fails to save (40 ms) 18:41:52 ✓ saves job error (44 ms) 18:41:52 ✓ saves job error even if database fails to save (39 ms) 18:41:52 ✓ does not save job result if job is unknown (6 ms) 18:41:52 ✓ does not save job error if job is unknown (4 ms) 18:41:52 ✓ cleans up old jobs periodically (424 ms) 18:41:52 18:41:52 Test Suites: 1 passed, 1 total 18:41:52 Tests: 96 passed, 96 total 18:41:52 Snapshots: 0 total 18:41:52 Time: 14.162 s 18:41:52 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 18:41:52 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?