Parent Log: http://ci.aztec-labs.com/3b68d8ef46724aee Command: 84c999e211bb73e4 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/3863bc02d671057d0cffd4ced17ca317788ff190 Env: REF_NAME=gh-readonly-queue/staging/pr-15238-db396f13b7ef374a230174bdc4078d9fbb3c788a CURRENT_VERSION=1.0.0 CI_FULL=1 Date: Tue Jun 24 12:13:26 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15238_amd64_x3-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_staging 12:13:30 [12:13:30.928] INFO: prover-client:proving-broker Proving Broker started 12:13:30 [12:13:30.936] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:88d65baffae0b483 epochNumber=42 {"provingJobId":"1:BASE_PARITY:88d65baffae0b483"} 12:13:30 [12:13:30.939] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:843022940de0e49c epochNumber=1 {"provingJobId":"1:BASE_PARITY:843022940de0e49c"} 12:13:30 [12:13:30.984] INFO: prover-client:proving-broker Proving Broker started 12:13:30 [12:13:30.985] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a5f4c062d8f3bbaf epochNumber=1 {"provingJobId":"1:BASE_PARITY:a5f4c062d8f3bbaf"} 12:13:30 [12:13:30.986] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a47e25a2a2c746c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a47e25a2a2c746c1"} 12:13:30 [12:13:30.988] INFO: prover-client:proving-broker Proving Broker started 12:13:30 [12:13:30.989] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e1cff3bef51358c5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e1cff3bef51358c5"} 12:13:30 [12:13:30.990] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e1cff3bef51358c5 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:e1cff3bef51358c5"} 12:13:30 [12:13:30.993] INFO: prover-client:proving-broker Proving Broker started 12:13:30 [12:13:30.995] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77322c40635be347 epochNumber=1 {"provingJobId":"1:BASE_PARITY:77322c40635be347"} 12:13:30 [12:13:30.996] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:77322c40635be347 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:77322c40635be347"} 12:13:30 [12:13:30.997] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:77322c40635be347 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:77322c40635be347"} 12:13:30 [12:13:30.998] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:77322c40635be347 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:77322c40635be347"} 12:13:30 [12:13:30.999] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.000] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:99ce8c2ec1aea914 epochNumber=1 {"provingJobId":"1:BASE_PARITY:99ce8c2ec1aea914"} 12:13:31 [12:13:31.022] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.023] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.024] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:118668a59687b249 epochNumber=1 {"provingJobId":"1:BASE_PARITY:118668a59687b249"} 12:13:31 [12:13:31.024] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:118668a59687b249 {"provingJobId":"1:BASE_PARITY:118668a59687b249"} 12:13:31 [12:13:31.025] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:118668a59687b249 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:118668a59687b249"} 12:13:31 [12:13:31.025] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:118668a59687b249 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:118668a59687b249"} 12:13:31 [12:13:31.026] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.027] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c9890c97fa3180a2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c9890c97fa3180a2"} 12:13:31 [12:13:31.028] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c9890c97fa3180a2 {"provingJobId":"1:BASE_PARITY:c9890c97fa3180a2"} 12:13:31 [12:13:31.028] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c9890c97fa3180a2 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c9890c97fa3180a2"} 12:13:31 [12:13:31.029] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.030] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e328ffdbf1e3ac4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e328ffdbf1e3ac4"} 12:13:31 [12:13:31.037] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7e328ffdbf1e3ac4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7e328ffdbf1e3ac4"} 12:13:31 [12:13:31.037] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7e328ffdbf1e3ac4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7e328ffdbf1e3ac4"} 12:13:31 [12:13:31.039] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.039] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:78eba52f822071d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:78eba52f822071d3"} 12:13:31 [12:13:31.040] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:78eba52f822071d3 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:78eba52f822071d3"} 12:13:31 [12:13:31.040] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:78eba52f822071d3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:78eba52f822071d3"} 12:13:31 [12:13:31.041] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.041] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:42916d7de9231ea3 epochNumber=0 {"provingJobId":"1:BASE_PARITY:42916d7de9231ea3"} 12:13:31 [12:13:31.042] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:42916d7de9231ea3 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:42916d7de9231ea3"} 12:13:31 [12:13:31.042] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f1c1464ab039e451 epochNumber=0 {"provingJobId":"1:BASE_PARITY:f1c1464ab039e451"} 12:13:31 [12:13:31.042] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:42916d7de9231ea3 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:42916d7de9231ea3"} 12:13:31 [12:13:31.042] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c6af2f02441cca2d epochNumber=0 {"provingJobId":"1:BASE_PARITY:c6af2f02441cca2d"} 12:13:31 [12:13:31.043] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.044] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.045] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b85f3b3f54abab58 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b85f3b3f54abab58"} 12:13:31 [12:13:31.045] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8a11438fec85a96 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a8a11438fec85a96"} 12:13:31 [12:13:31.046] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.046] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e06216012d9b4f64 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e06216012d9b4f64"} 12:13:31 [12:13:31.047] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.047] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2487ec10894cf03d epochNumber=1 {"provingJobId":"1:BASE_PARITY:2487ec10894cf03d"} 12:13:31 [12:13:31.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a01585dba552fa4c epochNumber=1 {"provingJobId":"1:BASE_PARITY:a01585dba552fa4c"} 12:13:31 [12:13:31.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5e329982d6db0075 epochNumber=2 {"provingJobId":"1:BASE_PARITY:5e329982d6db0075"} 12:13:31 [12:13:31.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:57aad44d06ccfa46 epochNumber=1 {"provingJobId":"1:BASE_PARITY:57aad44d06ccfa46"} 12:13:31 [12:13:31.049] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.050] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b47578d60c754032 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b47578d60c754032"} 12:13:31 [12:13:31.050] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b73c9e20e4565775 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b73c9e20e4565775"} 12:13:31 [12:13:31.050] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c77e335a40509f04 epochNumber=2 {"provingJobId":"1:BASE_PARITY:c77e335a40509f04"} 12:13:31 [12:13:31.051] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dd5dcdd9c8ca937f epochNumber=1 {"provingJobId":"1:BASE_PARITY:dd5dcdd9c8ca937f"} 12:13:31 [12:13:31.052] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.052] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:319503589bd295a9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:319503589bd295a9"} 12:13:31 [12:13:31.052] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8a59742a242f96d2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8a59742a242f96d2"} 12:13:31 [12:13:31.053] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b1eaee17ae014a6a epochNumber=2 {"provingJobId":"1:BASE_PARITY:b1eaee17ae014a6a"} 12:13:31 [12:13:31.054] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1835c2067994483e epochNumber=1 {"provingJobId":"1:BASE_PARITY:1835c2067994483e"} 12:13:31 [12:13:31.057] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.058] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08732b4e5a198170 epochNumber=1 {"provingJobId":"1:BASE_PARITY:08732b4e5a198170"} 12:13:31 [12:13:31.058] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f175ac762b7c94c3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f175ac762b7c94c3"} 12:13:31 [12:13:31.059] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:08732b4e5a198170 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:08732b4e5a198170"} 12:13:31 [12:13:31.060] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.061] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e42cc3fb3ee30954 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e42cc3fb3ee30954"} 12:13:31 [12:13:31.062] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac2f558f2c1a8da0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac2f558f2c1a8da0"} 12:13:31 [12:13:31.062] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e42cc3fb3ee30954 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:e42cc3fb3ee30954"} 12:13:31 [12:13:31.064] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.064] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fcb0b93b8f7f5f79 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fcb0b93b8f7f5f79"} 12:13:31 [12:13:31.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0c6910f24b64ae3a epochNumber=1 {"provingJobId":"1:BASE_PARITY:0c6910f24b64ae3a"} 12:13:31 [12:13:31.067] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fcb0b93b8f7f5f79 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:fcb0b93b8f7f5f79"} 12:13:31 [12:13:31.069] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.069] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0b8711aecb815d21 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0b8711aecb815d21"} 12:13:31 [12:13:31.070] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0b8711aecb815d21 {"provingJobId":"1:BASE_PARITY:0b8711aecb815d21"} 12:13:31 [12:13:31.070] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0b8711aecb815d21 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:0b8711aecb815d21"} 12:13:31 [12:13:31.071] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1d15fa0d3746b3b4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1d15fa0d3746b3b4"} 12:13:31 [12:13:31.071] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0b8711aecb815d21 has already been completed {"provingJobId":"1:BASE_PARITY:0b8711aecb815d21"} 12:13:31 [12:13:31.073] INFO: prover-client:proving-broker Proving Broker started 12:13:31 [12:13:31.074] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5dbf10df86a93bf2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5dbf10df86a93bf2"} 12:13:31 [12:13:31.074] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e8884bc78aa02b4a epochNumber=2 {"provingJobId":"1:BASE_PARITY:e8884bc78aa02b4a"} 12:13:32 [12:13:32.128] INFO: prover-client:proving-broker Proving Broker started 12:13:32 [12:13:32.128] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5dbf10df86a93bf2 settled=false {"provingJobId":"1:BASE_PARITY:5dbf10df86a93bf2","status":"pending"} 12:13:32 [12:13:32.129] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e8884bc78aa02b4a settled=false {"provingJobId":"1:BASE_PARITY:e8884bc78aa02b4a","status":"pending"} 12:13:32 [12:13:32.130] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:5dbf10df86a93bf2 type=BASE_PARITY startedAt=1750767211075 older agent has taken job {"provingJobId":"1:BASE_PARITY:5dbf10df86a93bf2"} 12:13:32 [12:13:32.130] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5dbf10df86a93bf2 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:5dbf10df86a93bf2"} 12:13:32 [12:13:32.132] INFO: prover-client:proving-broker Proving Broker started 12:13:32 [12:13:32.132] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:df9ebf5bfcda2ade epochNumber=1 {"provingJobId":"1:BASE_PARITY:df9ebf5bfcda2ade"} 12:13:32 [12:13:32.133] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dc56d07c6a1e97bb epochNumber=2 {"provingJobId":"1:BASE_PARITY:dc56d07c6a1e97bb"} 12:13:33 [12:13:33.135] INFO: prover-client:proving-broker Proving Broker started 12:13:33 [12:13:33.135] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:df9ebf5bfcda2ade settled=false {"provingJobId":"1:BASE_PARITY:df9ebf5bfcda2ade","status":"pending"} 12:13:33 [12:13:33.135] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dc56d07c6a1e97bb settled=false {"provingJobId":"1:BASE_PARITY:dc56d07c6a1e97bb","status":"pending"} 12:13:33 [12:13:33.136] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:df9ebf5bfcda2ade type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:df9ebf5bfcda2ade"} 12:13:33 [12:13:33.137] INFO: prover-client:proving-broker Proving Broker started 12:13:33 [12:13:33.138] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77a7296850e2a596 epochNumber=1 {"provingJobId":"1:BASE_PARITY:77a7296850e2a596"} 12:13:33 [12:13:33.138] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5ceb955e16c225fc epochNumber=2 {"provingJobId":"1:BASE_PARITY:5ceb955e16c225fc"} 12:13:34 [12:13:34.139] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.139] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:77a7296850e2a596 settled=false {"provingJobId":"1:BASE_PARITY:77a7296850e2a596","status":"pending"} 12:13:34 [12:13:34.140] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5ceb955e16c225fc settled=false {"provingJobId":"1:BASE_PARITY:5ceb955e16c225fc","status":"pending"} 12:13:34 [12:13:34.140] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:77a7296850e2a596 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:77a7296850e2a596"} 12:13:34 [12:13:34.140] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:77a7296850e2a596 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:77a7296850e2a596"} 12:13:34 [12:13:34.142] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.142] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:381d9cb443bbb117 epochNumber=1 {"provingJobId":"1:BASE_PARITY:381d9cb443bbb117"} 12:13:34 [12:13:34.143] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9982b794f6c4d9f3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9982b794f6c4d9f3"} 12:13:34 [12:13:34.143] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:381d9cb443bbb117 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:381d9cb443bbb117"} 12:13:34 [12:13:34.144] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9982b794f6c4d9f3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:9982b794f6c4d9f3"} 12:13:34 [12:13:34.145] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.146] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:686264c80835462e epochNumber=1 {"provingJobId":"1:BASE_PARITY:686264c80835462e"} 12:13:34 [12:13:34.146] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0481171e3607204 epochNumber=2 {"provingJobId":"1:BASE_PARITY:c0481171e3607204"} 12:13:34 [12:13:34.146] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:686264c80835462e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:686264c80835462e"} 12:13:34 [12:13:34.147] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:686264c80835462e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:686264c80835462e"} 12:13:34 [12:13:34.147] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c0481171e3607204 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c0481171e3607204"} 12:13:34 [12:13:34.148] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.149] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:47eeeb20c8622131 err=test error {"provingJoId":"1:BASE_PARITY:47eeeb20c8622131"} 12:13:34 [12:13:34.150] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.150] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:35619d41a090f212 not found {"provingJobId":"1:BASE_PARITY:35619d41a090f212"} 12:13:34 [12:13:34.152] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.152] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5f36be4f1b7be14b epochNumber=1 {"provingJobId":"1:BASE_PARITY:5f36be4f1b7be14b"} 12:13:34 [12:13:34.154] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.154] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:01f6937ff956fb0d epochNumber=1 {"provingJobId":"1:BASE_PARITY:01f6937ff956fb0d"} 12:13:34 [12:13:34.256] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:01f6937ff956fb0d timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:01f6937ff956fb0d"} 12:13:34 [12:13:34.263] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.266] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e8e42018c106697f epochNumber=1 {"provingJobId":"1:BASE_PARITY:e8e42018c106697f"} 12:13:34 [12:13:34.392] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e8e42018c106697f timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:e8e42018c106697f"} 12:13:34 [12:13:34.399] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1cb71a5d348f0db epochNumber=10 {"provingJobId":"1:BASE_PARITY:a1cb71a5d348f0db"} 12:13:34 [12:13:34.418] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:34 [12:13:34.526] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.527] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba8e7f8c2f76f2c7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba8e7f8c2f76f2c7"} 12:13:34 [12:13:34.683] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ba8e7f8c2f76f2c7 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ba8e7f8c2f76f2c7"} 12:13:34 [12:13:34.731] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.732] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:13ccaa5959d060ec epochNumber=1 {"provingJobId":"1:BASE_PARITY:13ccaa5959d060ec"} 12:13:34 [12:13:34.733] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:13ccaa5959d060ec type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:13ccaa5959d060ec"} 12:13:34 [12:13:34.734] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.734] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:73a1d4f76968298d epochNumber=1 {"provingJobId":"1:BASE_PARITY:73a1d4f76968298d"} 12:13:34 [12:13:34.735] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:73a1d4f76968298d type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:73a1d4f76968298d"} 12:13:34 [12:13:34.735] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:73a1d4f76968298d type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:73a1d4f76968298d"} 12:13:34 [12:13:34.736] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.737] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04d8deb68a00c12d epochNumber=1 {"provingJobId":"1:BASE_PARITY:04d8deb68a00c12d"} 12:13:34 [12:13:34.737] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:04d8deb68a00c12d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:04d8deb68a00c12d"} 12:13:34 [12:13:34.738] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.739] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b7a09c43323ba4e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b7a09c43323ba4e5"} 12:13:34 [12:13:34.739] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b7a09c43323ba4e5 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:b7a09c43323ba4e5"} 12:13:34 [12:13:34.740] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eebb0887def463e7 epochNumber=3 {"provingJobId":"1:BASE_PARITY:eebb0887def463e7"} 12:13:34 [12:13:34.764] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:34 [12:13:34.765] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b7a09c43323ba4e5 err=test error {"provingJoId":"1:BASE_PARITY:b7a09c43323ba4e5"} 12:13:34 [12:13:34.767] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.773] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fb81840f45966ced settled=false {"provingJobId":"1:BASE_PARITY:fb81840f45966ced","status":"pending"} 12:13:34 [12:13:34.774] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1ae151800fec4564 settled=false {"provingJobId":"1:BASE_PARITY:1ae151800fec4564","status":"pending"} 12:13:34 [12:13:34.777] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.778] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b5ee1a45adb310d7 settled=true {"provingJobId":"1:BASE_PARITY:b5ee1a45adb310d7","status":"fulfilled"} 12:13:34 [12:13:34.778] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:c6b734d64733fc61 settled=true {"provingJobId":"2:BASE_PARITY:c6b734d64733fc61","status":"fulfilled"} 12:13:34 [12:13:34.780] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.780] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b494b9b7fc7a02ab settled=true {"provingJobId":"1:BASE_PARITY:b494b9b7fc7a02ab","status":"fulfilled"} 12:13:34 [12:13:34.780] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a997fe9274619d86 settled=false {"provingJobId":"1:BASE_PARITY:a997fe9274619d86","status":"pending"} 12:13:34 [12:13:34.782] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.784] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:394d5f9fb20bccab epochNumber=1 {"provingJobId":"1:BASE_PARITY:394d5f9fb20bccab"} 12:13:34 [12:13:34.786] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.787] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51200978b7439f8a epochNumber=1 {"provingJobId":"1:BASE_PARITY:51200978b7439f8a"} 12:13:34 [12:13:34.788] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:51200978b7439f8a: Error: db error: Error: db error 12:13:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:51200978b7439f8a"} 12:13:34 [12:13:34.790] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.790] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6ca95452974c443a epochNumber=1 {"provingJobId":"1:BASE_PARITY:6ca95452974c443a"} 12:13:34 [12:13:34.791] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6ca95452974c443a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6ca95452974c443a"} 12:13:34 [12:13:34.791] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6ca95452974c443a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6ca95452974c443a"} 12:13:34 [12:13:34.792] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.793] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d064cac63676cb00 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d064cac63676cb00"} 12:13:34 [12:13:34.793] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d064cac63676cb00 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d064cac63676cb00"} 12:13:34 [12:13:34.794] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d064cac63676cb00 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d064cac63676cb00"} 12:13:34 [12:13:34.795] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:d064cac63676cb00: Error: db error 12:13:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:d064cac63676cb00"} 12:13:34 [12:13:34.796] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.797] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ac7904601850a7ed epochNumber=1 {"provingJobId":"1:BASE_PARITY:ac7904601850a7ed"} 12:13:34 [12:13:34.798] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ac7904601850a7ed type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ac7904601850a7ed"} 12:13:34 [12:13:34.798] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ac7904601850a7ed type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ac7904601850a7ed"} 12:13:34 [12:13:34.800] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.800] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:749117794bed7fb9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:749117794bed7fb9"} 12:13:34 [12:13:34.800] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:749117794bed7fb9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:749117794bed7fb9"} 12:13:34 [12:13:34.801] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:749117794bed7fb9 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:749117794bed7fb9"} 12:13:34 [12:13:34.801] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:749117794bed7fb9 jobErr=test error: Error: db error 12:13:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:749117794bed7fb9"} 12:13:34 [12:13:34.803] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.803] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bf64efa8dc3bc3f6 not found {"provingJobId":"1:BASE_PARITY:bf64efa8dc3bc3f6"} 12:13:34 [12:13:34.804] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.805] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2dcba9c67a053015 err=test error {"provingJoId":"1:BASE_PARITY:2dcba9c67a053015"} 12:13:34 [12:13:34.806] INFO: prover-client:proving-broker Proving Broker started 12:13:34 [12:13:34.817] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:db3ed5b60b1593dd epochNumber=1 {"provingJobId":"1:BASE_PARITY:db3ed5b60b1593dd"} 12:13:34 [12:13:34.818] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:db3ed5b60b1593dd type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:db3ed5b60b1593dd"} 12:13:34 [12:13:34.818] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:db3ed5b60b1593dd type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:db3ed5b60b1593dd"} 12:13:34 [12:13:34.829] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:23f21df5c144aa3c epochNumber=2 {"provingJobId":"2:BASE_PARITY:23f21df5c144aa3c"} 12:13:34 [12:13:34.830] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:23f21df5c144aa3c type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:23f21df5c144aa3c"} 12:13:34 [12:13:34.830] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:23f21df5c144aa3c type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:23f21df5c144aa3c"} 12:13:34 [12:13:34.891] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:b611384927e93c13 epochNumber=3 {"provingJobId":"3:BASE_PARITY:b611384927e93c13"} 12:13:34 [12:13:34.908] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:34 [12:13:34.954] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:9c94260837f0fa31 epochNumber=4 {"provingJobId":"4:BASE_PARITY:9c94260837f0fa31"} 12:13:34 [12:13:34.959] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:35 [12:13:35.015] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:a626474a1116d208 epochNumber=5 {"provingJobId":"5:BASE_PARITY:a626474a1116d208"} 12:13:35 [12:13:35.036] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:35 [12:13:35.070] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.070] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f4683950fe20d796 epochNumber=42 {"provingJobId":"1:BASE_PARITY:f4683950fe20d796"} 12:13:35 [12:13:35.072] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-test2bZlWg/42 with map size 1073741824 12:13:35 [12:13:35.076] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.103] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:322324c6732fe148 epochNumber=1 {"provingJobId":"1:BASE_PARITY:322324c6732fe148"} 12:13:35 [12:13:35.109] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:41b0d55a1ec24b07 epochNumber=1 {"provingJobId":"1:BASE_PARITY:41b0d55a1ec24b07"} 12:13:35 [12:13:35.111] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAXDVJs/1 with map size 1073741824 12:13:35 [12:13:35.112] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.134] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61b0a137dce377fa epochNumber=1 {"provingJobId":"1:BASE_PARITY:61b0a137dce377fa"} 12:13:35 [12:13:35.144] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.145] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b487d1c6b83a2980 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b487d1c6b83a2980"} 12:13:35 [12:13:35.145] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3uD30O/1 with map size 1073741824 12:13:35 [12:13:35.148] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.176] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b487d1c6b83a2980 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b487d1c6b83a2980"} 12:13:35 [12:13:35.180] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.181] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7307c1537604cf6b epochNumber=1 {"provingJobId":"1:BASE_PARITY:7307c1537604cf6b"} 12:13:35 [12:13:35.182] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl5R59A/1 with map size 1073741824 12:13:35 [12:13:35.184] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.204] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7307c1537604cf6b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7307c1537604cf6b"} 12:13:35 [12:13:35.205] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7307c1537604cf6b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7307c1537604cf6b"} 12:13:35 [12:13:35.211] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7307c1537604cf6b epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7307c1537604cf6b"} 12:13:35 [12:13:35.217] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.217] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:781dcc21c25f1ba4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:781dcc21c25f1ba4"} 12:13:35 [12:13:35.219] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testj9z2f4/1 with map size 1073741824 12:13:35 [12:13:35.223] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.247] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.251] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.251] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:43f2f359bb3231f9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:43f2f359bb3231f9"} 12:13:35 [12:13:35.252] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuELLfK/1 with map size 1073741824 12:13:35 [12:13:35.256] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.277] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:43f2f359bb3231f9 {"provingJobId":"1:BASE_PARITY:43f2f359bb3231f9"} 12:13:35 [12:13:35.278] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:43f2f359bb3231f9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:43f2f359bb3231f9"} 12:13:35 [12:13:35.278] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:43f2f359bb3231f9 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:43f2f359bb3231f9"} 12:13:35 [12:13:35.287] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.287] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:205a08a0f9e4fb81 epochNumber=1 {"provingJobId":"1:BASE_PARITY:205a08a0f9e4fb81"} 12:13:35 [12:13:35.288] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDoZwsi/1 with map size 1073741824 12:13:35 [12:13:35.289] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.307] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:205a08a0f9e4fb81 {"provingJobId":"1:BASE_PARITY:205a08a0f9e4fb81"} 12:13:35 [12:13:35.308] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:205a08a0f9e4fb81 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:205a08a0f9e4fb81"} 12:13:35 [12:13:35.316] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.317] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a39343a5822a418 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a39343a5822a418"} 12:13:35 [12:13:35.318] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1S5e8m/1 with map size 1073741824 12:13:35 [12:13:35.321] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.339] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5a39343a5822a418 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5a39343a5822a418"} 12:13:35 [12:13:35.340] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5a39343a5822a418 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5a39343a5822a418"} 12:13:35 [12:13:35.348] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.348] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:80f80e333941be52 epochNumber=1 {"provingJobId":"1:BASE_PARITY:80f80e333941be52"} 12:13:35 [12:13:35.350] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2N797i/1 with map size 1073741824 12:13:35 [12:13:35.352] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.375] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:80f80e333941be52 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:80f80e333941be52"} 12:13:35 [12:13:35.376] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:80f80e333941be52 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:80f80e333941be52"} 12:13:35 [12:13:35.385] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.386] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ee63a1e8e0694c35 epochNumber=0 {"provingJobId":"1:BASE_PARITY:ee63a1e8e0694c35"} 12:13:35 [12:13:35.386] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testvCbhee/0 with map size 1073741824 12:13:35 [12:13:35.389] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.408] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ee63a1e8e0694c35 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ee63a1e8e0694c35"} 12:13:35 [12:13:35.410] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc850fc0a5a1b76f epochNumber=0 {"provingJobId":"1:BASE_PARITY:cc850fc0a5a1b76f"} 12:13:35 [12:13:35.410] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ee63a1e8e0694c35 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ee63a1e8e0694c35"} 12:13:35 [12:13:35.411] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:30a56ec628123c83 epochNumber=0 {"provingJobId":"1:BASE_PARITY:30a56ec628123c83"} 12:13:35 [12:13:35.426] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.429] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.429] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:97a76c3563166ff4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:97a76c3563166ff4"} 12:13:35 [12:13:35.430] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testgYlETo/2 with map size 1073741824 12:13:35 [12:13:35.432] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.449] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:537ab512fcc2fc8e epochNumber=1 {"provingJobId":"1:BASE_PARITY:537ab512fcc2fc8e"} 12:13:35 [12:13:35.450] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgYlETo/1 with map size 1073741824 12:13:35 [12:13:35.452] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.473] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.474] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:81db94a554a73388 epochNumber=1 {"provingJobId":"1:BASE_PARITY:81db94a554a73388"} 12:13:35 [12:13:35.475] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJqssEf/1 with map size 1073741824 12:13:35 [12:13:35.479] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.501] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.502] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b3868b83ca81592c epochNumber=1 {"provingJobId":"1:BASE_PARITY:b3868b83ca81592c"} 12:13:35 [12:13:35.503] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testA8sYpJ/1 with map size 1073741824 12:13:35 [12:13:35.505] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.522] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e0fb6fa6a56336b epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e0fb6fa6a56336b"} 12:13:35 [12:13:35.528] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:66119acc2b400bdb epochNumber=2 {"provingJobId":"1:BASE_PARITY:66119acc2b400bdb"} 12:13:35 [12:13:35.529] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testA8sYpJ/2 with map size 1073741824 12:13:35 [12:13:35.531] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.549] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:18e281b8934309a7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:18e281b8934309a7"} 12:13:35 [12:13:35.559] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.560] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dee43165258422e9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dee43165258422e9"} 12:13:35 [12:13:35.560] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVrmY8m/1 with map size 1073741824 12:13:35 [12:13:35.562] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.587] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:176cf900c4df939f epochNumber=1 {"provingJobId":"1:BASE_PARITY:176cf900c4df939f"} 12:13:35 [12:13:35.595] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b7ae7f7b628be081 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b7ae7f7b628be081"} 12:13:35 [12:13:35.596] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVrmY8m/2 with map size 1073741824 12:13:35 [12:13:35.599] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.620] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e9806ced87829452 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e9806ced87829452"} 12:13:35 [12:13:35.632] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.632] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8e9a90b63b52d5f6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8e9a90b63b52d5f6"} 12:13:35 [12:13:35.633] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhjsESQ/1 with map size 1073741824 12:13:35 [12:13:35.635] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.658] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:592c7bb77ef2d678 epochNumber=1 {"provingJobId":"1:BASE_PARITY:592c7bb77ef2d678"} 12:13:35 [12:13:35.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ae75b00e57030fa epochNumber=2 {"provingJobId":"1:BASE_PARITY:8ae75b00e57030fa"} 12:13:35 [12:13:35.668] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhjsESQ/2 with map size 1073741824 12:13:35 [12:13:35.673] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.692] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5036e7f6e1bb9d0a epochNumber=1 {"provingJobId":"1:BASE_PARITY:5036e7f6e1bb9d0a"} 12:13:35 [12:13:35.701] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.701] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:48d0e0f06b027fd7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:48d0e0f06b027fd7"} 12:13:35 [12:13:35.702] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3lT3pK/1 with map size 1073741824 12:13:35 [12:13:35.705] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.723] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b1b5f33ba09deec9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b1b5f33ba09deec9"} 12:13:35 [12:13:35.730] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:48d0e0f06b027fd7 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:48d0e0f06b027fd7"} 12:13:35 [12:13:35.740] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.740] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d68054c65d54ade epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d68054c65d54ade"} 12:13:35 [12:13:35.741] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkzi7Dh/1 with map size 1073741824 12:13:35 [12:13:35.743] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.766] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b89bba63bbc54d87 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b89bba63bbc54d87"} 12:13:35 [12:13:35.774] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3d68054c65d54ade type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:3d68054c65d54ade"} 12:13:35 [12:13:35.782] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.783] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a1136d2d50c7a4ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:a1136d2d50c7a4ba"} 12:13:35 [12:13:35.785] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkOrnZe/1 with map size 1073741824 12:13:35 [12:13:35.786] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.804] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1dbe2e572e7bb480 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1dbe2e572e7bb480"} 12:13:35 [12:13:35.811] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a1136d2d50c7a4ba type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:a1136d2d50c7a4ba"} 12:13:35 [12:13:35.814] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.815] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:413f4c83afac6c81 epochNumber=1 {"provingJobId":"1:BASE_PARITY:413f4c83afac6c81"} 12:13:35 [12:13:35.815] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFOB38l/1 with map size 1073741824 12:13:35 [12:13:35.819] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.836] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:413f4c83afac6c81 {"provingJobId":"1:BASE_PARITY:413f4c83afac6c81"} 12:13:35 [12:13:35.837] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:413f4c83afac6c81 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:413f4c83afac6c81"} 12:13:35 [12:13:35.843] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11a8805dbc0ee3cc epochNumber=1 {"provingJobId":"1:BASE_PARITY:11a8805dbc0ee3cc"} 12:13:35 [12:13:35.848] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:413f4c83afac6c81 has already been completed {"provingJobId":"1:BASE_PARITY:413f4c83afac6c81"} 12:13:35 [12:13:35.852] INFO: prover-client:proving-broker Proving Broker started 12:13:35 [12:13:35.853] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:604852dd8698385d epochNumber=1 {"provingJobId":"1:BASE_PARITY:604852dd8698385d"} 12:13:35 [12:13:35.870] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw9eotd/1 with map size 1073741824 12:13:35 [12:13:35.874] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:35 [12:13:35.895] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9a18267bc63904c3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9a18267bc63904c3"} 12:13:35 [12:13:35.896] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testw9eotd/2 with map size 1073741824 12:13:35 [12:13:35.898] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:36 [12:13:36.968] INFO: prover-client:proving-broker Proving Broker started 12:13:36 [12:13:36.975] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:604852dd8698385d settled=false {"provingJobId":"1:BASE_PARITY:604852dd8698385d","status":"pending"} 12:13:36 [12:13:36.981] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9a18267bc63904c3 settled=false {"provingJobId":"1:BASE_PARITY:9a18267bc63904c3","status":"pending"} 12:13:36 [12:13:36.982] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:604852dd8698385d type=BASE_PARITY startedAt=1750767215915 older agent has taken job {"provingJobId":"1:BASE_PARITY:604852dd8698385d"} 12:13:36 [12:13:36.982] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:604852dd8698385d type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:604852dd8698385d"} 12:13:36 [12:13:36.986] INFO: prover-client:proving-broker Proving Broker started 12:13:36 [12:13:36.987] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7328184658388427 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7328184658388427"} 12:13:36 [12:13:36.989] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaWRgVN/1 with map size 1073741824 12:13:36 [12:13:36.991] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:37 [12:13:37.009] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:72648e31db5d1564 epochNumber=2 {"provingJobId":"1:BASE_PARITY:72648e31db5d1564"} 12:13:37 [12:13:37.011] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testaWRgVN/2 with map size 1073741824 12:13:37 [12:13:37.014] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:38 [12:13:38.031] INFO: prover-client:proving-broker Proving Broker started 12:13:38 [12:13:38.033] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7328184658388427 settled=false {"provingJobId":"1:BASE_PARITY:7328184658388427","status":"pending"} 12:13:38 [12:13:38.034] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:72648e31db5d1564 settled=false {"provingJobId":"1:BASE_PARITY:72648e31db5d1564","status":"pending"} 12:13:38 [12:13:38.035] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7328184658388427 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:7328184658388427"} 12:13:38 [12:13:38.038] INFO: prover-client:proving-broker Proving Broker started 12:13:38 [12:13:38.039] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2eb70b3d4f45d2a epochNumber=1 {"provingJobId":"1:BASE_PARITY:e2eb70b3d4f45d2a"} 12:13:38 [12:13:38.039] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test10KsJ4/1 with map size 1073741824 12:13:38 [12:13:38.041] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:38 [12:13:38.071] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2eb739784a27b363 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2eb739784a27b363"} 12:13:38 [12:13:38.076] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test10KsJ4/2 with map size 1073741824 12:13:38 [12:13:38.079] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.102] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.104] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e2eb70b3d4f45d2a settled=false {"provingJobId":"1:BASE_PARITY:e2eb70b3d4f45d2a","status":"pending"} 12:13:39 [12:13:39.105] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2eb739784a27b363 settled=false {"provingJobId":"1:BASE_PARITY:2eb739784a27b363","status":"pending"} 12:13:39 [12:13:39.106] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e2eb70b3d4f45d2a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e2eb70b3d4f45d2a"} 12:13:39 [12:13:39.106] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e2eb70b3d4f45d2a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e2eb70b3d4f45d2a"} 12:13:39 [12:13:39.117] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.118] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9431abad7166ce61 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9431abad7166ce61"} 12:13:39 [12:13:39.119] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGJG44k/1 with map size 1073741824 12:13:39 [12:13:39.121] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.136] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9a0fd1700f237906 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9a0fd1700f237906"} 12:13:39 [12:13:39.137] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testGJG44k/2 with map size 1073741824 12:13:39 [12:13:39.138] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.155] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9431abad7166ce61 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9431abad7166ce61"} 12:13:39 [12:13:39.162] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9a0fd1700f237906 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:9a0fd1700f237906"} 12:13:39 [12:13:39.174] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.175] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7332738010d19f7b epochNumber=1 {"provingJobId":"1:BASE_PARITY:7332738010d19f7b"} 12:13:39 [12:13:39.178] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIMwg1r/1 with map size 1073741824 12:13:39 [12:13:39.183] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.201] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:50aeddc84fe26fac epochNumber=2 {"provingJobId":"1:BASE_PARITY:50aeddc84fe26fac"} 12:13:39 [12:13:39.202] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testIMwg1r/2 with map size 1073741824 12:13:39 [12:13:39.204] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.221] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7332738010d19f7b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7332738010d19f7b"} 12:13:39 [12:13:39.221] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7332738010d19f7b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7332738010d19f7b"} 12:13:39 [12:13:39.227] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:50aeddc84fe26fac type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:50aeddc84fe26fac"} 12:13:39 [12:13:39.236] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.237] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:83d93d1eafcab3fc err=test error {"provingJoId":"1:BASE_PARITY:83d93d1eafcab3fc"} 12:13:39 [12:13:39.240] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.240] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8c7e3d8fe6e638be not found {"provingJobId":"1:BASE_PARITY:8c7e3d8fe6e638be"} 12:13:39 [12:13:39.242] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.243] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b097c1548144ad25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b097c1548144ad25"} 12:13:39 [12:13:39.245] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmt0NE3/1 with map size 1073741824 12:13:39 [12:13:39.246] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.270] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.271] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b06d09aaf8ae8081 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b06d09aaf8ae8081"} 12:13:39 [12:13:39.272] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg5seqx/1 with map size 1073741824 12:13:39 [12:13:39.275] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.398] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b06d09aaf8ae8081 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:b06d09aaf8ae8081"} 12:13:39 [12:13:39.403] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.404] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ff9fe25fe2a482a epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ff9fe25fe2a482a"} 12:13:39 [12:13:39.404] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9FsIUL/1 with map size 1073741824 12:13:39 [12:13:39.406] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.529] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2ff9fe25fe2a482a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:2ff9fe25fe2a482a"} 12:13:39 [12:13:39.537] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8b76f8e19e68225f epochNumber=10 {"provingJobId":"1:BASE_PARITY:8b76f8e19e68225f"} 12:13:39 [12:13:39.538] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test9FsIUL/10 with map size 1073741824 12:13:39 [12:13:39.540] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.555] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:39 [12:13:39.556] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:13:39 [12:13:39.684] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.685] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3fff5fad6173586 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3fff5fad6173586"} 12:13:39 [12:13:39.686] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpZyKKZ/1 with map size 1073741824 12:13:39 [12:13:39.688] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.864] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a3fff5fad6173586 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:a3fff5fad6173586"} 12:13:39 [12:13:39.913] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.914] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:71d8bd4e4880ab2b epochNumber=1 {"provingJobId":"1:BASE_PARITY:71d8bd4e4880ab2b"} 12:13:39 [12:13:39.915] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdAxSiu/1 with map size 1073741824 12:13:39 [12:13:39.918] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.935] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:71d8bd4e4880ab2b type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:71d8bd4e4880ab2b"} 12:13:39 [12:13:39.938] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fd1769f0915c5ab6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fd1769f0915c5ab6"} 12:13:39 [12:13:39.940] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvmQZ1j/1 with map size 1073741824 12:13:39 [12:13:39.941] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.958] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fd1769f0915c5ab6 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:fd1769f0915c5ab6"} 12:13:39 [12:13:39.959] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fd1769f0915c5ab6 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:fd1769f0915c5ab6"} 12:13:39 [12:13:39.969] INFO: prover-client:proving-broker Proving Broker started 12:13:39 [12:13:39.969] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3cf8307a6e2ea6c epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3cf8307a6e2ea6c"} 12:13:39 [12:13:39.970] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9H5vHm/1 with map size 1073741824 12:13:39 [12:13:39.973] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:39 [12:13:39.992] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3cf8307a6e2ea6c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a3cf8307a6e2ea6c"} 12:13:40 [12:13:40.002] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.003] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ec991747629df30 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4ec991747629df30"} 12:13:40 [12:13:40.003] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwr1MTj/1 with map size 1073741824 12:13:40 [12:13:40.007] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.023] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:4ec991747629df30 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:4ec991747629df30"} 12:13:40 [12:13:40.024] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b6a61aeb2e112a4 epochNumber=3 {"provingJobId":"1:BASE_PARITY:9b6a61aeb2e112a4"} 12:13:40 [12:13:40.025] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testwr1MTj/3 with map size 1073741824 12:13:40 [12:13:40.029] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.052] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:40 [12:13:40.052] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:13:40 [12:13:40.071] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:4ec991747629df30 err=test error {"provingJoId":"1:BASE_PARITY:4ec991747629df30"} 12:13:40 [12:13:40.081] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwn6xbz/1 with map size 1073741824 12:13:40 [12:13:40.083] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.103] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwn6xbz/2 with map size 1073741824 12:13:40 [12:13:40.107] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.132] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.134] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a5843e859ea97a08 settled=false {"provingJobId":"1:BASE_PARITY:a5843e859ea97a08","status":"pending"} 12:13:40 [12:13:40.135] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c8fde61f0030ef46 settled=false {"provingJobId":"1:BASE_PARITY:c8fde61f0030ef46","status":"pending"} 12:13:40 [12:13:40.143] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNj8mID/1 with map size 1073741824 12:13:40 [12:13:40.146] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.168] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNj8mID/2 with map size 1073741824 12:13:40 [12:13:40.171] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.199] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.203] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2056b4c0eacadf92 settled=true {"provingJobId":"1:BASE_PARITY:2056b4c0eacadf92","status":"fulfilled"} 12:13:40 [12:13:40.204] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:c8b8bb21bf5c39c1 settled=true {"provingJobId":"2:BASE_PARITY:c8b8bb21bf5c39c1","status":"fulfilled"} 12:13:40 [12:13:40.210] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9BvOUn/1 with map size 1073741824 12:13:40 [12:13:40.213] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.235] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test9BvOUn/2 with map size 1073741824 12:13:40 [12:13:40.237] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.257] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.259] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e659f2cb267e3006 settled=true {"provingJobId":"1:BASE_PARITY:e659f2cb267e3006","status":"fulfilled"} 12:13:40 [12:13:40.261] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a5ba9d68743ee5e0 settled=false {"provingJobId":"1:BASE_PARITY:a5ba9d68743ee5e0","status":"pending"} 12:13:40 [12:13:40.269] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fc84f7d16d9d0219 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fc84f7d16d9d0219"} 12:13:40 [12:13:40.270] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDAgY1v/1 with map size 1073741824 12:13:40 [12:13:40.273] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.295] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.296] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:513d135a8a939cea epochNumber=1 {"provingJobId":"1:BASE_PARITY:513d135a8a939cea"} 12:13:40 [12:13:40.297] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:513d135a8a939cea: Error: db error: Error: db error 12:13:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:513d135a8a939cea"} 12:13:40 [12:13:40.301] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.302] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77d49f5c9bc2bbd8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:77d49f5c9bc2bbd8"} 12:13:40 [12:13:40.302] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvjtzr4/1 with map size 1073741824 12:13:40 [12:13:40.304] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.320] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:77d49f5c9bc2bbd8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:77d49f5c9bc2bbd8"} 12:13:40 [12:13:40.321] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:77d49f5c9bc2bbd8 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:77d49f5c9bc2bbd8"} 12:13:40 [12:13:40.330] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.330] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7911e0c84db57247 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7911e0c84db57247"} 12:13:40 [12:13:40.332] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testt9y6BE/1 with map size 1073741824 12:13:40 [12:13:40.334] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.357] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7911e0c84db57247 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7911e0c84db57247"} 12:13:40 [12:13:40.358] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7911e0c84db57247 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7911e0c84db57247"} 12:13:40 [12:13:40.359] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:7911e0c84db57247: Error: db error 12:13:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:7911e0c84db57247"} 12:13:40 [12:13:40.363] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.364] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2d541fdf8ffc0315 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2d541fdf8ffc0315"} 12:13:40 [12:13:40.366] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsVICmD/1 with map size 1073741824 12:13:40 [12:13:40.368] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.386] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2d541fdf8ffc0315 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2d541fdf8ffc0315"} 12:13:40 [12:13:40.387] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2d541fdf8ffc0315 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2d541fdf8ffc0315"} 12:13:40 [12:13:40.399] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.399] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4920f650c18cd753 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4920f650c18cd753"} 12:13:40 [12:13:40.400] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEwx6Vc/1 with map size 1073741824 12:13:40 [12:13:40.402] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.422] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4920f650c18cd753 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4920f650c18cd753"} 12:13:40 [12:13:40.423] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4920f650c18cd753 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4920f650c18cd753"} 12:13:40 [12:13:40.423] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:4920f650c18cd753 jobErr=test error: Error: db error 12:13:40 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:4920f650c18cd753"} 12:13:40 [12:13:40.430] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.431] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4951006a0387844f not found {"provingJobId":"1:BASE_PARITY:4951006a0387844f"} 12:13:40 [12:13:40.436] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.436] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:48bbf0b0f0483088 err=test error {"provingJoId":"1:BASE_PARITY:48bbf0b0f0483088"} 12:13:40 [12:13:40.440] INFO: prover-client:proving-broker Proving Broker started 12:13:40 [12:13:40.451] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a70935911afb34b4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a70935911afb34b4"} 12:13:40 [12:13:40.453] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2a1uEm/1 with map size 1073741824 12:13:40 [12:13:40.455] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.474] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a70935911afb34b4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a70935911afb34b4"} 12:13:40 [12:13:40.474] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a70935911afb34b4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a70935911afb34b4"} 12:13:40 [12:13:40.491] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:1a4047409ec74103 epochNumber=2 {"provingJobId":"2:BASE_PARITY:1a4047409ec74103"} 12:13:40 [12:13:40.492] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test2a1uEm/2 with map size 1073741824 12:13:40 [12:13:40.494] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.512] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:1a4047409ec74103 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:1a4047409ec74103"} 12:13:40 [12:13:40.513] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:1a4047409ec74103 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:1a4047409ec74103"} 12:13:40 [12:13:40.580] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:7be4be479f9c09e0 epochNumber=3 {"provingJobId":"3:BASE_PARITY:7be4be479f9c09e0"} 12:13:40 [12:13:40.581] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test2a1uEm/3 with map size 1073741824 12:13:40 [12:13:40.585] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.619] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:40 [12:13:40.619] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 12:13:40 [12:13:40.664] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:ea2b6c6a26a22252 epochNumber=4 {"provingJobId":"4:BASE_PARITY:ea2b6c6a26a22252"} 12:13:40 [12:13:40.666] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test2a1uEm/4 with map size 1073741824 12:13:40 [12:13:40.669] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.697] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:40 [12:13:40.698] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 12:13:40 [12:13:40.751] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:8e499514ba54f53e epochNumber=5 {"provingJobId":"5:BASE_PARITY:8e499514ba54f53e"} 12:13:40 [12:13:40.752] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test2a1uEm/5 with map size 1073741824 12:13:40 [12:13:40.758] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 12:13:40 [12:13:40.801] INFO: prover-client:proving-broker Cleaned up jobs=1 12:13:40 [12:13:40.802] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 12:13:40 PASS src/proving_broker/proving_broker.test.ts (12.925 s) 12:13:40 ProvingBroker 12:13:40 Producer API 12:13:40 ✓ refuses stale jobs (58 ms) 12:13:40 ✓ enqueues jobs (4 ms) 12:13:40 ✓ ignores duplicate jobs (5 ms) 12:13:40 ✓ reports correct status when enqueuing repeat jobs (5 ms) 12:13:40 ✓ throws an error in case of duplicate job IDs (21 ms) 12:13:40 ✓ returns not-found status for non-existing jobs (2 ms) 12:13:40 ✓ cancels jobs in queue (3 ms) 12:13:40 ✓ cancels jobs in-progress (2 ms) 12:13:40 ✓ returns job result if successful (9 ms) 12:13:40 ✓ returns job error if failed (2 ms) 12:13:40 ✓ correctly returns job status for concurrent writes (2 ms) 12:13:40 ✓ refuses stale jobs (40 ms) 12:13:40 ✓ enqueues jobs (35 ms) 12:13:40 ✓ ignores duplicate jobs (34 ms) 12:13:40 ✓ reports correct status when enqueuing repeat jobs (35 ms) 12:13:40 ✓ throws an error in case of duplicate job IDs (32 ms) 12:13:40 ✓ returns not-found status for non-existing jobs (4 ms) 12:13:40 ✓ cancels jobs in queue (36 ms) 12:13:40 ✓ cancels jobs in-progress (28 ms) 12:13:40 ✓ returns job result if successful (31 ms) 12:13:40 ✓ returns job error if failed (37 ms) 12:13:40 ✓ correctly returns job status for concurrent writes (41 ms) 12:13:40 Consumer API 12:13:40 ✓ returns undefined if no jobs are available (1 ms) 12:13:40 ✓ returns jobs in priority order (2 ms) 12:13:40 ✓ returns undefined if no jobs are available for the given allowList (1 ms) 12:13:40 ✓ returns a job if it is in the allowList (2 ms) 12:13:40 ✓ returns the most important job if it is in the allowList (2 ms) 12:13:40 ✓ returns any job if filter is empty (5 ms) 12:13:40 ✓ returns a new job when reporting job success (4 ms) 12:13:40 ✓ returns a new job when reporting permanent error (3 ms) 12:13:40 ✓ returns a new job when reporting retry-able error (4 ms) 12:13:40 ✓ returns a new job when reporting progress if current one is cancelled (4 ms) 12:13:40 ✓ returns a new job if job is already in progress elsewhere (1058 ms) 12:13:40 ✓ avoids sending the same job to a new agent after a restart (1006 ms) 12:13:40 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 12:13:40 ✓ tracks job result if in progress (3 ms) 12:13:40 ✓ tracks job result even if job is in queue (3 ms) 12:13:40 ✓ ignores reported job error if unknown job (1 ms) 12:13:40 ✓ ignores job result if unknown job (2 ms) 12:13:40 ✓ returns undefined if no jobs are available (3 ms) 12:13:40 ✓ returns jobs in priority order (44 ms) 12:13:40 ✓ returns undefined if no jobs are available for the given allowList (27 ms) 12:13:40 ✓ returns a job if it is in the allowList (58 ms) 12:13:40 ✓ returns the most important job if it is in the allowList (72 ms) 12:13:40 ✓ returns any job if filter is empty (69 ms) 12:13:40 ✓ returns a new job when reporting job success (39 ms) 12:13:40 ✓ returns a new job when reporting permanent error (42 ms) 12:13:40 ✓ returns a new job when reporting retry-able error (31 ms) 12:13:40 ✓ returns a new job when reporting progress if current one is cancelled (37 ms) 12:13:40 ✓ returns a new job if job is already in progress elsewhere (1134 ms) 12:13:40 ✓ avoids sending the same job to a new agent after a restart (1052 ms) 12:13:40 ✓ avoids sending a completed job to a new agent after a restart (1078 ms) 12:13:40 ✓ tracks job result if in progress (55 ms) 12:13:40 ✓ tracks job result even if job is in queue (63 ms) 12:13:40 ✓ ignores reported job error if unknown job (3 ms) 12:13:40 ✓ ignores job result if unknown job (3 ms) 12:13:40 Timeouts 12:13:40 ✓ tracks in progress jobs (2 ms) 12:13:40 ✓ re-enqueues jobs that time out (109 ms) 12:13:40 ✓ cancel stale jobs that time out (262 ms) 12:13:40 ✓ keeps the jobs in progress while it is alive (204 ms) 12:13:40 ✓ tracks in progress jobs (27 ms) 12:13:40 ✓ re-enqueues jobs that time out (133 ms) 12:13:40 ✓ cancel stale jobs that time out (280 ms) 12:13:40 ✓ keeps the jobs in progress while it is alive (227 ms) 12:13:40 Retries 12:13:40 ✓ retries jobs (3 ms) 12:13:40 ✓ retries up to a maximum number of times (3 ms) 12:13:40 ✓ passing retry=false does not retry (2 ms) 12:13:40 ✓ does not retry if job is stale (28 ms) 12:13:40 ✓ retries jobs (26 ms) 12:13:40 ✓ retries up to a maximum number of times (30 ms) 12:13:40 ✓ passing retry=false does not retry (32 ms) 12:13:40 ✓ does not retry if job is stale (74 ms) 12:13:40 Database management 12:13:40 ✓ re-enqueues proof requests on start (10 ms) 12:13:40 ✓ restores proof results on start (3 ms) 12:13:40 ✓ only re-enqueues unfinished jobs (2 ms) 12:13:40 ✓ saves job when enqueued (3 ms) 12:13:40 ✓ does not retain job if database fails to save (3 ms) 12:13:40 ✓ saves job result (3 ms) 12:13:40 ✓ saves result even if database fails to save (4 ms) 12:13:40 ✓ saves job error (3 ms) 12:13:40 ✓ saves job error even if database fails to save (3 ms) 12:13:40 ✓ does not save job result if job is unknown (2 ms) 12:13:40 ✓ does not save job error if job is unknown (2 ms) 12:13:40 ✓ cleans up old jobs periodically (261 ms) 12:13:40 ✓ re-enqueues proof requests on start (62 ms) 12:13:40 ✓ restores proof results on start (69 ms) 12:13:40 ✓ only re-enqueues unfinished jobs (58 ms) 12:13:40 ✓ saves job when enqueued (27 ms) 12:13:40 ✓ does not retain job if database fails to save (5 ms) 12:13:40 ✓ saves job result (30 ms) 12:13:40 ✓ saves result even if database fails to save (33 ms) 12:13:40 ✓ saves job error (35 ms) 12:13:40 ✓ saves job error even if database fails to save (31 ms) 12:13:40 ✓ does not save job result if job is unknown (5 ms) 12:13:40 ✓ does not save job error if job is unknown (5 ms) 12:13:40 ✓ cleans up old jobs periodically (397 ms) 12:13:40 12:13:40 Test Suites: 1 passed, 1 total 12:13:40 Tests: 96 passed, 96 total 12:13:40 Snapshots: 0 total 12:13:40 Time: 13.023 s 12:13:40 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 12:13:40 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?