Parent Log: http://ci.aztec-labs.com/f390c288863afd04 Command: e67004c310c3c2c0 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/6e2f55a89973b55c8c3c210b7f3a692b4d3d6473 Env: REF_NAME=gh-readonly-queue/next/pr-15216-b52f0ff5ec39d6832a15613ed349e6e2f8077a2d CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Mon Jun 23 16:00:40 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15216_amd64_x4-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 16:00:44 [16:00:44.941] INFO: prover-client:proving-broker Proving Broker started 16:00:44 [16:00:44.949] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:02563653e5e80c4d epochNumber=42 {"provingJobId":"1:BASE_PARITY:02563653e5e80c4d"} 16:00:44 [16:00:44.953] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:ebdee2c8ced1249f epochNumber=1 {"provingJobId":"1:BASE_PARITY:ebdee2c8ced1249f"} 16:00:45 [16:00:45.002] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.003] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e1c641059ecf25ad epochNumber=1 {"provingJobId":"1:BASE_PARITY:e1c641059ecf25ad"} 16:00:45 [16:00:45.005] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:29df77832ad2d92f epochNumber=1 {"provingJobId":"1:BASE_PARITY:29df77832ad2d92f"} 16:00:45 [16:00:45.010] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.012] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5f5f42a0a461bcd5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5f5f42a0a461bcd5"} 16:00:45 [16:00:45.014] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:5f5f42a0a461bcd5 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:5f5f42a0a461bcd5"} 16:00:45 [16:00:45.016] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.017] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1737b2f7e78189d4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1737b2f7e78189d4"} 16:00:45 [16:00:45.019] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1737b2f7e78189d4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1737b2f7e78189d4"} 16:00:45 [16:00:45.020] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1737b2f7e78189d4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1737b2f7e78189d4"} 16:00:45 [16:00:45.020] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1737b2f7e78189d4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:1737b2f7e78189d4"} 16:00:45 [16:00:45.022] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.023] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:03789ad180676806 epochNumber=1 {"provingJobId":"1:BASE_PARITY:03789ad180676806"} 16:00:45 [16:00:45.052] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.055] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.056] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e4e97fdf85d3405 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e4e97fdf85d3405"} 16:00:45 [16:00:45.057] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2e4e97fdf85d3405 {"provingJobId":"1:BASE_PARITY:2e4e97fdf85d3405"} 16:00:45 [16:00:45.058] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2e4e97fdf85d3405 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2e4e97fdf85d3405"} 16:00:45 [16:00:45.058] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2e4e97fdf85d3405 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:2e4e97fdf85d3405"} 16:00:45 [16:00:45.060] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.061] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dd80471e65a901e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dd80471e65a901e8"} 16:00:45 [16:00:45.062] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:dd80471e65a901e8 {"provingJobId":"1:BASE_PARITY:dd80471e65a901e8"} 16:00:45 [16:00:45.062] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dd80471e65a901e8 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:dd80471e65a901e8"} 16:00:45 [16:00:45.064] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f2f934f3da8054c0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f2f934f3da8054c0"} 16:00:45 [16:00:45.065] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f2f934f3da8054c0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f2f934f3da8054c0"} 16:00:45 [16:00:45.066] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f2f934f3da8054c0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f2f934f3da8054c0"} 16:00:45 [16:00:45.067] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.068] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2546b1ac182a059f epochNumber=1 {"provingJobId":"1:BASE_PARITY:2546b1ac182a059f"} 16:00:45 [16:00:45.068] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2546b1ac182a059f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2546b1ac182a059f"} 16:00:45 [16:00:45.069] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2546b1ac182a059f type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2546b1ac182a059f"} 16:00:45 [16:00:45.070] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.070] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d5ede433f91ea762 epochNumber=0 {"provingJobId":"1:BASE_PARITY:d5ede433f91ea762"} 16:00:45 [16:00:45.071] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d5ede433f91ea762 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d5ede433f91ea762"} 16:00:45 [16:00:45.071] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:83b57dd6a0cc8856 epochNumber=0 {"provingJobId":"1:BASE_PARITY:83b57dd6a0cc8856"} 16:00:45 [16:00:45.071] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d5ede433f91ea762 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d5ede433f91ea762"} 16:00:45 [16:00:45.071] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2509b2161f2af0f2 epochNumber=0 {"provingJobId":"1:BASE_PARITY:2509b2161f2af0f2"} 16:00:45 [16:00:45.073] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.086] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.101] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe0ab92d30cb1ba5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fe0ab92d30cb1ba5"} 16:00:45 [16:00:45.102] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fccb632ad227cd29 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fccb632ad227cd29"} 16:00:45 [16:00:45.104] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:020f78b197f763e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:020f78b197f763e7"} 16:00:45 [16:00:45.106] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5c15bd7c53684c9b epochNumber=1 {"provingJobId":"1:BASE_PARITY:5c15bd7c53684c9b"} 16:00:45 [16:00:45.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5678dfb94c45b099 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5678dfb94c45b099"} 16:00:45 [16:00:45.107] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e223f5abca27982c epochNumber=2 {"provingJobId":"1:BASE_PARITY:e223f5abca27982c"} 16:00:45 [16:00:45.108] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d87966899f9a9511 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d87966899f9a9511"} 16:00:45 [16:00:45.109] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.109] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2d401550e99cd8ca epochNumber=1 {"provingJobId":"1:BASE_PARITY:2d401550e99cd8ca"} 16:00:45 [16:00:45.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:63923d3123bdc7e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:63923d3123bdc7e5"} 16:00:45 [16:00:45.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8407384ee43c287f epochNumber=2 {"provingJobId":"1:BASE_PARITY:8407384ee43c287f"} 16:00:45 [16:00:45.110] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33246764f4450b76 epochNumber=1 {"provingJobId":"1:BASE_PARITY:33246764f4450b76"} 16:00:45 [16:00:45.111] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.112] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a77ac79cc53cb2da epochNumber=1 {"provingJobId":"1:BASE_PARITY:a77ac79cc53cb2da"} 16:00:45 [16:00:45.112] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9de62abe06cba5ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:9de62abe06cba5ee"} 16:00:45 [16:00:45.112] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:59451a99c3e65f75 epochNumber=2 {"provingJobId":"1:BASE_PARITY:59451a99c3e65f75"} 16:00:45 [16:00:45.113] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f24e90d5c15a75c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f24e90d5c15a75c4"} 16:00:45 [16:00:45.114] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4e69c79acdeb122 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4e69c79acdeb122"} 16:00:45 [16:00:45.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d7641436d8750f8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d7641436d8750f8"} 16:00:45 [16:00:45.115] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c4e69c79acdeb122 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c4e69c79acdeb122"} 16:00:45 [16:00:45.117] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.118] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b04a89b62276ef02 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b04a89b62276ef02"} 16:00:45 [16:00:45.118] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fdc262c31b3b4f6e epochNumber=1 {"provingJobId":"1:BASE_PARITY:fdc262c31b3b4f6e"} 16:00:45 [16:00:45.119] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b04a89b62276ef02 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:b04a89b62276ef02"} 16:00:45 [16:00:45.120] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.120] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb76afbf245606ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:eb76afbf245606ba"} 16:00:45 [16:00:45.121] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:14207270392684f3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:14207270392684f3"} 16:00:45 [16:00:45.121] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:eb76afbf245606ba type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:eb76afbf245606ba"} 16:00:45 [16:00:45.122] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.123] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab4e3ef08cdb324f epochNumber=1 {"provingJobId":"1:BASE_PARITY:ab4e3ef08cdb324f"} 16:00:45 [16:00:45.123] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ab4e3ef08cdb324f {"provingJobId":"1:BASE_PARITY:ab4e3ef08cdb324f"} 16:00:45 [16:00:45.124] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ab4e3ef08cdb324f type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:ab4e3ef08cdb324f"} 16:00:45 [16:00:45.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0d1d151d3e80f18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b0d1d151d3e80f18"} 16:00:45 [16:00:45.125] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ab4e3ef08cdb324f has already been completed {"provingJobId":"1:BASE_PARITY:ab4e3ef08cdb324f"} 16:00:45 [16:00:45.126] INFO: prover-client:proving-broker Proving Broker started 16:00:45 [16:00:45.126] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f418aefa7b93ff7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4f418aefa7b93ff7"} 16:00:45 [16:00:45.126] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8f16eedc3875f5b2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:8f16eedc3875f5b2"} 16:00:46 [16:00:46.179] INFO: prover-client:proving-broker Proving Broker started 16:00:46 [16:00:46.180] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4f418aefa7b93ff7 settled=false {"provingJobId":"1:BASE_PARITY:4f418aefa7b93ff7","status":"pending"} 16:00:46 [16:00:46.180] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8f16eedc3875f5b2 settled=false {"provingJobId":"1:BASE_PARITY:8f16eedc3875f5b2","status":"pending"} 16:00:46 [16:00:46.181] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:4f418aefa7b93ff7 type=BASE_PARITY startedAt=1750694445127 older agent has taken job {"provingJobId":"1:BASE_PARITY:4f418aefa7b93ff7"} 16:00:46 [16:00:46.181] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4f418aefa7b93ff7 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:4f418aefa7b93ff7"} 16:00:46 [16:00:46.182] INFO: prover-client:proving-broker Proving Broker started 16:00:46 [16:00:46.183] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cdeffda4c34346b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cdeffda4c34346b5"} 16:00:46 [16:00:46.183] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15e8065a2d298f43 epochNumber=2 {"provingJobId":"1:BASE_PARITY:15e8065a2d298f43"} 16:00:47 [16:00:47.187] INFO: prover-client:proving-broker Proving Broker started 16:00:47 [16:00:47.187] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cdeffda4c34346b5 settled=false {"provingJobId":"1:BASE_PARITY:cdeffda4c34346b5","status":"pending"} 16:00:47 [16:00:47.188] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:15e8065a2d298f43 settled=false {"provingJobId":"1:BASE_PARITY:15e8065a2d298f43","status":"pending"} 16:00:47 [16:00:47.188] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cdeffda4c34346b5 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:cdeffda4c34346b5"} 16:00:47 [16:00:47.194] INFO: prover-client:proving-broker Proving Broker started 16:00:47 [16:00:47.195] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:979f473a6f9c2f23 epochNumber=1 {"provingJobId":"1:BASE_PARITY:979f473a6f9c2f23"} 16:00:47 [16:00:47.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d420e7c4f6c0f8c epochNumber=2 {"provingJobId":"1:BASE_PARITY:6d420e7c4f6c0f8c"} 16:00:48 [16:00:48.196] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.197] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:979f473a6f9c2f23 settled=false {"provingJobId":"1:BASE_PARITY:979f473a6f9c2f23","status":"pending"} 16:00:48 [16:00:48.197] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6d420e7c4f6c0f8c settled=false {"provingJobId":"1:BASE_PARITY:6d420e7c4f6c0f8c","status":"pending"} 16:00:48 [16:00:48.198] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:979f473a6f9c2f23 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:979f473a6f9c2f23"} 16:00:48 [16:00:48.198] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:979f473a6f9c2f23 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:979f473a6f9c2f23"} 16:00:48 [16:00:48.200] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3d93294fbd3b164d epochNumber=1 {"provingJobId":"1:BASE_PARITY:3d93294fbd3b164d"} 16:00:48 [16:00:48.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4a8375bc18ffb45e epochNumber=2 {"provingJobId":"1:BASE_PARITY:4a8375bc18ffb45e"} 16:00:48 [16:00:48.201] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3d93294fbd3b164d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3d93294fbd3b164d"} 16:00:48 [16:00:48.202] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4a8375bc18ffb45e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4a8375bc18ffb45e"} 16:00:48 [16:00:48.203] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ff3d109f350bc92d epochNumber=1 {"provingJobId":"1:BASE_PARITY:ff3d109f350bc92d"} 16:00:48 [16:00:48.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a477849b217d14c2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:a477849b217d14c2"} 16:00:48 [16:00:48.204] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ff3d109f350bc92d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ff3d109f350bc92d"} 16:00:48 [16:00:48.204] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ff3d109f350bc92d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ff3d109f350bc92d"} 16:00:48 [16:00:48.204] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a477849b217d14c2 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a477849b217d14c2"} 16:00:48 [16:00:48.205] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.206] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:680b7c5e4555c9a0 err=test error {"provingJoId":"1:BASE_PARITY:680b7c5e4555c9a0"} 16:00:48 [16:00:48.207] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.207] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a9d0c6798a819f09 not found {"provingJobId":"1:BASE_PARITY:a9d0c6798a819f09"} 16:00:48 [16:00:48.208] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.209] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:706732257a839060 epochNumber=1 {"provingJobId":"1:BASE_PARITY:706732257a839060"} 16:00:48 [16:00:48.210] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.211] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:99ba2854dba3ed26 epochNumber=1 {"provingJobId":"1:BASE_PARITY:99ba2854dba3ed26"} 16:00:48 [16:00:48.313] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:99ba2854dba3ed26 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:99ba2854dba3ed26"} 16:00:48 [16:00:48.318] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.319] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3de823bf9f595733 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3de823bf9f595733"} 16:00:48 [16:00:48.444] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3de823bf9f595733 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:3de823bf9f595733"} 16:00:48 [16:00:48.447] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5464dbcb94be49ea epochNumber=10 {"provingJobId":"1:BASE_PARITY:5464dbcb94be49ea"} 16:00:48 [16:00:48.470] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:48 [16:00:48.575] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3634f4935e2f2bd9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3634f4935e2f2bd9"} 16:00:48 [16:00:48.729] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3634f4935e2f2bd9 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:3634f4935e2f2bd9"} 16:00:48 [16:00:48.780] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.781] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec3bc78c6d38c44b epochNumber=1 {"provingJobId":"1:BASE_PARITY:ec3bc78c6d38c44b"} 16:00:48 [16:00:48.782] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ec3bc78c6d38c44b type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:ec3bc78c6d38c44b"} 16:00:48 [16:00:48.783] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.783] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ae373fc0c8de9ae2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ae373fc0c8de9ae2"} 16:00:48 [16:00:48.784] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ae373fc0c8de9ae2 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:ae373fc0c8de9ae2"} 16:00:48 [16:00:48.785] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ae373fc0c8de9ae2 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:ae373fc0c8de9ae2"} 16:00:48 [16:00:48.786] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.787] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:be88fc72622fe327 epochNumber=1 {"provingJobId":"1:BASE_PARITY:be88fc72622fe327"} 16:00:48 [16:00:48.787] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:be88fc72622fe327 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:be88fc72622fe327"} 16:00:48 [16:00:48.788] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.789] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ed176a0739c2000 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ed176a0739c2000"} 16:00:48 [16:00:48.789] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2ed176a0739c2000 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:2ed176a0739c2000"} 16:00:48 [16:00:48.790] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fc7433497cfc5f13 epochNumber=3 {"provingJobId":"1:BASE_PARITY:fc7433497cfc5f13"} 16:00:48 [16:00:48.814] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:48 [16:00:48.816] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2ed176a0739c2000 err=test error {"provingJoId":"1:BASE_PARITY:2ed176a0739c2000"} 16:00:48 [16:00:48.818] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.818] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9c488f7989d49b45 settled=false {"provingJobId":"1:BASE_PARITY:9c488f7989d49b45","status":"pending"} 16:00:48 [16:00:48.818] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7b51c0dfd29f0af4 settled=false {"provingJobId":"1:BASE_PARITY:7b51c0dfd29f0af4","status":"pending"} 16:00:48 [16:00:48.820] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.820] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:efb346a15a3e2c32 settled=true {"provingJobId":"1:BASE_PARITY:efb346a15a3e2c32","status":"fulfilled"} 16:00:48 [16:00:48.820] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:148316919ad514d9 settled=true {"provingJobId":"2:BASE_PARITY:148316919ad514d9","status":"fulfilled"} 16:00:48 [16:00:48.822] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.822] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8bd65bb73334b919 settled=true {"provingJobId":"1:BASE_PARITY:8bd65bb73334b919","status":"fulfilled"} 16:00:48 [16:00:48.822] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:afc943dc36a7e058 settled=false {"provingJobId":"1:BASE_PARITY:afc943dc36a7e058","status":"pending"} 16:00:48 [16:00:48.823] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.824] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b70d55b7693e44e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b70d55b7693e44e8"} 16:00:48 [16:00:48.826] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.826] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf28f2d541598b8e epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf28f2d541598b8e"} 16:00:48 [16:00:48.827] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:cf28f2d541598b8e: Error: db error: Error: db error 16:00:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:cf28f2d541598b8e"} 16:00:48 [16:00:48.828] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.829] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:63fabaa516f79eaf epochNumber=1 {"provingJobId":"1:BASE_PARITY:63fabaa516f79eaf"} 16:00:48 [16:00:48.829] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:63fabaa516f79eaf type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:63fabaa516f79eaf"} 16:00:48 [16:00:48.829] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:63fabaa516f79eaf type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:63fabaa516f79eaf"} 16:00:48 [16:00:48.830] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.831] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ffdc485ac1827be8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ffdc485ac1827be8"} 16:00:48 [16:00:48.831] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ffdc485ac1827be8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ffdc485ac1827be8"} 16:00:48 [16:00:48.831] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ffdc485ac1827be8 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ffdc485ac1827be8"} 16:00:48 [16:00:48.832] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:ffdc485ac1827be8: Error: db error 16:00:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:ffdc485ac1827be8"} 16:00:48 [16:00:48.833] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.834] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:13cf338f3b907427 epochNumber=1 {"provingJobId":"1:BASE_PARITY:13cf338f3b907427"} 16:00:48 [16:00:48.834] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:13cf338f3b907427 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:13cf338f3b907427"} 16:00:48 [16:00:48.834] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:13cf338f3b907427 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:13cf338f3b907427"} 16:00:48 [16:00:48.835] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.836] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:491db607b07f1604 epochNumber=1 {"provingJobId":"1:BASE_PARITY:491db607b07f1604"} 16:00:48 [16:00:48.836] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:491db607b07f1604 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:491db607b07f1604"} 16:00:48 [16:00:48.836] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:491db607b07f1604 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:491db607b07f1604"} 16:00:48 [16:00:48.837] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:491db607b07f1604 jobErr=test error: Error: db error 16:00:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:491db607b07f1604"} 16:00:48 [16:00:48.837] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.838] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1c8514de33161cee not found {"provingJobId":"1:BASE_PARITY:1c8514de33161cee"} 16:00:48 [16:00:48.839] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.840] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:30e778ebc3876055 err=test error {"provingJoId":"1:BASE_PARITY:30e778ebc3876055"} 16:00:48 [16:00:48.841] INFO: prover-client:proving-broker Proving Broker started 16:00:48 [16:00:48.851] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4c4ad5f0faa7639 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4c4ad5f0faa7639"} 16:00:48 [16:00:48.851] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d4c4ad5f0faa7639 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d4c4ad5f0faa7639"} 16:00:48 [16:00:48.852] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d4c4ad5f0faa7639 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d4c4ad5f0faa7639"} 16:00:48 [16:00:48.862] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:359e81592692c06d epochNumber=2 {"provingJobId":"2:BASE_PARITY:359e81592692c06d"} 16:00:48 [16:00:48.862] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:359e81592692c06d type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:359e81592692c06d"} 16:00:48 [16:00:48.863] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:359e81592692c06d type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:359e81592692c06d"} 16:00:48 [16:00:48.924] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:a27812202a4c9a92 epochNumber=3 {"provingJobId":"3:BASE_PARITY:a27812202a4c9a92"} 16:00:48 [16:00:48.943] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:48 [16:00:48.986] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:05aae8f38828099a epochNumber=4 {"provingJobId":"4:BASE_PARITY:05aae8f38828099a"} 16:00:48 [16:00:48.995] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:49 [16:00:49.049] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:0518c2722e5489ef epochNumber=5 {"provingJobId":"5:BASE_PARITY:0518c2722e5489ef"} 16:00:49 [16:00:49.071] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:49 [16:00:49.104] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:34902d716684f9c1 epochNumber=42 {"provingJobId":"1:BASE_PARITY:34902d716684f9c1"} 16:00:49 [16:00:49.105] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testXf1yU5/42 with map size 1073741824 16:00:49 [16:00:49.110] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.131] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:3c08c8d63775a303 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c08c8d63775a303"} 16:00:49 [16:00:49.137] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.138] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eed552aed6279d7e epochNumber=1 {"provingJobId":"1:BASE_PARITY:eed552aed6279d7e"} 16:00:49 [16:00:49.139] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test80ZqDr/1 with map size 1073741824 16:00:49 [16:00:49.141] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.157] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1db2da7143095b17 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1db2da7143095b17"} 16:00:49 [16:00:49.165] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5c49e4332f30288e epochNumber=1 {"provingJobId":"1:BASE_PARITY:5c49e4332f30288e"} 16:00:49 [16:00:49.167] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test33A3Cf/1 with map size 1073741824 16:00:49 [16:00:49.169] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.189] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:5c49e4332f30288e epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:5c49e4332f30288e"} 16:00:49 [16:00:49.193] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.194] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:05e7aebe45973c40 epochNumber=1 {"provingJobId":"1:BASE_PARITY:05e7aebe45973c40"} 16:00:49 [16:00:49.195] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testp4meGa/1 with map size 1073741824 16:00:49 [16:00:49.198] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.215] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:05e7aebe45973c40 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:05e7aebe45973c40"} 16:00:49 [16:00:49.216] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:05e7aebe45973c40 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:05e7aebe45973c40"} 16:00:49 [16:00:49.222] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:05e7aebe45973c40 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:05e7aebe45973c40"} 16:00:49 [16:00:49.226] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.227] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fecdf54a5ebaac38 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fecdf54a5ebaac38"} 16:00:49 [16:00:49.228] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testP1Uqst/1 with map size 1073741824 16:00:49 [16:00:49.232] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.252] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.254] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.255] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:df300745b793783c epochNumber=1 {"provingJobId":"1:BASE_PARITY:df300745b793783c"} 16:00:49 [16:00:49.255] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testA8kZr4/1 with map size 1073741824 16:00:49 [16:00:49.257] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.275] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:df300745b793783c {"provingJobId":"1:BASE_PARITY:df300745b793783c"} 16:00:49 [16:00:49.276] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:df300745b793783c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:df300745b793783c"} 16:00:49 [16:00:49.276] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:df300745b793783c type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:df300745b793783c"} 16:00:49 [16:00:49.288] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.289] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8bb74e27b9a73e35 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8bb74e27b9a73e35"} 16:00:49 [16:00:49.290] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3HweP8/1 with map size 1073741824 16:00:49 [16:00:49.291] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.310] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:8bb74e27b9a73e35 {"provingJobId":"1:BASE_PARITY:8bb74e27b9a73e35"} 16:00:49 [16:00:49.311] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8bb74e27b9a73e35 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:8bb74e27b9a73e35"} 16:00:49 [16:00:49.322] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.322] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4394248b015cb862 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4394248b015cb862"} 16:00:49 [16:00:49.323] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjGJwOz/1 with map size 1073741824 16:00:49 [16:00:49.325] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.341] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4394248b015cb862 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4394248b015cb862"} 16:00:49 [16:00:49.342] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4394248b015cb862 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4394248b015cb862"} 16:00:49 [16:00:49.351] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.352] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd6d9b3499b457b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bd6d9b3499b457b5"} 16:00:49 [16:00:49.352] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testG7ckCv/1 with map size 1073741824 16:00:49 [16:00:49.354] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.370] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bd6d9b3499b457b5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bd6d9b3499b457b5"} 16:00:49 [16:00:49.370] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bd6d9b3499b457b5 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:bd6d9b3499b457b5"} 16:00:49 [16:00:49.379] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.380] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:831560e05be4820f epochNumber=0 {"provingJobId":"1:BASE_PARITY:831560e05be4820f"} 16:00:49 [16:00:49.381] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testpoC0Up/0 with map size 1073741824 16:00:49 [16:00:49.386] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.404] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:831560e05be4820f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:831560e05be4820f"} 16:00:49 [16:00:49.406] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:463e43180615c495 epochNumber=0 {"provingJobId":"1:BASE_PARITY:463e43180615c495"} 16:00:49 [16:00:49.407] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:831560e05be4820f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:831560e05be4820f"} 16:00:49 [16:00:49.408] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed82fda0d9c40c3f epochNumber=0 {"provingJobId":"1:BASE_PARITY:ed82fda0d9c40c3f"} 16:00:49 [16:00:49.424] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.426] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.427] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20570bfabd70cd6b epochNumber=2 {"provingJobId":"1:BASE_PARITY:20570bfabd70cd6b"} 16:00:49 [16:00:49.429] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLT6FSm/2 with map size 1073741824 16:00:49 [16:00:49.433] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.456] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab7a3a336101ca0b epochNumber=1 {"provingJobId":"1:BASE_PARITY:ab7a3a336101ca0b"} 16:00:49 [16:00:49.458] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLT6FSm/1 with map size 1073741824 16:00:49 [16:00:49.462] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.483] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.484] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a306756b8e9ef45 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a306756b8e9ef45"} 16:00:49 [16:00:49.485] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrijrmX/1 with map size 1073741824 16:00:49 [16:00:49.488] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.518] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.518] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c915d15cd1710538 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c915d15cd1710538"} 16:00:49 [16:00:49.524] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQJlRFN/1 with map size 1073741824 16:00:49 [16:00:49.529] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.547] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:934f9f5f2d84ed0c epochNumber=1 {"provingJobId":"1:BASE_PARITY:934f9f5f2d84ed0c"} 16:00:49 [16:00:49.554] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f64aba2a5814ea37 epochNumber=2 {"provingJobId":"1:BASE_PARITY:f64aba2a5814ea37"} 16:00:49 [16:00:49.555] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testQJlRFN/2 with map size 1073741824 16:00:49 [16:00:49.559] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0379f8fed6457cb5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0379f8fed6457cb5"} 16:00:49 [16:00:49.586] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.586] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe674ce3a831ecd6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fe674ce3a831ecd6"} 16:00:49 [16:00:49.588] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTWYRc9/1 with map size 1073741824 16:00:49 [16:00:49.590] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.607] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7ec1b4967a476e89 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7ec1b4967a476e89"} 16:00:49 [16:00:49.614] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cab9cdca08029bc7 epochNumber=2 {"provingJobId":"1:BASE_PARITY:cab9cdca08029bc7"} 16:00:49 [16:00:49.615] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTWYRc9/2 with map size 1073741824 16:00:49 [16:00:49.617] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.638] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:83e21f29c2529421 epochNumber=1 {"provingJobId":"1:BASE_PARITY:83e21f29c2529421"} 16:00:49 [16:00:49.649] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.650] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eed8fe1dd0b32334 epochNumber=1 {"provingJobId":"1:BASE_PARITY:eed8fe1dd0b32334"} 16:00:49 [16:00:49.652] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOak6Ts/1 with map size 1073741824 16:00:49 [16:00:49.655] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.674] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d5e894063bbb9b18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d5e894063bbb9b18"} 16:00:49 [16:00:49.685] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4b663325fd02e09 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d4b663325fd02e09"} 16:00:49 [16:00:49.686] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOak6Ts/2 with map size 1073741824 16:00:49 [16:00:49.688] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.708] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:53547b69abc7bd6b epochNumber=1 {"provingJobId":"1:BASE_PARITY:53547b69abc7bd6b"} 16:00:49 [16:00:49.719] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.719] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:91d58409dbde1fbf epochNumber=1 {"provingJobId":"1:BASE_PARITY:91d58409dbde1fbf"} 16:00:49 [16:00:49.720] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsutSnK/1 with map size 1073741824 16:00:49 [16:00:49.722] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.739] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fa4e1adf9f78b5f2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fa4e1adf9f78b5f2"} 16:00:49 [16:00:49.745] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:91d58409dbde1fbf type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:91d58409dbde1fbf"} 16:00:49 [16:00:49.756] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.757] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7bd289e4597faaa0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7bd289e4597faaa0"} 16:00:49 [16:00:49.757] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1r0fCa/1 with map size 1073741824 16:00:49 [16:00:49.759] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.779] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5a3dcb10c363d94c epochNumber=1 {"provingJobId":"1:BASE_PARITY:5a3dcb10c363d94c"} 16:00:49 [16:00:49.789] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7bd289e4597faaa0 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:7bd289e4597faaa0"} 16:00:49 [16:00:49.799] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.800] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:335d281619259df2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:335d281619259df2"} 16:00:49 [16:00:49.801] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testo5qSBn/1 with map size 1073741824 16:00:49 [16:00:49.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.822] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:879fa5bfeca94306 epochNumber=1 {"provingJobId":"1:BASE_PARITY:879fa5bfeca94306"} 16:00:49 [16:00:49.828] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:335d281619259df2 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:335d281619259df2"} 16:00:49 [16:00:49.832] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.833] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:860f9fc324ea393d epochNumber=1 {"provingJobId":"1:BASE_PARITY:860f9fc324ea393d"} 16:00:49 [16:00:49.833] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzMHI5s/1 with map size 1073741824 16:00:49 [16:00:49.835] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.852] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:860f9fc324ea393d {"provingJobId":"1:BASE_PARITY:860f9fc324ea393d"} 16:00:49 [16:00:49.853] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:860f9fc324ea393d type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:860f9fc324ea393d"} 16:00:49 [16:00:49.860] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5e4bfd330c309f64 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5e4bfd330c309f64"} 16:00:49 [16:00:49.866] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:860f9fc324ea393d has already been completed {"provingJobId":"1:BASE_PARITY:860f9fc324ea393d"} 16:00:49 [16:00:49.869] INFO: prover-client:proving-broker Proving Broker started 16:00:49 [16:00:49.870] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:93dbe1cdf22c61eb epochNumber=1 {"provingJobId":"1:BASE_PARITY:93dbe1cdf22c61eb"} 16:00:49 [16:00:49.871] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcETeOV/1 with map size 1073741824 16:00:49 [16:00:49.873] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:49 [16:00:49.893] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:21c1239f17f4264c epochNumber=2 {"provingJobId":"1:BASE_PARITY:21c1239f17f4264c"} 16:00:49 [16:00:49.894] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testcETeOV/2 with map size 1073741824 16:00:49 [16:00:49.896] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:50 [16:00:50.969] INFO: prover-client:proving-broker Proving Broker started 16:00:50 [16:00:50.976] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:93dbe1cdf22c61eb settled=false {"provingJobId":"1:BASE_PARITY:93dbe1cdf22c61eb","status":"pending"} 16:00:50 [16:00:50.978] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:21c1239f17f4264c settled=false {"provingJobId":"1:BASE_PARITY:21c1239f17f4264c","status":"pending"} 16:00:50 [16:00:50.980] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:93dbe1cdf22c61eb type=BASE_PARITY startedAt=1750694449917 older agent has taken job {"provingJobId":"1:BASE_PARITY:93dbe1cdf22c61eb"} 16:00:50 [16:00:50.980] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:93dbe1cdf22c61eb type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:93dbe1cdf22c61eb"} 16:00:50 [16:00:50.987] INFO: prover-client:proving-broker Proving Broker started 16:00:50 [16:00:50.988] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed9242e60463800e epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed9242e60463800e"} 16:00:50 [16:00:50.989] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCpI0v8/1 with map size 1073741824 16:00:50 [16:00:50.992] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:51 [16:00:51.015] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5ff6e0ee723fb0d4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:5ff6e0ee723fb0d4"} 16:00:51 [16:00:51.016] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCpI0v8/2 with map size 1073741824 16:00:51 [16:00:51.018] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:52 [16:00:52.041] INFO: prover-client:proving-broker Proving Broker started 16:00:52 [16:00:52.043] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ed9242e60463800e settled=false {"provingJobId":"1:BASE_PARITY:ed9242e60463800e","status":"pending"} 16:00:52 [16:00:52.045] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5ff6e0ee723fb0d4 settled=false {"provingJobId":"1:BASE_PARITY:5ff6e0ee723fb0d4","status":"pending"} 16:00:52 [16:00:52.045] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ed9242e60463800e type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:ed9242e60463800e"} 16:00:52 [16:00:52.051] INFO: prover-client:proving-broker Proving Broker started 16:00:52 [16:00:52.052] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c634c20ee9b4501f epochNumber=1 {"provingJobId":"1:BASE_PARITY:c634c20ee9b4501f"} 16:00:52 [16:00:52.053] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlHlOvu/1 with map size 1073741824 16:00:52 [16:00:52.055] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:52 [16:00:52.077] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8aa719ee6320b12b epochNumber=2 {"provingJobId":"1:BASE_PARITY:8aa719ee6320b12b"} 16:00:52 [16:00:52.078] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testlHlOvu/2 with map size 1073741824 16:00:52 [16:00:52.084] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.108] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.111] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c634c20ee9b4501f settled=false {"provingJobId":"1:BASE_PARITY:c634c20ee9b4501f","status":"pending"} 16:00:53 [16:00:53.112] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8aa719ee6320b12b settled=false {"provingJobId":"1:BASE_PARITY:8aa719ee6320b12b","status":"pending"} 16:00:53 [16:00:53.113] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c634c20ee9b4501f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c634c20ee9b4501f"} 16:00:53 [16:00:53.113] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c634c20ee9b4501f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c634c20ee9b4501f"} 16:00:53 [16:00:53.124] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.125] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0b889b9fee4a0e86 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0b889b9fee4a0e86"} 16:00:53 [16:00:53.125] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testm3cozK/1 with map size 1073741824 16:00:53 [16:00:53.128] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.147] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:72962404b8af9853 epochNumber=2 {"provingJobId":"1:BASE_PARITY:72962404b8af9853"} 16:00:53 [16:00:53.149] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testm3cozK/2 with map size 1073741824 16:00:53 [16:00:53.151] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.170] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0b889b9fee4a0e86 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:0b889b9fee4a0e86"} 16:00:53 [16:00:53.176] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:72962404b8af9853 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:72962404b8af9853"} 16:00:53 [16:00:53.191] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.192] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5bed4d61362d4206 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5bed4d61362d4206"} 16:00:53 [16:00:53.193] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYEO9Ry/1 with map size 1073741824 16:00:53 [16:00:53.195] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.212] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20cefa265e3780b5 epochNumber=2 {"provingJobId":"1:BASE_PARITY:20cefa265e3780b5"} 16:00:53 [16:00:53.213] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testYEO9Ry/2 with map size 1073741824 16:00:53 [16:00:53.217] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.233] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5bed4d61362d4206 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5bed4d61362d4206"} 16:00:53 [16:00:53.234] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5bed4d61362d4206 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5bed4d61362d4206"} 16:00:53 [16:00:53.240] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:20cefa265e3780b5 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:20cefa265e3780b5"} 16:00:53 [16:00:53.249] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.250] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e0e7c51628208105 err=test error {"provingJoId":"1:BASE_PARITY:e0e7c51628208105"} 16:00:53 [16:00:53.253] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.254] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:548a45363a2d9042 not found {"provingJobId":"1:BASE_PARITY:548a45363a2d9042"} 16:00:53 [16:00:53.256] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.257] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0d315567c45efa4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0d315567c45efa4"} 16:00:53 [16:00:53.257] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnfM1QH/1 with map size 1073741824 16:00:53 [16:00:53.260] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.282] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.283] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:329b0e9d6b32c93a epochNumber=1 {"provingJobId":"1:BASE_PARITY:329b0e9d6b32c93a"} 16:00:53 [16:00:53.283] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testarq7O0/1 with map size 1073741824 16:00:53 [16:00:53.286] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.409] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:329b0e9d6b32c93a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:329b0e9d6b32c93a"} 16:00:53 [16:00:53.417] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.418] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f4e07b9ba886f32 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1f4e07b9ba886f32"} 16:00:53 [16:00:53.418] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtNyzuO/1 with map size 1073741824 16:00:53 [16:00:53.424] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.547] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1f4e07b9ba886f32 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:1f4e07b9ba886f32"} 16:00:53 [16:00:53.550] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f0db49cdf7c9aaa6 epochNumber=10 {"provingJobId":"1:BASE_PARITY:f0db49cdf7c9aaa6"} 16:00:53 [16:00:53.551] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testtNyzuO/10 with map size 1073741824 16:00:53 [16:00:53.554] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.573] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:53 [16:00:53.574] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 16:00:53 [16:00:53.702] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.704] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:185462e56a602813 epochNumber=1 {"provingJobId":"1:BASE_PARITY:185462e56a602813"} 16:00:53 [16:00:53.706] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGq1BS0/1 with map size 1073741824 16:00:53 [16:00:53.709] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.883] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:185462e56a602813 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:185462e56a602813"} 16:00:53 [16:00:53.934] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.935] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a9bbf65c531a231c epochNumber=1 {"provingJobId":"1:BASE_PARITY:a9bbf65c531a231c"} 16:00:53 [16:00:53.935] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZGKJRJ/1 with map size 1073741824 16:00:53 [16:00:53.938] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.955] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a9bbf65c531a231c type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:a9bbf65c531a231c"} 16:00:53 [16:00:53.961] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.961] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67fe5441c7eaab13 epochNumber=1 {"provingJobId":"1:BASE_PARITY:67fe5441c7eaab13"} 16:00:53 [16:00:53.962] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7l8NCy/1 with map size 1073741824 16:00:53 [16:00:53.964] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:53 [16:00:53.980] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:67fe5441c7eaab13 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:67fe5441c7eaab13"} 16:00:53 [16:00:53.981] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:67fe5441c7eaab13 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:67fe5441c7eaab13"} 16:00:53 [16:00:53.990] INFO: prover-client:proving-broker Proving Broker started 16:00:53 [16:00:53.991] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:05d246da083e6600 epochNumber=1 {"provingJobId":"1:BASE_PARITY:05d246da083e6600"} 16:00:53 [16:00:53.992] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwqdH23/1 with map size 1073741824 16:00:53 [16:00:53.995] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.012] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:05d246da083e6600 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:05d246da083e6600"} 16:00:54 [16:00:54.022] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.023] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0755e1a4739f3452 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0755e1a4739f3452"} 16:00:54 [16:00:54.024] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtO1QSf/1 with map size 1073741824 16:00:54 [16:00:54.026] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.042] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0755e1a4739f3452 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:0755e1a4739f3452"} 16:00:54 [16:00:54.043] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06e98fe083b08323 epochNumber=3 {"provingJobId":"1:BASE_PARITY:06e98fe083b08323"} 16:00:54 [16:00:54.044] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testtO1QSf/3 with map size 1073741824 16:00:54 [16:00:54.046] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.072] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:54 [16:00:54.073] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 16:00:54 [16:00:54.087] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0755e1a4739f3452 err=test error {"provingJoId":"1:BASE_PARITY:0755e1a4739f3452"} 16:00:54 [16:00:54.093] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDanE5L/1 with map size 1073741824 16:00:54 [16:00:54.095] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.113] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testDanE5L/2 with map size 1073741824 16:00:54 [16:00:54.115] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.136] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.138] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4549be7664c95d3e settled=false {"provingJobId":"1:BASE_PARITY:4549be7664c95d3e","status":"pending"} 16:00:54 [16:00:54.139] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4d57393d36ffd6da settled=false {"provingJobId":"1:BASE_PARITY:4d57393d36ffd6da","status":"pending"} 16:00:54 [16:00:54.147] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBErN2H/1 with map size 1073741824 16:00:54 [16:00:54.150] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.167] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBErN2H/2 with map size 1073741824 16:00:54 [16:00:54.169] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.201] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.204] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:87d9dcb54d69784f settled=true {"provingJobId":"1:BASE_PARITY:87d9dcb54d69784f","status":"fulfilled"} 16:00:54 [16:00:54.209] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:e3cd60cc0cdf4efc settled=true {"provingJobId":"2:BASE_PARITY:e3cd60cc0cdf4efc","status":"fulfilled"} 16:00:54 [16:00:54.215] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8bZGLA/1 with map size 1073741824 16:00:54 [16:00:54.218] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.241] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8bZGLA/2 with map size 1073741824 16:00:54 [16:00:54.244] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.265] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.267] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3115f8ef1b1f12b5 settled=true {"provingJobId":"1:BASE_PARITY:3115f8ef1b1f12b5","status":"fulfilled"} 16:00:54 [16:00:54.269] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a4bcaa55e6a9407f settled=false {"provingJobId":"1:BASE_PARITY:a4bcaa55e6a9407f","status":"pending"} 16:00:54 [16:00:54.274] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.275] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:91046a53d10d92ca epochNumber=1 {"provingJobId":"1:BASE_PARITY:91046a53d10d92ca"} 16:00:54 [16:00:54.277] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testonPYfO/1 with map size 1073741824 16:00:54 [16:00:54.281] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.309] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.310] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:efa72b605604aebf epochNumber=1 {"provingJobId":"1:BASE_PARITY:efa72b605604aebf"} 16:00:54 [16:00:54.311] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:efa72b605604aebf: Error: db error: Error: db error 16:00:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:efa72b605604aebf"} 16:00:54 [16:00:54.318] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.319] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ccf27f1b687f005 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4ccf27f1b687f005"} 16:00:54 [16:00:54.321] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIrUPCi/1 with map size 1073741824 16:00:54 [16:00:54.327] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.349] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4ccf27f1b687f005 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4ccf27f1b687f005"} 16:00:54 [16:00:54.350] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4ccf27f1b687f005 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4ccf27f1b687f005"} 16:00:54 [16:00:54.360] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.361] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3dfe125d72c421a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3dfe125d72c421a4"} 16:00:54 [16:00:54.362] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxWwTDz/1 with map size 1073741824 16:00:54 [16:00:54.365] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.387] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3dfe125d72c421a4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3dfe125d72c421a4"} 16:00:54 [16:00:54.388] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3dfe125d72c421a4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3dfe125d72c421a4"} 16:00:54 [16:00:54.389] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:3dfe125d72c421a4: Error: db error 16:00:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:3dfe125d72c421a4"} 16:00:54 [16:00:54.395] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.396] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bfa3e40abbde158e epochNumber=1 {"provingJobId":"1:BASE_PARITY:bfa3e40abbde158e"} 16:00:54 [16:00:54.396] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMOfPlV/1 with map size 1073741824 16:00:54 [16:00:54.399] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.418] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bfa3e40abbde158e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bfa3e40abbde158e"} 16:00:54 [16:00:54.418] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bfa3e40abbde158e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:bfa3e40abbde158e"} 16:00:54 [16:00:54.430] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.431] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:85ec6941868fced8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:85ec6941868fced8"} 16:00:54 [16:00:54.432] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYFoOLi/1 with map size 1073741824 16:00:54 [16:00:54.435] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.456] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:85ec6941868fced8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:85ec6941868fced8"} 16:00:54 [16:00:54.457] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:85ec6941868fced8 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:85ec6941868fced8"} 16:00:54 [16:00:54.457] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:85ec6941868fced8 jobErr=test error: Error: db error 16:00:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:85ec6941868fced8"} 16:00:54 [16:00:54.462] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.463] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7034cfce5be0cb09 not found {"provingJobId":"1:BASE_PARITY:7034cfce5be0cb09"} 16:00:54 [16:00:54.466] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.467] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e5ca3d28dcfef05e err=test error {"provingJoId":"1:BASE_PARITY:e5ca3d28dcfef05e"} 16:00:54 [16:00:54.470] INFO: prover-client:proving-broker Proving Broker started 16:00:54 [16:00:54.480] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab73dab0ca3a4da4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ab73dab0ca3a4da4"} 16:00:54 [16:00:54.481] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzmNUxi/1 with map size 1073741824 16:00:54 [16:00:54.484] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.503] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ab73dab0ca3a4da4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ab73dab0ca3a4da4"} 16:00:54 [16:00:54.503] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ab73dab0ca3a4da4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ab73dab0ca3a4da4"} 16:00:54 [16:00:54.520] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:d1561001423a18e1 epochNumber=2 {"provingJobId":"2:BASE_PARITY:d1561001423a18e1"} 16:00:54 [16:00:54.521] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzmNUxi/2 with map size 1073741824 16:00:54 [16:00:54.526] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.547] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:d1561001423a18e1 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:d1561001423a18e1"} 16:00:54 [16:00:54.548] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:d1561001423a18e1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:d1561001423a18e1"} 16:00:54 [16:00:54.618] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:300b76571e439c65 epochNumber=3 {"provingJobId":"3:BASE_PARITY:300b76571e439c65"} 16:00:54 [16:00:54.619] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testzmNUxi/3 with map size 1073741824 16:00:54 [16:00:54.622] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.646] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:54 [16:00:54.647] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 16:00:54 [16:00:54.704] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:638910c65ba68fb5 epochNumber=4 {"provingJobId":"4:BASE_PARITY:638910c65ba68fb5"} 16:00:54 [16:00:54.706] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testzmNUxi/4 with map size 1073741824 16:00:54 [16:00:54.710] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.753] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:54 [16:00:54.753] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 16:00:54 [16:00:54.794] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:532e292f87f1d254 epochNumber=5 {"provingJobId":"5:BASE_PARITY:532e292f87f1d254"} 16:00:54 [16:00:54.796] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testzmNUxi/5 with map size 1073741824 16:00:54 [16:00:54.802] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 16:00:54 [16:00:54.830] INFO: prover-client:proving-broker Cleaned up jobs=1 16:00:54 [16:00:54.831] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 16:00:54 PASS src/proving_broker/proving_broker.test.ts (13.371 s) 16:00:54 ProvingBroker 16:00:54 Producer API 16:00:54 ✓ refuses stale jobs (62 ms) 16:00:54 ✓ enqueues jobs (9 ms) 16:00:54 ✓ ignores duplicate jobs (7 ms) 16:00:54 ✓ reports correct status when enqueuing repeat jobs (5 ms) 16:00:54 ✓ throws an error in case of duplicate job IDs (29 ms) 16:00:54 ✓ returns not-found status for non-existing jobs (2 ms) 16:00:54 ✓ cancels jobs in queue (5 ms) 16:00:54 ✓ cancels jobs in-progress (4 ms) 16:00:54 ✓ returns job result if successful (3 ms) 16:00:54 ✓ returns job error if failed (2 ms) 16:00:54 ✓ correctly returns job status for concurrent writes (3 ms) 16:00:54 ✓ refuses stale jobs (34 ms) 16:00:54 ✓ enqueues jobs (28 ms) 16:00:54 ✓ ignores duplicate jobs (28 ms) 16:00:54 ✓ reports correct status when enqueuing repeat jobs (33 ms) 16:00:54 ✓ throws an error in case of duplicate job IDs (26 ms) 16:00:54 ✓ returns not-found status for non-existing jobs (2 ms) 16:00:54 ✓ cancels jobs in queue (33 ms) 16:00:54 ✓ cancels jobs in-progress (31 ms) 16:00:54 ✓ returns job result if successful (31 ms) 16:00:54 ✓ returns job error if failed (27 ms) 16:00:54 ✓ correctly returns job status for concurrent writes (45 ms) 16:00:54 Consumer API 16:00:54 ✓ returns undefined if no jobs are available (12 ms) 16:00:54 ✓ returns jobs in priority order (17 ms) 16:00:54 ✓ returns undefined if no jobs are available for the given allowList (2 ms) 16:00:54 ✓ returns a job if it is in the allowList (2 ms) 16:00:54 ✓ returns the most important job if it is in the allowList (3 ms) 16:00:54 ✓ returns any job if filter is empty (2 ms) 16:00:54 ✓ returns a new job when reporting job success (3 ms) 16:00:54 ✓ returns a new job when reporting permanent error (2 ms) 16:00:54 ✓ returns a new job when reporting retry-able error (2 ms) 16:00:54 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 16:00:54 ✓ returns a new job if job is already in progress elsewhere (1057 ms) 16:00:54 ✓ avoids sending the same job to a new agent after a restart (1007 ms) 16:00:54 ✓ avoids sending a completed job to a new agent after a restart (1010 ms) 16:00:54 ✓ tracks job result if in progress (3 ms) 16:00:54 ✓ tracks job result even if job is in queue (3 ms) 16:00:54 ✓ ignores reported job error if unknown job (1 ms) 16:00:54 ✓ ignores job result if unknown job (2 ms) 16:00:54 ✓ returns undefined if no jobs are available (3 ms) 16:00:54 ✓ returns jobs in priority order (56 ms) 16:00:54 ✓ returns undefined if no jobs are available for the given allowList (33 ms) 16:00:54 ✓ returns a job if it is in the allowList (69 ms) 16:00:54 ✓ returns the most important job if it is in the allowList (62 ms) 16:00:54 ✓ returns any job if filter is empty (69 ms) 16:00:54 ✓ returns a new job when reporting job success (35 ms) 16:00:54 ✓ returns a new job when reporting permanent error (45 ms) 16:00:54 ✓ returns a new job when reporting retry-able error (33 ms) 16:00:54 ✓ returns a new job when reporting progress if current one is cancelled (38 ms) 16:00:54 ✓ returns a new job if job is already in progress elsewhere (1115 ms) 16:00:54 ✓ avoids sending the same job to a new agent after a restart (1064 ms) 16:00:54 ✓ avoids sending a completed job to a new agent after a restart (1075 ms) 16:00:54 ✓ tracks job result if in progress (66 ms) 16:00:54 ✓ tracks job result even if job is in queue (58 ms) 16:00:54 ✓ ignores reported job error if unknown job (4 ms) 16:00:54 ✓ ignores job result if unknown job (3 ms) 16:00:54 Timeouts 16:00:54 ✓ tracks in progress jobs (2 ms) 16:00:54 ✓ re-enqueues jobs that time out (107 ms) 16:00:54 ✓ cancel stale jobs that time out (256 ms) 16:00:54 ✓ keeps the jobs in progress while it is alive (204 ms) 16:00:54 ✓ tracks in progress jobs (24 ms) 16:00:54 ✓ re-enqueues jobs that time out (136 ms) 16:00:54 ✓ cancel stale jobs that time out (282 ms) 16:00:54 ✓ keeps the jobs in progress while it is alive (233 ms) 16:00:54 Retries 16:00:54 ✓ retries jobs (3 ms) 16:00:54 ✓ retries up to a maximum number of times (2 ms) 16:00:54 ✓ passing retry=false does not retry (1 ms) 16:00:54 ✓ does not retry if job is stale (28 ms) 16:00:54 ✓ retries jobs (25 ms) 16:00:54 ✓ retries up to a maximum number of times (30 ms) 16:00:54 ✓ passing retry=false does not retry (31 ms) 16:00:54 ✓ does not retry if job is stale (69 ms) 16:00:54 Database management 16:00:54 ✓ re-enqueues proof requests on start (2 ms) 16:00:54 ✓ restores proof results on start (2 ms) 16:00:54 ✓ only re-enqueues unfinished jobs (2 ms) 16:00:54 ✓ saves job when enqueued (2 ms) 16:00:54 ✓ does not retain job if database fails to save (2 ms) 16:00:54 ✓ saves job result (2 ms) 16:00:54 ✓ saves result even if database fails to save (2 ms) 16:00:54 ✓ saves job error (2 ms) 16:00:54 ✓ saves job error even if database fails to save (2 ms) 16:00:54 ✓ does not save job result if job is unknown (2 ms) 16:00:54 ✓ does not save job error if job is unknown (1 ms) 16:00:54 ✓ cleans up old jobs periodically (261 ms) 16:00:54 ✓ re-enqueues proof requests on start (54 ms) 16:00:54 ✓ restores proof results on start (67 ms) 16:00:54 ✓ only re-enqueues unfinished jobs (60 ms) 16:00:54 ✓ saves job when enqueued (31 ms) 16:00:54 ✓ does not retain job if database fails to save (9 ms) 16:00:54 ✓ saves job result (45 ms) 16:00:54 ✓ saves result even if database fails to save (33 ms) 16:00:54 ✓ saves job error (34 ms) 16:00:54 ✓ saves job error even if database fails to save (33 ms) 16:00:54 ✓ does not save job result if job is unknown (3 ms) 16:00:54 ✓ does not save job error if job is unknown (4 ms) 16:00:54 ✓ cleans up old jobs periodically (407 ms) 16:00:54 16:00:54 Test Suites: 1 passed, 1 total 16:00:54 Tests: 96 passed, 96 total 16:00:54 Snapshots: 0 total 16:00:54 Time: 13.47 s 16:00:54 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 16:00:54 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?