Parent Log: http://ci.aztec-labs.com/b1d670ca3d7d471b Command: c0eccb395b706d67 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/92d685c3a802eb31897b6f13d28b9ee428c73744 Env: REF_NAME=gh-readonly-queue/next/pr-15093-4e3f550332c81006b8d3cdbeb646f4c567473f9d CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Thu Jun 19 20:10:53 UTC 2025 System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15093_arm64_a1-fast Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 20:10:56 [20:10:56.401] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.407] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:01c2b224209ef03a epochNumber=42 {"provingJobId":"1:BASE_PARITY:01c2b224209ef03a"} 20:10:56 [20:10:56.410] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:ef10753a9f3bb6af epochNumber=1 {"provingJobId":"1:BASE_PARITY:ef10753a9f3bb6af"} 20:10:56 [20:10:56.434] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.435] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a577857236455d23 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a577857236455d23"} 20:10:56 [20:10:56.436] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:65d7ad9bb9f5ca34 epochNumber=1 {"provingJobId":"1:BASE_PARITY:65d7ad9bb9f5ca34"} 20:10:56 [20:10:56.438] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.439] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a07dbdfe9bef974a epochNumber=1 {"provingJobId":"1:BASE_PARITY:a07dbdfe9bef974a"} 20:10:56 [20:10:56.440] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a07dbdfe9bef974a epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:a07dbdfe9bef974a"} 20:10:56 [20:10:56.442] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.443] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:675c9351150e3931 epochNumber=1 {"provingJobId":"1:BASE_PARITY:675c9351150e3931"} 20:10:56 [20:10:56.444] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:675c9351150e3931 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:675c9351150e3931"} 20:10:56 [20:10:56.444] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:675c9351150e3931 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:675c9351150e3931"} 20:10:56 [20:10:56.445] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:675c9351150e3931 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:675c9351150e3931"} 20:10:56 [20:10:56.446] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.447] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08e3e9d2b80acd25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:08e3e9d2b80acd25"} 20:10:56 [20:10:56.465] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.466] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.467] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:867e6bd54173248f epochNumber=1 {"provingJobId":"1:BASE_PARITY:867e6bd54173248f"} 20:10:56 [20:10:56.467] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:867e6bd54173248f {"provingJobId":"1:BASE_PARITY:867e6bd54173248f"} 20:10:56 [20:10:56.468] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:867e6bd54173248f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:867e6bd54173248f"} 20:10:56 [20:10:56.468] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:867e6bd54173248f type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:867e6bd54173248f"} 20:10:56 [20:10:56.469] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.470] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c6e172ac71acca0e epochNumber=1 {"provingJobId":"1:BASE_PARITY:c6e172ac71acca0e"} 20:10:56 [20:10:56.470] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c6e172ac71acca0e {"provingJobId":"1:BASE_PARITY:c6e172ac71acca0e"} 20:10:56 [20:10:56.471] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c6e172ac71acca0e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:c6e172ac71acca0e"} 20:10:56 [20:10:56.472] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.473] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf30a2384af0470d epochNumber=1 {"provingJobId":"1:BASE_PARITY:bf30a2384af0470d"} 20:10:56 [20:10:56.473] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bf30a2384af0470d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bf30a2384af0470d"} 20:10:56 [20:10:56.474] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bf30a2384af0470d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:bf30a2384af0470d"} 20:10:56 [20:10:56.475] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.476] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf2ebd78c42d5fcd epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf2ebd78c42d5fcd"} 20:10:56 [20:10:56.476] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cf2ebd78c42d5fcd type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:cf2ebd78c42d5fcd"} 20:10:56 [20:10:56.476] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cf2ebd78c42d5fcd type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:cf2ebd78c42d5fcd"} 20:10:56 [20:10:56.477] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.478] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d9927e7ef68346a9 epochNumber=0 {"provingJobId":"1:BASE_PARITY:d9927e7ef68346a9"} 20:10:56 [20:10:56.478] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d9927e7ef68346a9 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d9927e7ef68346a9"} 20:10:56 [20:10:56.478] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:48e2cfb1b2fc452f epochNumber=0 {"provingJobId":"1:BASE_PARITY:48e2cfb1b2fc452f"} 20:10:56 [20:10:56.479] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d9927e7ef68346a9 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:d9927e7ef68346a9"} 20:10:56 [20:10:56.479] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f01bcc4242e9d041 epochNumber=0 {"provingJobId":"1:BASE_PARITY:f01bcc4242e9d041"} 20:10:56 [20:10:56.480] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.481] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.481] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2658c5cbee5bb877 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2658c5cbee5bb877"} 20:10:56 [20:10:56.482] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bd15f3d7228cf9e6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bd15f3d7228cf9e6"} 20:10:56 [20:10:56.483] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.483] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:290d47c1f7729680 epochNumber=1 {"provingJobId":"1:BASE_PARITY:290d47c1f7729680"} 20:10:56 [20:10:56.488] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.488] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61b8ac3c284fba1c epochNumber=1 {"provingJobId":"1:BASE_PARITY:61b8ac3c284fba1c"} 20:10:56 [20:10:56.489] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e457a259484132fc epochNumber=1 {"provingJobId":"1:BASE_PARITY:e457a259484132fc"} 20:10:56 [20:10:56.489] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:828300af61407453 epochNumber=2 {"provingJobId":"1:BASE_PARITY:828300af61407453"} 20:10:56 [20:10:56.489] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f43237d17e03951 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1f43237d17e03951"} 20:10:56 [20:10:56.490] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.491] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0ba336d0aca2bf37 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0ba336d0aca2bf37"} 20:10:56 [20:10:56.491] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cc2ade44cad27b93 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cc2ade44cad27b93"} 20:10:56 [20:10:56.491] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:acf367c7365f9a52 epochNumber=2 {"provingJobId":"1:BASE_PARITY:acf367c7365f9a52"} 20:10:56 [20:10:56.492] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a7a9ecbce1978baf epochNumber=1 {"provingJobId":"1:BASE_PARITY:a7a9ecbce1978baf"} 20:10:56 [20:10:56.493] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.493] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ad6ff3010de6bc83 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ad6ff3010de6bc83"} 20:10:56 [20:10:56.493] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:927e68cf8ec7a623 epochNumber=1 {"provingJobId":"1:BASE_PARITY:927e68cf8ec7a623"} 20:10:56 [20:10:56.493] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e57bf05f54c8aed2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e57bf05f54c8aed2"} 20:10:56 [20:10:56.494] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:91a5634ebc22bc2e epochNumber=1 {"provingJobId":"1:BASE_PARITY:91a5634ebc22bc2e"} 20:10:56 [20:10:56.495] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.495] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:69f6f1de2a5209fd epochNumber=1 {"provingJobId":"1:BASE_PARITY:69f6f1de2a5209fd"} 20:10:56 [20:10:56.496] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a43e3d88d496664a epochNumber=1 {"provingJobId":"1:BASE_PARITY:a43e3d88d496664a"} 20:10:56 [20:10:56.496] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:69f6f1de2a5209fd type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:69f6f1de2a5209fd"} 20:10:56 [20:10:56.497] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.497] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d36831bee7f87b21 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d36831bee7f87b21"} 20:10:56 [20:10:56.498] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f43cf8983bb2e9ff epochNumber=1 {"provingJobId":"1:BASE_PARITY:f43cf8983bb2e9ff"} 20:10:56 [20:10:56.498] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d36831bee7f87b21 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:d36831bee7f87b21"} 20:10:56 [20:10:56.499] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.499] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e84a7e575761bc17 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e84a7e575761bc17"} 20:10:56 [20:10:56.500] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9ab74211c4f78dba epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ab74211c4f78dba"} 20:10:56 [20:10:56.500] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e84a7e575761bc17 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:e84a7e575761bc17"} 20:10:56 [20:10:56.501] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.501] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aec5ca2d49503a47 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aec5ca2d49503a47"} 20:10:56 [20:10:56.501] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:aec5ca2d49503a47 {"provingJobId":"1:BASE_PARITY:aec5ca2d49503a47"} 20:10:56 [20:10:56.502] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:aec5ca2d49503a47 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:aec5ca2d49503a47"} 20:10:56 [20:10:56.502] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:681d0cac59c5a96e epochNumber=1 {"provingJobId":"1:BASE_PARITY:681d0cac59c5a96e"} 20:10:56 [20:10:56.502] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:aec5ca2d49503a47 has already been completed {"provingJobId":"1:BASE_PARITY:aec5ca2d49503a47"} 20:10:56 [20:10:56.503] INFO: prover-client:proving-broker Proving Broker started 20:10:56 [20:10:56.504] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a967d15997ada0a9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a967d15997ada0a9"} 20:10:56 [20:10:56.504] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f0a46a4f54c84f66 epochNumber=2 {"provingJobId":"1:BASE_PARITY:f0a46a4f54c84f66"} 20:10:57 [20:10:57.555] INFO: prover-client:proving-broker Proving Broker started 20:10:57 [20:10:57.556] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a967d15997ada0a9 settled=false {"provingJobId":"1:BASE_PARITY:a967d15997ada0a9","status":"pending"} 20:10:57 [20:10:57.556] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f0a46a4f54c84f66 settled=false {"provingJobId":"1:BASE_PARITY:f0a46a4f54c84f66","status":"pending"} 20:10:57 [20:10:57.557] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:a967d15997ada0a9 type=BASE_PARITY startedAt=1750363856504 older agent has taken job {"provingJobId":"1:BASE_PARITY:a967d15997ada0a9"} 20:10:57 [20:10:57.557] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a967d15997ada0a9 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:a967d15997ada0a9"} 20:10:57 [20:10:57.558] INFO: prover-client:proving-broker Proving Broker started 20:10:57 [20:10:57.558] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:575bd0c6a2721b97 epochNumber=1 {"provingJobId":"1:BASE_PARITY:575bd0c6a2721b97"} 20:10:57 [20:10:57.559] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:da67674f102ba7e2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:da67674f102ba7e2"} 20:10:58 [20:10:58.559] INFO: prover-client:proving-broker Proving Broker started 20:10:58 [20:10:58.559] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:575bd0c6a2721b97 settled=false {"provingJobId":"1:BASE_PARITY:575bd0c6a2721b97","status":"pending"} 20:10:58 [20:10:58.560] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:da67674f102ba7e2 settled=false {"provingJobId":"1:BASE_PARITY:da67674f102ba7e2","status":"pending"} 20:10:58 [20:10:58.560] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:575bd0c6a2721b97 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:575bd0c6a2721b97"} 20:10:58 [20:10:58.561] INFO: prover-client:proving-broker Proving Broker started 20:10:58 [20:10:58.562] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a76977d668b8a9e5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a76977d668b8a9e5"} 20:10:58 [20:10:58.562] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:27988fba50156f2c epochNumber=2 {"provingJobId":"1:BASE_PARITY:27988fba50156f2c"} 20:10:59 [20:10:59.563] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.563] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a76977d668b8a9e5 settled=false {"provingJobId":"1:BASE_PARITY:a76977d668b8a9e5","status":"pending"} 20:10:59 [20:10:59.564] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:27988fba50156f2c settled=false {"provingJobId":"1:BASE_PARITY:27988fba50156f2c","status":"pending"} 20:10:59 [20:10:59.564] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a76977d668b8a9e5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a76977d668b8a9e5"} 20:10:59 [20:10:59.564] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a76977d668b8a9e5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a76977d668b8a9e5"} 20:10:59 [20:10:59.565] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.566] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:187e6b3903d7191d epochNumber=1 {"provingJobId":"1:BASE_PARITY:187e6b3903d7191d"} 20:10:59 [20:10:59.566] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fbcb23644b384d97 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fbcb23644b384d97"} 20:10:59 [20:10:59.567] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:187e6b3903d7191d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:187e6b3903d7191d"} 20:10:59 [20:10:59.567] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fbcb23644b384d97 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:fbcb23644b384d97"} 20:10:59 [20:10:59.568] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.568] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:661d0242177f1431 epochNumber=1 {"provingJobId":"1:BASE_PARITY:661d0242177f1431"} 20:10:59 [20:10:59.569] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:39d534c1ff80a5a2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:39d534c1ff80a5a2"} 20:10:59 [20:10:59.569] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:661d0242177f1431 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:661d0242177f1431"} 20:10:59 [20:10:59.569] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:661d0242177f1431 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:661d0242177f1431"} 20:10:59 [20:10:59.570] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:39d534c1ff80a5a2 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:39d534c1ff80a5a2"} 20:10:59 [20:10:59.570] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.571] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:5b73e89b752122bd err=test error {"provingJoId":"1:BASE_PARITY:5b73e89b752122bd"} 20:10:59 [20:10:59.572] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.572] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9f3022ec083740fd not found {"provingJobId":"1:BASE_PARITY:9f3022ec083740fd"} 20:10:59 [20:10:59.573] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.574] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0728c8905cdd36d epochNumber=1 {"provingJobId":"1:BASE_PARITY:b0728c8905cdd36d"} 20:10:59 [20:10:59.575] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.575] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a57cfd55ee04d834 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a57cfd55ee04d834"} 20:10:59 [20:10:59.676] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a57cfd55ee04d834 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:a57cfd55ee04d834"} 20:10:59 [20:10:59.684] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.684] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3196d80ddd6994c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3196d80ddd6994c8"} 20:10:59 [20:10:59.785] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3196d80ddd6994c8 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:3196d80ddd6994c8"} 20:10:59 [20:10:59.786] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bc7c3d72312430db epochNumber=10 {"provingJobId":"1:BASE_PARITY:bc7c3d72312430db"} 20:10:59 [20:10:59.811] INFO: prover-client:proving-broker Cleaned up jobs=1 20:10:59 [20:10:59.913] INFO: prover-client:proving-broker Proving Broker started 20:10:59 [20:10:59.914] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:10ce2e6631a8f7e1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:10ce2e6631a8f7e1"} 20:11:00 [20:11:00.065] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:10ce2e6631a8f7e1 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:10ce2e6631a8f7e1"} 20:11:00 [20:11:00.116] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.117] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c8c8a0c9a67972c epochNumber=1 {"provingJobId":"1:BASE_PARITY:1c8c8a0c9a67972c"} 20:11:00 [20:11:00.118] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:1c8c8a0c9a67972c type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:1c8c8a0c9a67972c"} 20:11:00 [20:11:00.119] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c47ddbed02f3b75 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1c47ddbed02f3b75"} 20:11:00 [20:11:00.120] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:1c47ddbed02f3b75 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:1c47ddbed02f3b75"} 20:11:00 [20:11:00.120] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1c47ddbed02f3b75 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:1c47ddbed02f3b75"} 20:11:00 [20:11:00.121] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.121] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f7214149fd2d53c epochNumber=1 {"provingJobId":"1:BASE_PARITY:4f7214149fd2d53c"} 20:11:00 [20:11:00.122] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4f7214149fd2d53c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4f7214149fd2d53c"} 20:11:00 [20:11:00.123] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.123] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e32a270d0f1c68a5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e32a270d0f1c68a5"} 20:11:00 [20:11:00.123] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e32a270d0f1c68a5 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:e32a270d0f1c68a5"} 20:11:00 [20:11:00.124] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb713d47a5f4915b epochNumber=3 {"provingJobId":"1:BASE_PARITY:eb713d47a5f4915b"} 20:11:00 [20:11:00.148] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:00 [20:11:00.149] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e32a270d0f1c68a5 err=test error {"provingJoId":"1:BASE_PARITY:e32a270d0f1c68a5"} 20:11:00 [20:11:00.151] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.151] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e617631953fe7dfb settled=false {"provingJobId":"1:BASE_PARITY:e617631953fe7dfb","status":"pending"} 20:11:00 [20:11:00.151] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:56f0d9c66b7f7c3e settled=false {"provingJobId":"1:BASE_PARITY:56f0d9c66b7f7c3e","status":"pending"} 20:11:00 [20:11:00.153] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.154] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0430d8b7ca813efb settled=true {"provingJobId":"1:BASE_PARITY:0430d8b7ca813efb","status":"fulfilled"} 20:11:00 [20:11:00.154] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:e5eb9ed97aefee20 settled=true {"provingJobId":"2:BASE_PARITY:e5eb9ed97aefee20","status":"fulfilled"} 20:11:00 [20:11:00.155] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.155] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:689af3a446baebdb settled=true {"provingJobId":"1:BASE_PARITY:689af3a446baebdb","status":"fulfilled"} 20:11:00 [20:11:00.156] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a76fb12d5ce21c4d settled=false {"provingJobId":"1:BASE_PARITY:a76fb12d5ce21c4d","status":"pending"} 20:11:00 [20:11:00.157] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.158] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4643d38f4de6593 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4643d38f4de6593"} 20:11:00 [20:11:00.159] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.160] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c472300da89c8b8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c472300da89c8b8"} 20:11:00 [20:11:00.160] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:3c472300da89c8b8: Error: db error: Error: db error 20:11:00 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:3c472300da89c8b8"} 20:11:00 [20:11:00.162] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.162] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4f4791dbacaeb77 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4f4791dbacaeb77"} 20:11:00 [20:11:00.162] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c4f4791dbacaeb77 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c4f4791dbacaeb77"} 20:11:00 [20:11:00.163] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c4f4791dbacaeb77 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c4f4791dbacaeb77"} 20:11:00 [20:11:00.164] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.164] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9fba21b4ba4bbbd6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9fba21b4ba4bbbd6"} 20:11:00 [20:11:00.164] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9fba21b4ba4bbbd6 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9fba21b4ba4bbbd6"} 20:11:00 [20:11:00.165] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9fba21b4ba4bbbd6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9fba21b4ba4bbbd6"} 20:11:00 [20:11:00.165] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:9fba21b4ba4bbbd6: Error: db error 20:11:00 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:9fba21b4ba4bbbd6"} 20:11:00 [20:11:00.166] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.167] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b2efe43fdea0bf0c epochNumber=1 {"provingJobId":"1:BASE_PARITY:b2efe43fdea0bf0c"} 20:11:00 [20:11:00.167] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b2efe43fdea0bf0c type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b2efe43fdea0bf0c"} 20:11:00 [20:11:00.167] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b2efe43fdea0bf0c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:b2efe43fdea0bf0c"} 20:11:00 [20:11:00.168] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.169] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ecb0059654d20009 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ecb0059654d20009"} 20:11:00 [20:11:00.169] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ecb0059654d20009 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ecb0059654d20009"} 20:11:00 [20:11:00.169] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ecb0059654d20009 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ecb0059654d20009"} 20:11:00 [20:11:00.170] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:ecb0059654d20009 jobErr=test error: Error: db error 20:11:00 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:ecb0059654d20009"} 20:11:00 [20:11:00.173] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.173] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b172572d6495660d not found {"provingJobId":"1:BASE_PARITY:b172572d6495660d"} 20:11:00 [20:11:00.175] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.175] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0299d0890cd6e94a err=test error {"provingJoId":"1:BASE_PARITY:0299d0890cd6e94a"} 20:11:00 [20:11:00.176] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:095f62c6993546cc epochNumber=1 {"provingJobId":"1:BASE_PARITY:095f62c6993546cc"} 20:11:00 [20:11:00.187] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:095f62c6993546cc type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:095f62c6993546cc"} 20:11:00 [20:11:00.187] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:095f62c6993546cc type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:095f62c6993546cc"} 20:11:00 [20:11:00.198] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:426961f122a0d925 epochNumber=2 {"provingJobId":"2:BASE_PARITY:426961f122a0d925"} 20:11:00 [20:11:00.198] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:426961f122a0d925 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:426961f122a0d925"} 20:11:00 [20:11:00.198] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:426961f122a0d925 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:426961f122a0d925"} 20:11:00 [20:11:00.258] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:74ef5418874bda7c epochNumber=3 {"provingJobId":"3:BASE_PARITY:74ef5418874bda7c"} 20:11:00 [20:11:00.279] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:00 [20:11:00.320] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:d27005538af1d2df epochNumber=4 {"provingJobId":"4:BASE_PARITY:d27005538af1d2df"} 20:11:00 [20:11:00.330] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:00 [20:11:00.381] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:a099cfb01a10d434 epochNumber=5 {"provingJobId":"5:BASE_PARITY:a099cfb01a10d434"} 20:11:00 [20:11:00.407] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:00 [20:11:00.434] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.435] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4b18085efb984708 epochNumber=42 {"provingJobId":"1:BASE_PARITY:4b18085efb984708"} 20:11:00 [20:11:00.435] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testRkJheC/42 with map size 1073741824 20:11:00 [20:11:00.438] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.458] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:4a79b13ebf3598bd epochNumber=1 {"provingJobId":"1:BASE_PARITY:4a79b13ebf3598bd"} 20:11:00 [20:11:00.462] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.462] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:133533536d13519f epochNumber=1 {"provingJobId":"1:BASE_PARITY:133533536d13519f"} 20:11:00 [20:11:00.463] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpjBCbD/1 with map size 1073741824 20:11:00 [20:11:00.464] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.482] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c975536636042a52 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c975536636042a52"} 20:11:00 [20:11:00.492] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.492] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:49aa16530ad87dbc epochNumber=1 {"provingJobId":"1:BASE_PARITY:49aa16530ad87dbc"} 20:11:00 [20:11:00.493] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testS6HIob/1 with map size 1073741824 20:11:00 [20:11:00.494] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.511] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:49aa16530ad87dbc epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:49aa16530ad87dbc"} 20:11:00 [20:11:00.513] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.514] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0eba76dac942149 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b0eba76dac942149"} 20:11:00 [20:11:00.514] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9uzmiB/1 with map size 1073741824 20:11:00 [20:11:00.515] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.531] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b0eba76dac942149 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b0eba76dac942149"} 20:11:00 [20:11:00.531] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b0eba76dac942149 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b0eba76dac942149"} 20:11:00 [20:11:00.536] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b0eba76dac942149 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:b0eba76dac942149"} 20:11:00 [20:11:00.539] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.540] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:832a404dc9ab585c epochNumber=1 {"provingJobId":"1:BASE_PARITY:832a404dc9ab585c"} 20:11:00 [20:11:00.540] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAQLiFd/1 with map size 1073741824 20:11:00 [20:11:00.541] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.559] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.561] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.562] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:066501b590e905de epochNumber=1 {"provingJobId":"1:BASE_PARITY:066501b590e905de"} 20:11:00 [20:11:00.562] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD1L64K/1 with map size 1073741824 20:11:00 [20:11:00.563] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.578] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:066501b590e905de {"provingJobId":"1:BASE_PARITY:066501b590e905de"} 20:11:00 [20:11:00.578] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:066501b590e905de type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:066501b590e905de"} 20:11:00 [20:11:00.578] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:066501b590e905de type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:066501b590e905de"} 20:11:00 [20:11:00.586] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.587] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:19462a82ec03bde6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:19462a82ec03bde6"} 20:11:00 [20:11:00.587] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUGNfXV/1 with map size 1073741824 20:11:00 [20:11:00.589] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.603] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:19462a82ec03bde6 {"provingJobId":"1:BASE_PARITY:19462a82ec03bde6"} 20:11:00 [20:11:00.603] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:19462a82ec03bde6 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:19462a82ec03bde6"} 20:11:00 [20:11:00.611] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.611] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c996ff6bf767ecfb epochNumber=1 {"provingJobId":"1:BASE_PARITY:c996ff6bf767ecfb"} 20:11:00 [20:11:00.612] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtDqCYw/1 with map size 1073741824 20:11:00 [20:11:00.613] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.627] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c996ff6bf767ecfb type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c996ff6bf767ecfb"} 20:11:00 [20:11:00.628] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c996ff6bf767ecfb type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c996ff6bf767ecfb"} 20:11:00 [20:11:00.634] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.634] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ad74aeecbfbe615a epochNumber=1 {"provingJobId":"1:BASE_PARITY:ad74aeecbfbe615a"} 20:11:00 [20:11:00.635] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcTSBx7/1 with map size 1073741824 20:11:00 [20:11:00.637] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.651] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ad74aeecbfbe615a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ad74aeecbfbe615a"} 20:11:00 [20:11:00.651] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ad74aeecbfbe615a type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ad74aeecbfbe615a"} 20:11:00 [20:11:00.658] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.658] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3ae1f4d305f9c52f epochNumber=0 {"provingJobId":"1:BASE_PARITY:3ae1f4d305f9c52f"} 20:11:00 [20:11:00.659] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testmPl9qI/0 with map size 1073741824 20:11:00 [20:11:00.660] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.674] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3ae1f4d305f9c52f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3ae1f4d305f9c52f"} 20:11:00 [20:11:00.674] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20b01157cf357d79 epochNumber=0 {"provingJobId":"1:BASE_PARITY:20b01157cf357d79"} 20:11:00 [20:11:00.675] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3ae1f4d305f9c52f epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3ae1f4d305f9c52f"} 20:11:00 [20:11:00.675] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fcd6f60c1416a7af epochNumber=0 {"provingJobId":"1:BASE_PARITY:fcd6f60c1416a7af"} 20:11:00 [20:11:00.686] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.687] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.688] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f4dd74cd62afec1d epochNumber=2 {"provingJobId":"1:BASE_PARITY:f4dd74cd62afec1d"} 20:11:00 [20:11:00.688] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVOpW6I/2 with map size 1073741824 20:11:00 [20:11:00.689] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.702] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:430b8e454f7c2fe2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:430b8e454f7c2fe2"} 20:11:00 [20:11:00.703] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVOpW6I/1 with map size 1073741824 20:11:00 [20:11:00.704] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.721] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.721] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f7af0105496dbcf epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f7af0105496dbcf"} 20:11:00 [20:11:00.722] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMlb6HI/1 with map size 1073741824 20:11:00 [20:11:00.723] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.738] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.739] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ccbd8fc9d222ed13 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ccbd8fc9d222ed13"} 20:11:00 [20:11:00.739] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7OiqZg/1 with map size 1073741824 20:11:00 [20:11:00.740] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.754] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf31153caa99641a epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf31153caa99641a"} 20:11:00 [20:11:00.759] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:42746f7fbceb585a epochNumber=2 {"provingJobId":"1:BASE_PARITY:42746f7fbceb585a"} 20:11:00 [20:11:00.759] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7OiqZg/2 with map size 1073741824 20:11:00 [20:11:00.760] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.774] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c205240d5efcb06 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c205240d5efcb06"} 20:11:00 [20:11:00.781] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.782] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5c4196ce6e53e7cf epochNumber=1 {"provingJobId":"1:BASE_PARITY:5c4196ce6e53e7cf"} 20:11:00 [20:11:00.782] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testALV2iY/1 with map size 1073741824 20:11:00 [20:11:00.784] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.797] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11a7e4aba02d339f epochNumber=1 {"provingJobId":"1:BASE_PARITY:11a7e4aba02d339f"} 20:11:00 [20:11:00.802] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3241f0a0fd8eecf2 epochNumber=2 {"provingJobId":"1:BASE_PARITY:3241f0a0fd8eecf2"} 20:11:00 [20:11:00.803] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testALV2iY/2 with map size 1073741824 20:11:00 [20:11:00.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.817] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e62983a88c74950 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e62983a88c74950"} 20:11:00 [20:11:00.824] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.825] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:375717695135122d epochNumber=1 {"provingJobId":"1:BASE_PARITY:375717695135122d"} 20:11:00 [20:11:00.825] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXAzwEb/1 with map size 1073741824 20:11:00 [20:11:00.827] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.844] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:633068030eb66b36 epochNumber=1 {"provingJobId":"1:BASE_PARITY:633068030eb66b36"} 20:11:00 [20:11:00.849] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c09a699cfd1f77dc epochNumber=2 {"provingJobId":"1:BASE_PARITY:c09a699cfd1f77dc"} 20:11:00 [20:11:00.850] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXAzwEb/2 with map size 1073741824 20:11:00 [20:11:00.851] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.865] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4b4eec88e0043692 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4b4eec88e0043692"} 20:11:00 [20:11:00.872] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.873] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca324f12aa6188cb epochNumber=1 {"provingJobId":"1:BASE_PARITY:ca324f12aa6188cb"} 20:11:00 [20:11:00.874] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqo2fku/1 with map size 1073741824 20:11:00 [20:11:00.875] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.888] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aaba0194efb7917d epochNumber=1 {"provingJobId":"1:BASE_PARITY:aaba0194efb7917d"} 20:11:00 [20:11:00.893] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ca324f12aa6188cb type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ca324f12aa6188cb"} 20:11:00 [20:11:00.901] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.901] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3c13b5bebc66b9e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3c13b5bebc66b9e7"} 20:11:00 [20:11:00.902] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnM6l7I/1 with map size 1073741824 20:11:00 [20:11:00.903] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.916] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:09c54553063f9148 epochNumber=1 {"provingJobId":"1:BASE_PARITY:09c54553063f9148"} 20:11:00 [20:11:00.921] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3c13b5bebc66b9e7 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:3c13b5bebc66b9e7"} 20:11:00 [20:11:00.928] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.929] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f407f29e667556af epochNumber=1 {"provingJobId":"1:BASE_PARITY:f407f29e667556af"} 20:11:00 [20:11:00.929] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6U50S3/1 with map size 1073741824 20:11:00 [20:11:00.931] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.945] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fd4abc5cd1bb42f0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fd4abc5cd1bb42f0"} 20:11:00 [20:11:00.950] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f407f29e667556af type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:f407f29e667556af"} 20:11:00 [20:11:00.953] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.953] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:de9c9092a85c2ef3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:de9c9092a85c2ef3"} 20:11:00 [20:11:00.954] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhVG5MY/1 with map size 1073741824 20:11:00 [20:11:00.955] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.969] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:de9c9092a85c2ef3 {"provingJobId":"1:BASE_PARITY:de9c9092a85c2ef3"} 20:11:00 [20:11:00.969] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:de9c9092a85c2ef3 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:de9c9092a85c2ef3"} 20:11:00 [20:11:00.975] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:54fde35caa78b624 epochNumber=1 {"provingJobId":"1:BASE_PARITY:54fde35caa78b624"} 20:11:00 [20:11:00.980] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:de9c9092a85c2ef3 has already been completed {"provingJobId":"1:BASE_PARITY:de9c9092a85c2ef3"} 20:11:00 [20:11:00.982] INFO: prover-client:proving-broker Proving Broker started 20:11:00 [20:11:00.983] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:711ca836913c066e epochNumber=1 {"provingJobId":"1:BASE_PARITY:711ca836913c066e"} 20:11:00 [20:11:00.984] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb9DNWy/1 with map size 1073741824 20:11:00 [20:11:00.985] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:00 [20:11:00.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bfd2eae50b4aaaa6 epochNumber=2 {"provingJobId":"1:BASE_PARITY:bfd2eae50b4aaaa6"} 20:11:00 [20:11:00.999] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testb9DNWy/2 with map size 1073741824 20:11:00 [20:11:01.000] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:02 [20:11:02.066] INFO: prover-client:proving-broker Proving Broker started 20:11:02 [20:11:02.070] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:711ca836913c066e settled=false {"provingJobId":"1:BASE_PARITY:711ca836913c066e","status":"pending"} 20:11:02 [20:11:02.071] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bfd2eae50b4aaaa6 settled=false {"provingJobId":"1:BASE_PARITY:bfd2eae50b4aaaa6","status":"pending"} 20:11:02 [20:11:02.072] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:711ca836913c066e type=BASE_PARITY startedAt=1750363861014 older agent has taken job {"provingJobId":"1:BASE_PARITY:711ca836913c066e"} 20:11:02 [20:11:02.072] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:711ca836913c066e type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:711ca836913c066e"} 20:11:02 [20:11:02.075] INFO: prover-client:proving-broker Proving Broker started 20:11:02 [20:11:02.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0913a2d6a01dc3c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0913a2d6a01dc3c1"} 20:11:02 [20:11:02.076] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoXxK8q/1 with map size 1073741824 20:11:02 [20:11:02.078] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:02 [20:11:02.091] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:22c430f514c6562f epochNumber=2 {"provingJobId":"1:BASE_PARITY:22c430f514c6562f"} 20:11:02 [20:11:02.092] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testoXxK8q/2 with map size 1073741824 20:11:02 [20:11:02.093] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:03 [20:11:03.109] INFO: prover-client:proving-broker Proving Broker started 20:11:03 [20:11:03.110] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0913a2d6a01dc3c1 settled=false {"provingJobId":"1:BASE_PARITY:0913a2d6a01dc3c1","status":"pending"} 20:11:03 [20:11:03.112] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:22c430f514c6562f settled=false {"provingJobId":"1:BASE_PARITY:22c430f514c6562f","status":"pending"} 20:11:03 [20:11:03.113] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0913a2d6a01dc3c1 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:0913a2d6a01dc3c1"} 20:11:03 [20:11:03.116] INFO: prover-client:proving-broker Proving Broker started 20:11:03 [20:11:03.116] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b64a54be2326af7e epochNumber=1 {"provingJobId":"1:BASE_PARITY:b64a54be2326af7e"} 20:11:03 [20:11:03.117] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiRLGM6/1 with map size 1073741824 20:11:03 [20:11:03.118] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:03 [20:11:03.133] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c32ea9035cdc1410 epochNumber=2 {"provingJobId":"1:BASE_PARITY:c32ea9035cdc1410"} 20:11:03 [20:11:03.134] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiRLGM6/2 with map size 1073741824 20:11:03 [20:11:03.135] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.150] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.152] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b64a54be2326af7e settled=false {"provingJobId":"1:BASE_PARITY:b64a54be2326af7e","status":"pending"} 20:11:04 [20:11:04.153] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c32ea9035cdc1410 settled=false {"provingJobId":"1:BASE_PARITY:c32ea9035cdc1410","status":"pending"} 20:11:04 [20:11:04.154] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b64a54be2326af7e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:b64a54be2326af7e"} 20:11:04 [20:11:04.155] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b64a54be2326af7e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:b64a54be2326af7e"} 20:11:04 [20:11:04.165] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba442b08f82d8959 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba442b08f82d8959"} 20:11:04 [20:11:04.166] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcJ66sE/1 with map size 1073741824 20:11:04 [20:11:04.167] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.183] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2aff715ab79776a3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2aff715ab79776a3"} 20:11:04 [20:11:04.183] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testcJ66sE/2 with map size 1073741824 20:11:04 [20:11:04.185] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.200] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ba442b08f82d8959 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ba442b08f82d8959"} 20:11:04 [20:11:04.205] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2aff715ab79776a3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2aff715ab79776a3"} 20:11:04 [20:11:04.214] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.214] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c10da97fa4402c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1c10da97fa4402c8"} 20:11:04 [20:11:04.215] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzReRS9/1 with map size 1073741824 20:11:04 [20:11:04.216] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.230] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:069ba7227d398375 epochNumber=2 {"provingJobId":"1:BASE_PARITY:069ba7227d398375"} 20:11:04 [20:11:04.231] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzReRS9/2 with map size 1073741824 20:11:04 [20:11:04.232] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.246] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1c10da97fa4402c8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1c10da97fa4402c8"} 20:11:04 [20:11:04.247] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1c10da97fa4402c8 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1c10da97fa4402c8"} 20:11:04 [20:11:04.252] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:069ba7227d398375 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:069ba7227d398375"} 20:11:04 [20:11:04.260] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.261] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2cbc7c010d05f324 err=test error {"provingJoId":"1:BASE_PARITY:2cbc7c010d05f324"} 20:11:04 [20:11:04.263] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.264] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f43f825b38bbd0b1 not found {"provingJobId":"1:BASE_PARITY:f43f825b38bbd0b1"} 20:11:04 [20:11:04.267] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.268] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9ff0e4a482276b0b epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ff0e4a482276b0b"} 20:11:04 [20:11:04.268] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXezGuT/1 with map size 1073741824 20:11:04 [20:11:04.269] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.286] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.287] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:368c77df1af68e9b epochNumber=1 {"provingJobId":"1:BASE_PARITY:368c77df1af68e9b"} 20:11:04 [20:11:04.287] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeeObL1/1 with map size 1073741824 20:11:04 [20:11:04.288] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.413] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:368c77df1af68e9b timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:368c77df1af68e9b"} 20:11:04 [20:11:04.420] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cdbef0fdb800da1c epochNumber=1 {"provingJobId":"1:BASE_PARITY:cdbef0fdb800da1c"} 20:11:04 [20:11:04.421] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiBeMmx/1 with map size 1073741824 20:11:04 [20:11:04.422] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.546] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cdbef0fdb800da1c timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:cdbef0fdb800da1c"} 20:11:04 [20:11:04.549] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04b52c708b4dfd98 epochNumber=10 {"provingJobId":"1:BASE_PARITY:04b52c708b4dfd98"} 20:11:04 [20:11:04.549] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testiBeMmx/10 with map size 1073741824 20:11:04 [20:11:04.551] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.571] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:04 [20:11:04.571] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 20:11:04 [20:11:04.694] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.695] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1cfdfd69935e06b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1cfdfd69935e06b6"} 20:11:04 [20:11:04.696] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgnpJx2/1 with map size 1073741824 20:11:04 [20:11:04.697] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.873] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1cfdfd69935e06b6 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:1cfdfd69935e06b6"} 20:11:04 [20:11:04.919] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.920] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:27a0503393f37edc epochNumber=1 {"provingJobId":"1:BASE_PARITY:27a0503393f37edc"} 20:11:04 [20:11:04.921] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDnP0h4/1 with map size 1073741824 20:11:04 [20:11:04.922] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.936] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:27a0503393f37edc type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:27a0503393f37edc"} 20:11:04 [20:11:04.939] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3578c379f91c6494 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3578c379f91c6494"} 20:11:04 [20:11:04.940] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testunowEs/1 with map size 1073741824 20:11:04 [20:11:04.941] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.955] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3578c379f91c6494 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:3578c379f91c6494"} 20:11:04 [20:11:04.956] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3578c379f91c6494 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:3578c379f91c6494"} 20:11:04 [20:11:04.963] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.964] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2403da9f37fe49f2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2403da9f37fe49f2"} 20:11:04 [20:11:04.964] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDqmHWp/1 with map size 1073741824 20:11:04 [20:11:04.965] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:04 [20:11:04.981] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2403da9f37fe49f2 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2403da9f37fe49f2"} 20:11:04 [20:11:04.988] INFO: prover-client:proving-broker Proving Broker started 20:11:04 [20:11:04.989] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:28e1cd191422158f epochNumber=1 {"provingJobId":"1:BASE_PARITY:28e1cd191422158f"} 20:11:04 [20:11:04.990] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJGaMhi/1 with map size 1073741824 20:11:04 [20:11:04.992] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.005] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:28e1cd191422158f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:28e1cd191422158f"} 20:11:05 [20:11:05.006] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:794daaffe44aa944 epochNumber=3 {"provingJobId":"1:BASE_PARITY:794daaffe44aa944"} 20:11:05 [20:11:05.007] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testJGaMhi/3 with map size 1073741824 20:11:05 [20:11:05.008] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.040] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:05 [20:11:05.040] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 20:11:05 [20:11:05.047] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:28e1cd191422158f err=test error {"provingJoId":"1:BASE_PARITY:28e1cd191422158f"} 20:11:05 [20:11:05.050] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHFqPak/1 with map size 1073741824 20:11:05 [20:11:05.052] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.067] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testHFqPak/2 with map size 1073741824 20:11:05 [20:11:05.068] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.082] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.083] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bfb363115a1ca119 settled=false {"provingJobId":"1:BASE_PARITY:bfb363115a1ca119","status":"pending"} 20:11:05 [20:11:05.085] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:acd6806c3da40d34 settled=false {"provingJobId":"1:BASE_PARITY:acd6806c3da40d34","status":"pending"} 20:11:05 [20:11:05.090] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMQOYl5/1 with map size 1073741824 20:11:05 [20:11:05.091] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.106] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testMQOYl5/2 with map size 1073741824 20:11:05 [20:11:05.107] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.131] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.133] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ce6eb23f407f2965 settled=true {"provingJobId":"1:BASE_PARITY:ce6eb23f407f2965","status":"fulfilled"} 20:11:05 [20:11:05.135] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:48b55fa5248b3ed6 settled=true {"provingJobId":"2:BASE_PARITY:48b55fa5248b3ed6","status":"fulfilled"} 20:11:05 [20:11:05.139] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOumrgp/1 with map size 1073741824 20:11:05 [20:11:05.141] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.162] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOumrgp/2 with map size 1073741824 20:11:05 [20:11:05.164] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.178] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.180] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e6c435e474bd62ae settled=true {"provingJobId":"1:BASE_PARITY:e6c435e474bd62ae","status":"fulfilled"} 20:11:05 [20:11:05.182] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:907515b26e623b78 settled=false {"provingJobId":"1:BASE_PARITY:907515b26e623b78","status":"pending"} 20:11:05 [20:11:05.186] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.186] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:53eff4d3d32100b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:53eff4d3d32100b5"} 20:11:05 [20:11:05.187] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQWeHIg/1 with map size 1073741824 20:11:05 [20:11:05.190] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.210] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4e0ff4e397f5dc3b epochNumber=1 {"provingJobId":"1:BASE_PARITY:4e0ff4e397f5dc3b"} 20:11:05 [20:11:05.211] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:4e0ff4e397f5dc3b: Error: db error: Error: db error 20:11:05 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:4e0ff4e397f5dc3b"} 20:11:05 [20:11:05.214] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.215] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:73efebb9e8385aba epochNumber=1 {"provingJobId":"1:BASE_PARITY:73efebb9e8385aba"} 20:11:05 [20:11:05.216] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test99da3r/1 with map size 1073741824 20:11:05 [20:11:05.219] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.235] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:73efebb9e8385aba type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:73efebb9e8385aba"} 20:11:05 [20:11:05.236] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:73efebb9e8385aba type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:73efebb9e8385aba"} 20:11:05 [20:11:05.244] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.245] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:04b8509f1b6a198e epochNumber=1 {"provingJobId":"1:BASE_PARITY:04b8509f1b6a198e"} 20:11:05 [20:11:05.246] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdc4MS0/1 with map size 1073741824 20:11:05 [20:11:05.248] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.262] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:04b8509f1b6a198e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:04b8509f1b6a198e"} 20:11:05 [20:11:05.262] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:04b8509f1b6a198e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:04b8509f1b6a198e"} 20:11:05 [20:11:05.263] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:04b8509f1b6a198e: Error: db error 20:11:05 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:04b8509f1b6a198e"} 20:11:05 [20:11:05.267] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.267] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7a014b4dcbbcd5be epochNumber=1 {"provingJobId":"1:BASE_PARITY:7a014b4dcbbcd5be"} 20:11:05 [20:11:05.269] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoLWsM2/1 with map size 1073741824 20:11:05 [20:11:05.271] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.288] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7a014b4dcbbcd5be type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7a014b4dcbbcd5be"} 20:11:05 [20:11:05.288] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a014b4dcbbcd5be type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:7a014b4dcbbcd5be"} 20:11:05 [20:11:05.297] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.298] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:884e8c05b704d372 epochNumber=1 {"provingJobId":"1:BASE_PARITY:884e8c05b704d372"} 20:11:05 [20:11:05.298] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmhjTc2/1 with map size 1073741824 20:11:05 [20:11:05.299] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.317] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:884e8c05b704d372 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:884e8c05b704d372"} 20:11:05 [20:11:05.318] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:884e8c05b704d372 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:884e8c05b704d372"} 20:11:05 [20:11:05.318] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:884e8c05b704d372 jobErr=test error: Error: db error 20:11:05 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:884e8c05b704d372"} 20:11:05 [20:11:05.322] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.323] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:89254b6824780e01 not found {"provingJobId":"1:BASE_PARITY:89254b6824780e01"} 20:11:05 [20:11:05.327] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.327] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0d8d69f993b8725d err=test error {"provingJoId":"1:BASE_PARITY:0d8d69f993b8725d"} 20:11:05 [20:11:05.331] INFO: prover-client:proving-broker Proving Broker started 20:11:05 [20:11:05.341] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1242f30612de5345 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1242f30612de5345"} 20:11:05 [20:11:05.343] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8UfNbO/1 with map size 1073741824 20:11:05 [20:11:05.345] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.361] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1242f30612de5345 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1242f30612de5345"} 20:11:05 [20:11:05.362] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1242f30612de5345 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1242f30612de5345"} 20:11:05 [20:11:05.379] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:e22b6b51e68bd045 epochNumber=2 {"provingJobId":"2:BASE_PARITY:e22b6b51e68bd045"} 20:11:05 [20:11:05.380] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8UfNbO/2 with map size 1073741824 20:11:05 [20:11:05.381] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.397] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:e22b6b51e68bd045 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:e22b6b51e68bd045"} 20:11:05 [20:11:05.398] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:e22b6b51e68bd045 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:e22b6b51e68bd045"} 20:11:05 [20:11:05.463] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:321388088df5f646 epochNumber=3 {"provingJobId":"3:BASE_PARITY:321388088df5f646"} 20:11:05 [20:11:05.463] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test8UfNbO/3 with map size 1073741824 20:11:05 [20:11:05.465] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.483] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:05 [20:11:05.484] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 20:11:05 [20:11:05.541] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:be32176f85abc14e epochNumber=4 {"provingJobId":"4:BASE_PARITY:be32176f85abc14e"} 20:11:05 [20:11:05.541] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test8UfNbO/4 with map size 1073741824 20:11:05 [20:11:05.543] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.562] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:05 [20:11:05.562] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 20:11:05 [20:11:05.619] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:3644a6687f5dd8b1 epochNumber=5 {"provingJobId":"5:BASE_PARITY:3644a6687f5dd8b1"} 20:11:05 [20:11:05.619] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test8UfNbO/5 with map size 1073741824 20:11:05 [20:11:05.621] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 20:11:05 [20:11:05.639] INFO: prover-client:proving-broker Cleaned up jobs=1 20:11:05 [20:11:05.640] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 20:11:05 PASS src/proving_broker/proving_broker.test.ts (11.189 s) 20:11:05 ProvingBroker 20:11:05 Producer API 20:11:05 ✓ refuses stale jobs (35 ms) 20:11:05 ✓ enqueues jobs (3 ms) 20:11:05 ✓ ignores duplicate jobs (4 ms) 20:11:05 ✓ reports correct status when enqueuing repeat jobs (4 ms) 20:11:05 ✓ throws an error in case of duplicate job IDs (18 ms) 20:11:05 ✓ returns not-found status for non-existing jobs (1 ms) 20:11:05 ✓ cancels jobs in queue (3 ms) 20:11:05 ✓ cancels jobs in-progress (2 ms) 20:11:05 ✓ returns job result if successful (3 ms) 20:11:05 ✓ returns job error if failed (2 ms) 20:11:05 ✓ correctly returns job status for concurrent writes (2 ms) 20:11:05 ✓ refuses stale jobs (28 ms) 20:11:05 ✓ enqueues jobs (29 ms) 20:11:05 ✓ ignores duplicate jobs (22 ms) 20:11:05 ✓ reports correct status when enqueuing repeat jobs (26 ms) 20:11:05 ✓ throws an error in case of duplicate job IDs (20 ms) 20:11:05 ✓ returns not-found status for non-existing jobs (2 ms) 20:11:05 ✓ cancels jobs in queue (25 ms) 20:11:05 ✓ cancels jobs in-progress (24 ms) 20:11:05 ✓ returns job result if successful (24 ms) 20:11:05 ✓ returns job error if failed (23 ms) 20:11:05 ✓ correctly returns job status for concurrent writes (27 ms) 20:11:05 Consumer API 20:11:05 ✓ returns undefined if no jobs are available 20:11:05 ✓ returns jobs in priority order (1 ms) 20:11:05 ✓ returns undefined if no jobs are available for the given allowList (5 ms) 20:11:05 ✓ returns a job if it is in the allowList (3 ms) 20:11:05 ✓ returns the most important job if it is in the allowList (2 ms) 20:11:05 ✓ returns any job if filter is empty (2 ms) 20:11:05 ✓ returns a new job when reporting job success (2 ms) 20:11:05 ✓ returns a new job when reporting permanent error (1 ms) 20:11:05 ✓ returns a new job when reporting retry-able error (1 ms) 20:11:05 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 20:11:05 ✓ returns a new job if job is already in progress elsewhere (1054 ms) 20:11:05 ✓ avoids sending the same job to a new agent after a restart (1003 ms) 20:11:05 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 20:11:05 ✓ tracks job result if in progress (3 ms) 20:11:05 ✓ tracks job result even if job is in queue (2 ms) 20:11:05 ✓ ignores reported job error if unknown job (1 ms) 20:11:05 ✓ ignores job result if unknown job (2 ms) 20:11:05 ✓ returns undefined if no jobs are available (1 ms) 20:11:05 ✓ returns jobs in priority order (32 ms) 20:11:05 ✓ returns undefined if no jobs are available for the given allowList (18 ms) 20:11:05 ✓ returns a job if it is in the allowList (43 ms) 20:11:05 ✓ returns the most important job if it is in the allowList (43 ms) 20:11:05 ✓ returns any job if filter is empty (48 ms) 20:11:05 ✓ returns a new job when reporting job success (28 ms) 20:11:05 ✓ returns a new job when reporting permanent error (28 ms) 20:11:05 ✓ returns a new job when reporting retry-able error (24 ms) 20:11:05 ✓ returns a new job when reporting progress if current one is cancelled (30 ms) 20:11:05 ✓ returns a new job if job is already in progress elsewhere (1093 ms) 20:11:05 ✓ avoids sending the same job to a new agent after a restart (1040 ms) 20:11:05 ✓ avoids sending a completed job to a new agent after a restart (1049 ms) 20:11:05 ✓ tracks job result if in progress (48 ms) 20:11:05 ✓ tracks job result even if job is in queue (46 ms) 20:11:05 ✓ ignores reported job error if unknown job (3 ms) 20:11:05 ✓ ignores job result if unknown job (3 ms) 20:11:05 Timeouts 20:11:05 ✓ tracks in progress jobs (1 ms) 20:11:05 ✓ re-enqueues jobs that time out (108 ms) 20:11:05 ✓ cancel stale jobs that time out (229 ms) 20:11:05 ✓ keeps the jobs in progress while it is alive (202 ms) 20:11:05 ✓ tracks in progress jobs (20 ms) 20:11:05 ✓ re-enqueues jobs that time out (133 ms) 20:11:05 ✓ cancel stale jobs that time out (274 ms) 20:11:05 ✓ keeps the jobs in progress while it is alive (225 ms) 20:11:05 Retries 20:11:05 ✓ retries jobs (2 ms) 20:11:05 ✓ retries up to a maximum number of times (3 ms) 20:11:05 ✓ passing retry=false does not retry (1 ms) 20:11:05 ✓ does not retry if job is stale (28 ms) 20:11:05 ✓ retries jobs (19 ms) 20:11:05 ✓ retries up to a maximum number of times (23 ms) 20:11:05 ✓ passing retry=false does not retry (25 ms) 20:11:05 ✓ does not retry if job is stale (61 ms) 20:11:05 Database management 20:11:05 ✓ re-enqueues proof requests on start (3 ms) 20:11:05 ✓ restores proof results on start (1 ms) 20:11:05 ✓ only re-enqueues unfinished jobs (1 ms) 20:11:05 ✓ saves job when enqueued (3 ms) 20:11:05 ✓ does not retain job if database fails to save (2 ms) 20:11:05 ✓ saves job result (2 ms) 20:11:05 ✓ saves result even if database fails to save (2 ms) 20:11:05 ✓ saves job error (2 ms) 20:11:05 ✓ saves job error even if database fails to save (2 ms) 20:11:05 ✓ does not save job result if job is unknown (4 ms) 20:11:05 ✓ does not save job error if job is unknown (2 ms) 20:11:05 ✓ cleans up old jobs periodically (256 ms) 20:11:05 ✓ re-enqueues proof requests on start (39 ms) 20:11:05 ✓ restores proof results on start (50 ms) 20:11:05 ✓ only re-enqueues unfinished jobs (47 ms) 20:11:05 ✓ saves job when enqueued (22 ms) 20:11:05 ✓ does not retain job if database fails to save (6 ms) 20:11:05 ✓ saves job result (29 ms) 20:11:05 ✓ saves result even if database fails to save (22 ms) 20:11:05 ✓ saves job error (29 ms) 20:11:05 ✓ saves job error even if database fails to save (25 ms) 20:11:05 ✓ does not save job result if job is unknown (4 ms) 20:11:05 ✓ does not save job error if job is unknown (4 ms) 20:11:05 ✓ cleans up old jobs periodically (359 ms) 20:11:05 20:11:05 Test Suites: 1 passed, 1 total 20:11:05 Tests: 96 passed, 96 total 20:11:05 Snapshots: 0 total 20:11:05 Time: 11.264 s 20:11:05 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 20:11:05 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?