Parent Log:
http://ci.aztec-labs.com/232a0193ad74c3c2
Command: 84c999e211bb73e4 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/3863bc02d671057d0cffd4ced17ca317788ff190
Env: REF_NAME=gh-readonly-queue/staging/pr-15238-db396f13b7ef374a230174bdc4078d9fbb3c788a CURRENT_VERSION=1.0.0 CI_FULL=1
Date: Tue Jun 24 12:13:05 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15238_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_staging
12:13:09 [12:13:09.953]
INFO:
prover-client:proving-broker Proving Broker started
12:13:09 [12:13:09.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:431d63ff179058d2 epochNumber=42
{"provingJobId":"1:BASE_PARITY:431d63ff179058d2"}
12:13:09 [12:13:09.962]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:b1a81e4b2ed3879e epochNumber=1
{"provingJobId":"1:BASE_PARITY:b1a81e4b2ed3879e"}
12:13:09 [12:13:09.987]
INFO:
prover-client:proving-broker Proving Broker started
12:13:09 [12:13:09.988]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f55e201e4f3bed8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f55e201e4f3bed8"}
12:13:09 [12:13:09.989]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:78685dcb510de468 epochNumber=1
{"provingJobId":"1:BASE_PARITY:78685dcb510de468"}
12:13:09 [12:13:09.990]
INFO:
prover-client:proving-broker Proving Broker started
12:13:09 [12:13:09.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8e1cbd83b0a48491 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8e1cbd83b0a48491"}
12:13:09 [12:13:09.992]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:8e1cbd83b0a48491 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:8e1cbd83b0a48491"}
12:13:09 [12:13:09.994]
INFO:
prover-client:proving-broker Proving Broker started
12:13:09 [12:13:09.995]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68d61d7602b85f24 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68d61d7602b85f24"}
12:13:09 [12:13:09.996]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:68d61d7602b85f24 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:68d61d7602b85f24"}
12:13:09 [12:13:09.996]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:68d61d7602b85f24 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:68d61d7602b85f24"}
12:13:09 [12:13:09.997]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:68d61d7602b85f24 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:68d61d7602b85f24"}
12:13:09 [12:13:09.998]
INFO:
prover-client:proving-broker Proving Broker started
12:13:09 [12:13:09.998]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8388c06976d13695 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8388c06976d13695"}
12:13:10 [12:13:10.023]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.024]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.024]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5c696ad20d4c8fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:c5c696ad20d4c8fd"}
12:13:10 [12:13:10.025]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c5c696ad20d4c8fd
{"provingJobId":"1:BASE_PARITY:c5c696ad20d4c8fd"}
12:13:10 [12:13:10.025]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c5c696ad20d4c8fd type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c5c696ad20d4c8fd"}
12:13:10 [12:13:10.026]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c5c696ad20d4c8fd type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c5c696ad20d4c8fd"}
12:13:10 [12:13:10.027]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.027]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe368f54b16b3f72 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe368f54b16b3f72"}
12:13:10 [12:13:10.028]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:fe368f54b16b3f72
{"provingJobId":"1:BASE_PARITY:fe368f54b16b3f72"}
12:13:10 [12:13:10.028]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fe368f54b16b3f72 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:fe368f54b16b3f72"}
12:13:10 [12:13:10.029]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.030]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0eb807ccf9cf983 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0eb807ccf9cf983"}
12:13:10 [12:13:10.039]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e0eb807ccf9cf983 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e0eb807ccf9cf983"}
12:13:10 [12:13:10.039]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e0eb807ccf9cf983 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e0eb807ccf9cf983"}
12:13:10 [12:13:10.044]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.046]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:88a1beb7fd3120ee epochNumber=1
{"provingJobId":"1:BASE_PARITY:88a1beb7fd3120ee"}
12:13:10 [12:13:10.047]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:88a1beb7fd3120ee type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:88a1beb7fd3120ee"}
12:13:10 [12:13:10.048]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:88a1beb7fd3120ee type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:88a1beb7fd3120ee"}
12:13:10 [12:13:10.051]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.053]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3845013a137bbecd epochNumber=0
{"provingJobId":"1:BASE_PARITY:3845013a137bbecd"}
12:13:10 [12:13:10.053]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3845013a137bbecd epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3845013a137bbecd"}
12:13:10 [12:13:10.054]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c904914ddf2dee4 epochNumber=0
{"provingJobId":"1:BASE_PARITY:3c904914ddf2dee4"}
12:13:10 [12:13:10.054]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3845013a137bbecd epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3845013a137bbecd"}
12:13:10 [12:13:10.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:357e5c992c5ee15b epochNumber=0
{"provingJobId":"1:BASE_PARITY:357e5c992c5ee15b"}
12:13:10 [12:13:10.058]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.061]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.061]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2916dd12a13f0928 epochNumber=2
{"provingJobId":"1:BASE_PARITY:2916dd12a13f0928"}
12:13:10 [12:13:10.062]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1ed7b3ce8184b99f epochNumber=1
{"provingJobId":"1:BASE_PARITY:1ed7b3ce8184b99f"}
12:13:10 [12:13:10.063]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.064]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:17c38cb2b182697b epochNumber=1
{"provingJobId":"1:BASE_PARITY:17c38cb2b182697b"}
12:13:10 [12:13:10.065]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.066]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:72dc6f5e78004fd9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:72dc6f5e78004fd9"}
12:13:10 [12:13:10.066]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:04eb7a070e821b88 epochNumber=1
{"provingJobId":"1:BASE_PARITY:04eb7a070e821b88"}
12:13:10 [12:13:10.067]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae883ab03afc36e4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ae883ab03afc36e4"}
12:13:10 [12:13:10.067]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd9966351a64827f epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd9966351a64827f"}
12:13:10 [12:13:10.069]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.070]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0e21352b95adec6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0e21352b95adec6"}
12:13:10 [12:13:10.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7bdba64efa24c82 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7bdba64efa24c82"}
12:13:10 [12:13:10.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aaac91be02f7421e epochNumber=2
{"provingJobId":"1:BASE_PARITY:aaac91be02f7421e"}
12:13:10 [12:13:10.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4828fcd227eb0565 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4828fcd227eb0565"}
12:13:10 [12:13:10.075]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.078]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81a10528b06dac17 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81a10528b06dac17"}
12:13:10 [12:13:10.079]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a9b87f6bfc28a1a5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a9b87f6bfc28a1a5"}
12:13:10 [12:13:10.081]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:87a737db9d07fb60 epochNumber=2
{"provingJobId":"1:BASE_PARITY:87a737db9d07fb60"}
12:13:10 [12:13:10.082]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4dd73022d718ca94 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4dd73022d718ca94"}
12:13:10 [12:13:10.085]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.087]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e5db305d2def2d4b epochNumber=1
{"provingJobId":"1:BASE_PARITY:e5db305d2def2d4b"}
12:13:10 [12:13:10.090]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8ea535f1630979d0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8ea535f1630979d0"}
12:13:10 [12:13:10.091]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e5db305d2def2d4b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e5db305d2def2d4b"}
12:13:10 [12:13:10.095]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.096]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c19a08f4a32b5802 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c19a08f4a32b5802"}
12:13:10 [12:13:10.098]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1b7c8b7ab562657 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c1b7c8b7ab562657"}
12:13:10 [12:13:10.099]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c19a08f4a32b5802 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:c19a08f4a32b5802"}
12:13:10 [12:13:10.100]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.101]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c6799d8429a2246 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c6799d8429a2246"}
12:13:10 [12:13:10.101]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ea01fef441e9468 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9ea01fef441e9468"}
12:13:10 [12:13:10.102]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3c6799d8429a2246 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:3c6799d8429a2246"}
12:13:10 [12:13:10.103]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.104]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:48acff560ff13ec9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:48acff560ff13ec9"}
12:13:10 [12:13:10.104]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:48acff560ff13ec9
{"provingJobId":"1:BASE_PARITY:48acff560ff13ec9"}
12:13:10 [12:13:10.104]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:48acff560ff13ec9 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:48acff560ff13ec9"}
12:13:10 [12:13:10.105]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:215e37b8d9601e8d epochNumber=1
{"provingJobId":"1:BASE_PARITY:215e37b8d9601e8d"}
12:13:10 [12:13:10.105]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:48acff560ff13ec9 has already been completed
{"provingJobId":"1:BASE_PARITY:48acff560ff13ec9"}
12:13:10 [12:13:10.107]
INFO:
prover-client:proving-broker Proving Broker started
12:13:10 [12:13:10.107]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60048abdfef86253 epochNumber=1
{"provingJobId":"1:BASE_PARITY:60048abdfef86253"}
12:13:10 [12:13:10.107]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5cfc19ad19c5ca69 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5cfc19ad19c5ca69"}
12:13:11 [12:13:11.160]
INFO:
prover-client:proving-broker Proving Broker started
12:13:11 [12:13:11.162]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:60048abdfef86253 settled=false
{"provingJobId":"1:BASE_PARITY:60048abdfef86253","status":"pending"}
12:13:11 [12:13:11.162]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5cfc19ad19c5ca69 settled=false
{"provingJobId":"1:BASE_PARITY:5cfc19ad19c5ca69","status":"pending"}
12:13:11 [12:13:11.164]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:60048abdfef86253 type=BASE_PARITY startedAt=1750767190108 older agent has taken job
{"provingJobId":"1:BASE_PARITY:60048abdfef86253"}
12:13:11 [12:13:11.166]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:60048abdfef86253 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:60048abdfef86253"}
12:13:11 [12:13:11.168]
INFO:
prover-client:proving-broker Proving Broker started
12:13:11 [12:13:11.168]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:18c1899eb379b0e0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:18c1899eb379b0e0"}
12:13:11 [12:13:11.169]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c94f531b96b1d5e5 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c94f531b96b1d5e5"}
12:13:12 [12:13:12.170]
INFO:
prover-client:proving-broker Proving Broker started
12:13:12 [12:13:12.170]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:18c1899eb379b0e0 settled=false
{"provingJobId":"1:BASE_PARITY:18c1899eb379b0e0","status":"pending"}
12:13:12 [12:13:12.171]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c94f531b96b1d5e5 settled=false
{"provingJobId":"1:BASE_PARITY:c94f531b96b1d5e5","status":"pending"}
12:13:12 [12:13:12.172]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:18c1899eb379b0e0 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:18c1899eb379b0e0"}
12:13:12 [12:13:12.173]
INFO:
prover-client:proving-broker Proving Broker started
12:13:12 [12:13:12.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:877b5dfd42592cfc epochNumber=1
{"provingJobId":"1:BASE_PARITY:877b5dfd42592cfc"}
12:13:12 [12:13:12.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01d62825aa956322 epochNumber=2
{"provingJobId":"1:BASE_PARITY:01d62825aa956322"}
12:13:13 [12:13:13.177]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.178]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:877b5dfd42592cfc settled=false
{"provingJobId":"1:BASE_PARITY:877b5dfd42592cfc","status":"pending"}
12:13:13 [12:13:13.178]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:01d62825aa956322 settled=false
{"provingJobId":"1:BASE_PARITY:01d62825aa956322","status":"pending"}
12:13:13 [12:13:13.179]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:877b5dfd42592cfc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:877b5dfd42592cfc"}
12:13:13 [12:13:13.179]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:877b5dfd42592cfc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:877b5dfd42592cfc"}
12:13:13 [12:13:13.181]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.181]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ee2660eb53ba9421 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ee2660eb53ba9421"}
12:13:13 [12:13:13.182]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0429ba5ba67e3e70 epochNumber=2
{"provingJobId":"1:BASE_PARITY:0429ba5ba67e3e70"}
12:13:13 [12:13:13.182]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ee2660eb53ba9421 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ee2660eb53ba9421"}
12:13:13 [12:13:13.183]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0429ba5ba67e3e70 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0429ba5ba67e3e70"}
12:13:13 [12:13:13.183]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.184]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b6042842935fa89 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7b6042842935fa89"}
12:13:13 [12:13:13.184]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:23dd18383cd702cc epochNumber=2
{"provingJobId":"1:BASE_PARITY:23dd18383cd702cc"}
12:13:13 [12:13:13.185]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7b6042842935fa89 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7b6042842935fa89"}
12:13:13 [12:13:13.185]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7b6042842935fa89 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7b6042842935fa89"}
12:13:13 [12:13:13.185]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:23dd18383cd702cc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:23dd18383cd702cc"}
12:13:13 [12:13:13.186]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.187]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c2caf4a7d29f4dfb err=test error
{"provingJoId":"1:BASE_PARITY:c2caf4a7d29f4dfb"}
12:13:13 [12:13:13.188]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.188]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5014f4d1bf6db17b not found
{"provingJobId":"1:BASE_PARITY:5014f4d1bf6db17b"}
12:13:13 [12:13:13.190]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.190]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12aaa55081273ee5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:12aaa55081273ee5"}
12:13:13 [12:13:13.191]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.192]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7c641a620c62211 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7c641a620c62211"}
12:13:13 [12:13:13.293]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b7c641a620c62211 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b7c641a620c62211"}
12:13:13 [12:13:13.299]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.300]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d1dd55eeec14bc69 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d1dd55eeec14bc69"}
12:13:13 [12:13:13.400]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d1dd55eeec14bc69 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:d1dd55eeec14bc69"}
12:13:13 [12:13:13.407]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c7009748466afa32 epochNumber=10
{"provingJobId":"1:BASE_PARITY:c7009748466afa32"}
12:13:13 [12:13:13.427]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:13 [12:13:13.534]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.534]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0fc5a83e10d11961 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0fc5a83e10d11961"}
12:13:13 [12:13:13.688]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0fc5a83e10d11961 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:0fc5a83e10d11961"}
12:13:13 [12:13:13.739]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.740]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a37699313c68af7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a37699313c68af7"}
12:13:13 [12:13:13.740]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7a37699313c68af7 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:7a37699313c68af7"}
12:13:13 [12:13:13.742]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.742]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:458548e6fc9a1634 epochNumber=1
{"provingJobId":"1:BASE_PARITY:458548e6fc9a1634"}
12:13:13 [12:13:13.743]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:458548e6fc9a1634 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:458548e6fc9a1634"}
12:13:13 [12:13:13.744]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:458548e6fc9a1634 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:458548e6fc9a1634"}
12:13:13 [12:13:13.745]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.745]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1b7314890793b33 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c1b7314890793b33"}
12:13:13 [12:13:13.746]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c1b7314890793b33 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c1b7314890793b33"}
12:13:13 [12:13:13.747]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.747]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e2c339370e7c33d2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e2c339370e7c33d2"}
12:13:13 [12:13:13.748]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e2c339370e7c33d2 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e2c339370e7c33d2"}
12:13:13 [12:13:13.748]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a78a2c41faf9957 epochNumber=3
{"provingJobId":"1:BASE_PARITY:5a78a2c41faf9957"}
12:13:13 [12:13:13.773]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:13 [12:13:13.773]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e2c339370e7c33d2 err=test error
{"provingJoId":"1:BASE_PARITY:e2c339370e7c33d2"}
12:13:13 [12:13:13.775]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.775]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c505e1f612840c87 settled=false
{"provingJobId":"1:BASE_PARITY:c505e1f612840c87","status":"pending"}
12:13:13 [12:13:13.775]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:08d1613107b170d8 settled=false
{"provingJobId":"1:BASE_PARITY:08d1613107b170d8","status":"pending"}
12:13:13 [12:13:13.777]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.778]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d11840b0ab8265b8 settled=true
{"provingJobId":"1:BASE_PARITY:d11840b0ab8265b8","status":"fulfilled"}
12:13:13 [12:13:13.778]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:85a0466133a5923f settled=true
{"provingJobId":"2:BASE_PARITY:85a0466133a5923f","status":"fulfilled"}
12:13:13 [12:13:13.779]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.779]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c239b5d88844f13a settled=true
{"provingJobId":"1:BASE_PARITY:c239b5d88844f13a","status":"fulfilled"}
12:13:13 [12:13:13.780]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:95ff8094b80b5b3c settled=false
{"provingJobId":"1:BASE_PARITY:95ff8094b80b5b3c","status":"pending"}
12:13:13 [12:13:13.781]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.782]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:00c74401e644e98c epochNumber=1
{"provingJobId":"1:BASE_PARITY:00c74401e644e98c"}
12:13:13 [12:13:13.783]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.784]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a868b175d23dc797 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a868b175d23dc797"}
12:13:13 [12:13:13.785]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:a868b175d23dc797: Error: db error: Error: db error
12:13:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:a868b175d23dc797"}
12:13:13 [12:13:13.786]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.786]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49704496541a9eb6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:49704496541a9eb6"}
12:13:13 [12:13:13.787]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:49704496541a9eb6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:49704496541a9eb6"}
12:13:13 [12:13:13.787]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:49704496541a9eb6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:49704496541a9eb6"}
12:13:13 [12:13:13.788]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.788]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:492068e6ef7f1347 epochNumber=1
{"provingJobId":"1:BASE_PARITY:492068e6ef7f1347"}
12:13:13 [12:13:13.789]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:492068e6ef7f1347 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:492068e6ef7f1347"}
12:13:13 [12:13:13.789]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:492068e6ef7f1347 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:492068e6ef7f1347"}
12:13:13 [12:13:13.790]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:492068e6ef7f1347: Error: db error
12:13:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:492068e6ef7f1347"}
12:13:13 [12:13:13.791]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.791]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1f54e833800539a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1f54e833800539a3"}
12:13:13 [12:13:13.792]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1f54e833800539a3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:1f54e833800539a3"}
12:13:13 [12:13:13.792]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1f54e833800539a3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1f54e833800539a3"}
12:13:13 [12:13:13.793]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.794]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bde116ef4e88625d epochNumber=1
{"provingJobId":"1:BASE_PARITY:bde116ef4e88625d"}
12:13:13 [12:13:13.794]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bde116ef4e88625d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bde116ef4e88625d"}
12:13:13 [12:13:13.794]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bde116ef4e88625d type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:bde116ef4e88625d"}
12:13:13 [12:13:13.795]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:bde116ef4e88625d jobErr=test error: Error: db error
12:13:13 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:bde116ef4e88625d"}
12:13:13 [12:13:13.796]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.796]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a4c705318a97aa75 not found
{"provingJobId":"1:BASE_PARITY:a4c705318a97aa75"}
12:13:13 [12:13:13.798]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.798]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f62a6dbce89ca602 err=test error
{"provingJoId":"1:BASE_PARITY:f62a6dbce89ca602"}
12:13:13 [12:13:13.799]
INFO:
prover-client:proving-broker Proving Broker started
12:13:13 [12:13:13.809]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b55df29991a03621 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b55df29991a03621"}
12:13:13 [12:13:13.810]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b55df29991a03621 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b55df29991a03621"}
12:13:13 [12:13:13.810]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b55df29991a03621 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b55df29991a03621"}
12:13:13 [12:13:13.820]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:ae9e40d7cdd5a0c7 epochNumber=2
{"provingJobId":"2:BASE_PARITY:ae9e40d7cdd5a0c7"}
12:13:13 [12:13:13.821]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:ae9e40d7cdd5a0c7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:ae9e40d7cdd5a0c7"}
12:13:13 [12:13:13.821]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:ae9e40d7cdd5a0c7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:ae9e40d7cdd5a0c7"}
12:13:13 [12:13:13.883]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:ca55b79c7c218352 epochNumber=3
{"provingJobId":"3:BASE_PARITY:ca55b79c7c218352"}
12:13:13 [12:13:13.899]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:13 [12:13:13.945]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:5b23c9284b4d8e10 epochNumber=4
{"provingJobId":"4:BASE_PARITY:5b23c9284b4d8e10"}
12:13:13 [12:13:13.951]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:14 [12:13:14.008]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:da10c951974238ec epochNumber=5
{"provingJobId":"5:BASE_PARITY:da10c951974238ec"}
12:13:14 [12:13:14.028]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:14 [12:13:14.063]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.063]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ad4f6b0555bed89 epochNumber=42
{"provingJobId":"1:BASE_PARITY:9ad4f6b0555bed89"}
12:13:14 [12:13:14.065]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testFQRK2E/42 with map size 1073741824
12:13:14 [12:13:14.074]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.099]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:298a8897d3e7bd82 epochNumber=1
{"provingJobId":"1:BASE_PARITY:298a8897d3e7bd82"}
12:13:14 [12:13:14.103]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.104]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bfb347d22baa8fc0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bfb347d22baa8fc0"}
12:13:14 [12:13:14.105]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhmHKqD/1 with map size 1073741824
12:13:14 [12:13:14.107]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.125]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e5a375ad173df70 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e5a375ad173df70"}
12:13:14 [12:13:14.137]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.138]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:45aab9324c8d7c75 epochNumber=1
{"provingJobId":"1:BASE_PARITY:45aab9324c8d7c75"}
12:13:14 [12:13:14.138]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCVU9pL/1 with map size 1073741824
12:13:14 [12:13:14.141]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.159]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:45aab9324c8d7c75 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:45aab9324c8d7c75"}
12:13:14 [12:13:14.164]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.165]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b78c5f6f0f8f8819 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b78c5f6f0f8f8819"}
12:13:14 [12:13:14.166]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test22t7iB/1 with map size 1073741824
12:13:14 [12:13:14.168]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.186]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b78c5f6f0f8f8819 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b78c5f6f0f8f8819"}
12:13:14 [12:13:14.187]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b78c5f6f0f8f8819 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b78c5f6f0f8f8819"}
12:13:14 [12:13:14.193]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b78c5f6f0f8f8819 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b78c5f6f0f8f8819"}
12:13:14 [12:13:14.200]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.200]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f68f427588ad4a22 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f68f427588ad4a22"}
12:13:14 [12:13:14.201]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzUcXAp/1 with map size 1073741824
12:13:14 [12:13:14.205]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.231]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.234]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.235]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cc8561aeab570e3a epochNumber=1
{"provingJobId":"1:BASE_PARITY:cc8561aeab570e3a"}
12:13:14 [12:13:14.235]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXwoTdX/1 with map size 1073741824
12:13:14 [12:13:14.238]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.256]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cc8561aeab570e3a
{"provingJobId":"1:BASE_PARITY:cc8561aeab570e3a"}
12:13:14 [12:13:14.257]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cc8561aeab570e3a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cc8561aeab570e3a"}
12:13:14 [12:13:14.257]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cc8561aeab570e3a type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:cc8561aeab570e3a"}
12:13:14 [12:13:14.268]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.269]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb4c40f07a9a408e epochNumber=1
{"provingJobId":"1:BASE_PARITY:cb4c40f07a9a408e"}
12:13:14 [12:13:14.269]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnZBuk7/1 with map size 1073741824
12:13:14 [12:13:14.271]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.305]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cb4c40f07a9a408e
{"provingJobId":"1:BASE_PARITY:cb4c40f07a9a408e"}
12:13:14 [12:13:14.306]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cb4c40f07a9a408e type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:cb4c40f07a9a408e"}
12:13:14 [12:13:14.315]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.315]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51fb7cc34e23925e epochNumber=1
{"provingJobId":"1:BASE_PARITY:51fb7cc34e23925e"}
12:13:14 [12:13:14.316]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcuShHJ/1 with map size 1073741824
12:13:14 [12:13:14.318]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.334]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:51fb7cc34e23925e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:51fb7cc34e23925e"}
12:13:14 [12:13:14.335]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:51fb7cc34e23925e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:51fb7cc34e23925e"}
12:13:14 [12:13:14.344]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.345]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3bbfb007c1e11b2c epochNumber=1
{"provingJobId":"1:BASE_PARITY:3bbfb007c1e11b2c"}
12:13:14 [12:13:14.345]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQc9Kz9/1 with map size 1073741824
12:13:14 [12:13:14.347]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.365]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3bbfb007c1e11b2c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3bbfb007c1e11b2c"}
12:13:14 [12:13:14.366]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3bbfb007c1e11b2c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3bbfb007c1e11b2c"}
12:13:14 [12:13:14.384]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.386]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0943da71bcb07ef6 epochNumber=0
{"provingJobId":"1:BASE_PARITY:0943da71bcb07ef6"}
12:13:14 [12:13:14.388]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testsakoZO/0 with map size 1073741824
12:13:14 [12:13:14.392]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.409]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0943da71bcb07ef6 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0943da71bcb07ef6"}
12:13:14 [12:13:14.410]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9bbd4b8e948e1f03 epochNumber=0
{"provingJobId":"1:BASE_PARITY:9bbd4b8e948e1f03"}
12:13:14 [12:13:14.410]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0943da71bcb07ef6 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0943da71bcb07ef6"}
12:13:14 [12:13:14.410]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4b1bcfbf1faa853c epochNumber=0
{"provingJobId":"1:BASE_PARITY:4b1bcfbf1faa853c"}
12:13:14 [12:13:14.423]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.425]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.426]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ffcab2857c2459a epochNumber=2
{"provingJobId":"1:BASE_PARITY:2ffcab2857c2459a"}
12:13:14 [12:13:14.426]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCU52TG/2 with map size 1073741824
12:13:14 [12:13:14.428]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.443]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f970070187bd2e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f970070187bd2e6"}
12:13:14 [12:13:14.444]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCU52TG/1 with map size 1073741824
12:13:14 [12:13:14.446]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.465]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.465]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a17fc0a88126cc1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a17fc0a88126cc1"}
12:13:14 [12:13:14.466]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMYZ9G4/1 with map size 1073741824
12:13:14 [12:13:14.480]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.510]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.511]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:961d7de23d806011 epochNumber=1
{"provingJobId":"1:BASE_PARITY:961d7de23d806011"}
12:13:14 [12:13:14.512]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3SazfR/1 with map size 1073741824
12:13:14 [12:13:14.514]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.530]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b69cea90c269c080 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b69cea90c269c080"}
12:13:14 [12:13:14.536]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30a83644bc2eed11 epochNumber=2
{"provingJobId":"1:BASE_PARITY:30a83644bc2eed11"}
12:13:14 [12:13:14.537]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3SazfR/2 with map size 1073741824
12:13:14 [12:13:14.539]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.556]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d9acd0fb78e03690 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d9acd0fb78e03690"}
12:13:14 [12:13:14.566]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.567]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a89f098ba16cf791 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a89f098ba16cf791"}
12:13:14 [12:13:14.572]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0PH4jz/1 with map size 1073741824
12:13:14 [12:13:14.578]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.602]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e629e565a050972d epochNumber=1
{"provingJobId":"1:BASE_PARITY:e629e565a050972d"}
12:13:14 [12:13:14.607]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b7c98eb7fb0754a epochNumber=2
{"provingJobId":"1:BASE_PARITY:7b7c98eb7fb0754a"}
12:13:14 [12:13:14.608]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0PH4jz/2 with map size 1073741824
12:13:14 [12:13:14.613]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.629]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:95a83217ec415bdd epochNumber=1
{"provingJobId":"1:BASE_PARITY:95a83217ec415bdd"}
12:13:14 [12:13:14.637]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.638]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5f6407844cc61142 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5f6407844cc61142"}
12:13:14 [12:13:14.639]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testE4IZMe/1 with map size 1073741824
12:13:14 [12:13:14.640]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.655]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0398a682537ec87 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b0398a682537ec87"}
12:13:14 [12:13:14.661]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3875dddbbaddde32 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3875dddbbaddde32"}
12:13:14 [12:13:14.663]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testE4IZMe/2 with map size 1073741824
12:13:14 [12:13:14.667]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.695]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bfa37094d80cefa8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bfa37094d80cefa8"}
12:13:14 [12:13:14.704]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.705]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4ad1bc68e3475d7d epochNumber=1
{"provingJobId":"1:BASE_PARITY:4ad1bc68e3475d7d"}
12:13:14 [12:13:14.706]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6v6NV6/1 with map size 1073741824
12:13:14 [12:13:14.707]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.725]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:345e94c75cbc9b43 epochNumber=1
{"provingJobId":"1:BASE_PARITY:345e94c75cbc9b43"}
12:13:14 [12:13:14.737]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4ad1bc68e3475d7d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4ad1bc68e3475d7d"}
12:13:14 [12:13:14.745]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.746]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:695fe7ea30e43394 epochNumber=1
{"provingJobId":"1:BASE_PARITY:695fe7ea30e43394"}
12:13:14 [12:13:14.747]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1mCKxC/1 with map size 1073741824
12:13:14 [12:13:14.748]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.767]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:afc6f4739f559ab3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:afc6f4739f559ab3"}
12:13:14 [12:13:14.783]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:695fe7ea30e43394 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:695fe7ea30e43394"}
12:13:14 [12:13:14.795]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.796]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a0e5088bada2d1b6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a0e5088bada2d1b6"}
12:13:14 [12:13:14.796]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRiOOjV/1 with map size 1073741824
12:13:14 [12:13:14.799]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.822]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2991b3a38830aad3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2991b3a38830aad3"}
12:13:14 [12:13:14.829]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a0e5088bada2d1b6 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:a0e5088bada2d1b6"}
12:13:14 [12:13:14.834]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.834]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:da8c40f85f52f6af epochNumber=1
{"provingJobId":"1:BASE_PARITY:da8c40f85f52f6af"}
12:13:14 [12:13:14.835]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTTD9mT/1 with map size 1073741824
12:13:14 [12:13:14.837]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.854]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:da8c40f85f52f6af
{"provingJobId":"1:BASE_PARITY:da8c40f85f52f6af"}
12:13:14 [12:13:14.854]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:da8c40f85f52f6af type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:da8c40f85f52f6af"}
12:13:14 [12:13:14.862]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a53cbedc662b243 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a53cbedc662b243"}
12:13:14 [12:13:14.868]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:da8c40f85f52f6af has already been completed
{"provingJobId":"1:BASE_PARITY:da8c40f85f52f6af"}
12:13:14 [12:13:14.880]
INFO:
prover-client:proving-broker Proving Broker started
12:13:14 [12:13:14.881]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3afd20f333d73702 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3afd20f333d73702"}
12:13:14 [12:13:14.889]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEzQuqW/1 with map size 1073741824
12:13:14 [12:13:14.891]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:14 [12:13:14.909]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c196872117aaa558 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c196872117aaa558"}
12:13:14 [12:13:14.910]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testEzQuqW/2 with map size 1073741824
12:13:14 [12:13:14.913]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:15 [12:13:15.981]
INFO:
prover-client:proving-broker Proving Broker started
12:13:15 [12:13:15.988]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3afd20f333d73702 settled=false
{"provingJobId":"1:BASE_PARITY:3afd20f333d73702","status":"pending"}
12:13:15 [12:13:15.990]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c196872117aaa558 settled=false
{"provingJobId":"1:BASE_PARITY:c196872117aaa558","status":"pending"}
12:13:15 [12:13:15.991]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3afd20f333d73702 type=BASE_PARITY startedAt=1750767194930 older agent has taken job
{"provingJobId":"1:BASE_PARITY:3afd20f333d73702"}
12:13:15 [12:13:15.991]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3afd20f333d73702 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:3afd20f333d73702"}
12:13:15 [12:13:15.996]
INFO:
prover-client:proving-broker Proving Broker started
12:13:15 [12:13:15.997]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5475433946322027 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5475433946322027"}
12:13:15 [12:13:15.998]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQKKGXd/1 with map size 1073741824
12:13:16 [12:13:16.002]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:16 [12:13:16.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3f4f73959efe339 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b3f4f73959efe339"}
12:13:16 [12:13:16.024]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testQKKGXd/2 with map size 1073741824
12:13:16 [12:13:16.027]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:17 [12:13:17.044]
INFO:
prover-client:proving-broker Proving Broker started
12:13:17 [12:13:17.046]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5475433946322027 settled=false
{"provingJobId":"1:BASE_PARITY:5475433946322027","status":"pending"}
12:13:17 [12:13:17.047]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b3f4f73959efe339 settled=false
{"provingJobId":"1:BASE_PARITY:b3f4f73959efe339","status":"pending"}
12:13:17 [12:13:17.048]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5475433946322027 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:5475433946322027"}
12:13:17 [12:13:17.052]
INFO:
prover-client:proving-broker Proving Broker started
12:13:17 [12:13:17.053]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:84b16cebb9fffcde epochNumber=1
{"provingJobId":"1:BASE_PARITY:84b16cebb9fffcde"}
12:13:17 [12:13:17.054]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwMNEAP/1 with map size 1073741824
12:13:17 [12:13:17.056]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:17 [12:13:17.076]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a1bca5bdcbf51394 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a1bca5bdcbf51394"}
12:13:17 [12:13:17.077]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwMNEAP/2 with map size 1073741824
12:13:17 [12:13:17.083]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.106]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.110]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:84b16cebb9fffcde settled=false
{"provingJobId":"1:BASE_PARITY:84b16cebb9fffcde","status":"pending"}
12:13:18 [12:13:18.114]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a1bca5bdcbf51394 settled=false
{"provingJobId":"1:BASE_PARITY:a1bca5bdcbf51394","status":"pending"}
12:13:18 [12:13:18.115]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:84b16cebb9fffcde type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:84b16cebb9fffcde"}
12:13:18 [12:13:18.115]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:84b16cebb9fffcde type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:84b16cebb9fffcde"}
12:13:18 [12:13:18.128]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.129]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6c97f80d2d85454 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6c97f80d2d85454"}
12:13:18 [12:13:18.129]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTqWNnl/1 with map size 1073741824
12:13:18 [12:13:18.131]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.149]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c06842de089e21b6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c06842de089e21b6"}
12:13:18 [12:13:18.150]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTqWNnl/2 with map size 1073741824
12:13:18 [12:13:18.152]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.171]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e6c97f80d2d85454 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e6c97f80d2d85454"}
12:13:18 [12:13:18.182]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c06842de089e21b6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c06842de089e21b6"}
12:13:18 [12:13:18.197]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.198]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0832b6f26d24442f epochNumber=1
{"provingJobId":"1:BASE_PARITY:0832b6f26d24442f"}
12:13:18 [12:13:18.199]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiF6VNJ/1 with map size 1073741824
12:13:18 [12:13:18.202]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.218]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:363ab1aed6ad9d55 epochNumber=2
{"provingJobId":"1:BASE_PARITY:363ab1aed6ad9d55"}
12:13:18 [12:13:18.219]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiF6VNJ/2 with map size 1073741824
12:13:18 [12:13:18.222]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.239]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0832b6f26d24442f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0832b6f26d24442f"}
12:13:18 [12:13:18.240]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0832b6f26d24442f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0832b6f26d24442f"}
12:13:18 [12:13:18.246]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:363ab1aed6ad9d55 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:363ab1aed6ad9d55"}
12:13:18 [12:13:18.258]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.259]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2e994af6e65963cc err=test error
{"provingJoId":"1:BASE_PARITY:2e994af6e65963cc"}
12:13:18 [12:13:18.261]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.262]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:55216b8ae9354426 not found
{"provingJobId":"1:BASE_PARITY:55216b8ae9354426"}
12:13:18 [12:13:18.265]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.266]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a49ccf1ef832ab74 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a49ccf1ef832ab74"}
12:13:18 [12:13:18.267]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPROibv/1 with map size 1073741824
12:13:18 [12:13:18.276]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.299]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.301]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b338bd559950cb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b338bd559950cb1"}
12:13:18 [12:13:18.301]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testr4RSme/1 with map size 1073741824
12:13:18 [12:13:18.304]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.426]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8b338bd559950cb1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:8b338bd559950cb1"}
12:13:18 [12:13:18.431]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.432]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:37a13e0414981277 epochNumber=1
{"provingJobId":"1:BASE_PARITY:37a13e0414981277"}
12:13:18 [12:13:18.433]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7FrXTt/1 with map size 1073741824
12:13:18 [12:13:18.434]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.558]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:37a13e0414981277 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:37a13e0414981277"}
12:13:18 [12:13:18.559]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a1098b3637770e89 epochNumber=10
{"provingJobId":"1:BASE_PARITY:a1098b3637770e89"}
12:13:18 [12:13:18.560]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test7FrXTt/10 with map size 1073741824
12:13:18 [12:13:18.562]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.584]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:18 [12:13:18.585]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
12:13:18 [12:13:18.712]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.713]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c04b3ab9939f355 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4c04b3ab9939f355"}
12:13:18 [12:13:18.714]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfJgSPa/1 with map size 1073741824
12:13:18 [12:13:18.716]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.889]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4c04b3ab9939f355 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:4c04b3ab9939f355"}
12:13:18 [12:13:18.942]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.943]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7357deaa87bd158 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d7357deaa87bd158"}
12:13:18 [12:13:18.944]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZOopUQ/1 with map size 1073741824
12:13:18 [12:13:18.946]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:18 [12:13:18.965]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d7357deaa87bd158 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:d7357deaa87bd158"}
12:13:18 [12:13:18.969]
INFO:
prover-client:proving-broker Proving Broker started
12:13:18 [12:13:18.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e9e11dbba387ae96 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e9e11dbba387ae96"}
12:13:18 [12:13:18.970]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQlrkoc/1 with map size 1073741824
12:13:18 [12:13:18.972]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.004]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e9e11dbba387ae96 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e9e11dbba387ae96"}
12:13:19 [12:13:19.004]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e9e11dbba387ae96 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:e9e11dbba387ae96"}
12:13:19 [12:13:19.014]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:307a7a216758ff9e epochNumber=1
{"provingJobId":"1:BASE_PARITY:307a7a216758ff9e"}
12:13:19 [12:13:19.016]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNiFOvF/1 with map size 1073741824
12:13:19 [12:13:19.018]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.038]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:307a7a216758ff9e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:307a7a216758ff9e"}
12:13:19 [12:13:19.048]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.049]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0bd300dab92f46f0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0bd300dab92f46f0"}
12:13:19 [12:13:19.050]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxAsVNQ/1 with map size 1073741824
12:13:19 [12:13:19.052]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.073]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0bd300dab92f46f0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:0bd300dab92f46f0"}
12:13:19 [12:13:19.074]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ec0f3c3af46906b epochNumber=3
{"provingJobId":"1:BASE_PARITY:5ec0f3c3af46906b"}
12:13:19 [12:13:19.075]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testxAsVNQ/3 with map size 1073741824
12:13:19 [12:13:19.084]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.127]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:19 [12:13:19.127]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
12:13:19 [12:13:19.128]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0bd300dab92f46f0 err=test error
{"provingJoId":"1:BASE_PARITY:0bd300dab92f46f0"}
12:13:19 [12:13:19.133]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPlEDYD/1 with map size 1073741824
12:13:19 [12:13:19.135]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.154]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPlEDYD/2 with map size 1073741824
12:13:19 [12:13:19.156]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.179]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.182]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:edb6c542dde652a0 settled=false
{"provingJobId":"1:BASE_PARITY:edb6c542dde652a0","status":"pending"}
12:13:19 [12:13:19.183]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6b2f0cc70098a776 settled=false
{"provingJobId":"1:BASE_PARITY:6b2f0cc70098a776","status":"pending"}
12:13:19 [12:13:19.191]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBWn95b/1 with map size 1073741824
12:13:19 [12:13:19.195]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.216]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBWn95b/2 with map size 1073741824
12:13:19 [12:13:19.220]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.250]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.255]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3943b30dda6fcf48 settled=true
{"provingJobId":"1:BASE_PARITY:3943b30dda6fcf48","status":"fulfilled"}
12:13:19 [12:13:19.256]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:cdb08b186d1ef87c settled=true
{"provingJobId":"2:BASE_PARITY:cdb08b186d1ef87c","status":"fulfilled"}
12:13:19 [12:13:19.265]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb58N6O/1 with map size 1073741824
12:13:19 [12:13:19.269]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.298]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testb58N6O/2 with map size 1073741824
12:13:19 [12:13:19.304]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.323]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.326]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e6207e39cbc1b500 settled=true
{"provingJobId":"1:BASE_PARITY:e6207e39cbc1b500","status":"fulfilled"}
12:13:19 [12:13:19.327]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ea44c5373ded62ba settled=false
{"provingJobId":"1:BASE_PARITY:ea44c5373ded62ba","status":"pending"}
12:13:19 [12:13:19.334]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.335]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31642511496fba0d epochNumber=1
{"provingJobId":"1:BASE_PARITY:31642511496fba0d"}
12:13:19 [12:13:19.336]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrOWuKG/1 with map size 1073741824
12:13:19 [12:13:19.338]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.361]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f1e0f4c3e781a0e epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f1e0f4c3e781a0e"}
12:13:19 [12:13:19.363]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:4f1e0f4c3e781a0e: Error: db error: Error: db error
12:13:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:4f1e0f4c3e781a0e"}
12:13:19 [12:13:19.367]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.368]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8498afbce284024c epochNumber=1
{"provingJobId":"1:BASE_PARITY:8498afbce284024c"}
12:13:19 [12:13:19.369]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvGnibI/1 with map size 1073741824
12:13:19 [12:13:19.372]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.391]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8498afbce284024c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8498afbce284024c"}
12:13:19 [12:13:19.392]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8498afbce284024c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8498afbce284024c"}
12:13:19 [12:13:19.409]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.410]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db6baa8c04b7b5f6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:db6baa8c04b7b5f6"}
12:13:19 [12:13:19.412]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testR3a3I0/1 with map size 1073741824
12:13:19 [12:13:19.414]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.431]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:db6baa8c04b7b5f6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:db6baa8c04b7b5f6"}
12:13:19 [12:13:19.432]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:db6baa8c04b7b5f6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:db6baa8c04b7b5f6"}
12:13:19 [12:13:19.432]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:db6baa8c04b7b5f6: Error: db error
12:13:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:db6baa8c04b7b5f6"}
12:13:19 [12:13:19.436]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.437]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4625db0635fa53a epochNumber=1
{"provingJobId":"1:BASE_PARITY:a4625db0635fa53a"}
12:13:19 [12:13:19.437]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCQiv1C/1 with map size 1073741824
12:13:19 [12:13:19.441]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.458]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a4625db0635fa53a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a4625db0635fa53a"}
12:13:19 [12:13:19.458]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a4625db0635fa53a type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a4625db0635fa53a"}
12:13:19 [12:13:19.479]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.480]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f0df52a3c179a2a1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f0df52a3c179a2a1"}
12:13:19 [12:13:19.481]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test352eh6/1 with map size 1073741824
12:13:19 [12:13:19.494]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.515]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f0df52a3c179a2a1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f0df52a3c179a2a1"}
12:13:19 [12:13:19.516]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f0df52a3c179a2a1 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f0df52a3c179a2a1"}
12:13:19 [12:13:19.518]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:f0df52a3c179a2a1 jobErr=test error: Error: db error
12:13:19 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:f0df52a3c179a2a1"}
12:13:19 [12:13:19.527]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.528]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9c9ca8dc4d0d4be3 not found
{"provingJobId":"1:BASE_PARITY:9c9ca8dc4d0d4be3"}
12:13:19 [12:13:19.533]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.534]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7246ef505ea5fbdd err=test error
{"provingJoId":"1:BASE_PARITY:7246ef505ea5fbdd"}
12:13:19 [12:13:19.539]
INFO:
prover-client:proving-broker Proving Broker started
12:13:19 [12:13:19.550]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54132b0c06d94f0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:54132b0c06d94f0f"}
12:13:19 [12:13:19.552]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzxOg71/1 with map size 1073741824
12:13:19 [12:13:19.555]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.579]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:54132b0c06d94f0f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:54132b0c06d94f0f"}
12:13:19 [12:13:19.580]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:54132b0c06d94f0f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:54132b0c06d94f0f"}
12:13:19 [12:13:19.598]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:19023af9befb05e9 epochNumber=2
{"provingJobId":"2:BASE_PARITY:19023af9befb05e9"}
12:13:19 [12:13:19.600]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzxOg71/2 with map size 1073741824
12:13:19 [12:13:19.602]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.620]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:19023af9befb05e9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:19023af9befb05e9"}
12:13:19 [12:13:19.620]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:19023af9befb05e9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:19023af9befb05e9"}
12:13:19 [12:13:19.687]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:00264109adcc279f epochNumber=3
{"provingJobId":"3:BASE_PARITY:00264109adcc279f"}
12:13:19 [12:13:19.689]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testzxOg71/3 with map size 1073741824
12:13:19 [12:13:19.691]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.721]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:19 [12:13:19.721]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
12:13:19 [12:13:19.775]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:c3bbdb06efc06a25 epochNumber=4
{"provingJobId":"4:BASE_PARITY:c3bbdb06efc06a25"}
12:13:19 [12:13:19.777]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testzxOg71/4 with map size 1073741824
12:13:19 [12:13:19.780]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.803]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:19 [12:13:19.803]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
12:13:19 [12:13:19.863]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:48056957567a8865 epochNumber=5
{"provingJobId":"5:BASE_PARITY:48056957567a8865"}
12:13:19 [12:13:19.864]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testzxOg71/5 with map size 1073741824
12:13:19 [12:13:19.869]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
12:13:19 [12:13:19.909]
INFO:
prover-client:proving-broker Cleaned up jobs=1
12:13:19 [12:13:19.909]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
12:13:19
PASS src/proving_broker/proving_broker.test.ts (
13.189 s)
12:13:19 ProvingBroker
12:13:19 Producer API
12:13:19
✓ refuses stale jobs (37 ms)
12:13:19
✓ enqueues jobs (4 ms)
12:13:19
✓ ignores duplicate jobs (4 ms)
12:13:19
✓ reports correct status when enqueuing repeat jobs (3 ms)
12:13:19
✓ throws an error in case of duplicate job IDs (24 ms)
12:13:19
✓ returns not-found status for non-existing jobs (1 ms)
12:13:19
✓ cancels jobs in queue (2 ms)
12:13:19
✓ cancels jobs in-progress (2 ms)
12:13:19
✓ returns job result if successful (12 ms)
12:13:19
✓ returns job error if failed (7 ms)
12:13:19
✓ correctly returns job status for concurrent writes (7 ms)
12:13:19
✓ refuses stale jobs (42 ms)
12:13:19
✓ enqueues jobs (33 ms)
12:13:19
✓ ignores duplicate jobs (26 ms)
12:13:19
✓ reports correct status when enqueuing repeat jobs (33 ms)
12:13:19
✓ throws an error in case of duplicate job IDs (34 ms)
12:13:19
✓ returns not-found status for non-existing jobs (3 ms)
12:13:19
✓ cancels jobs in queue (33 ms)
12:13:19
✓ cancels jobs in-progress (47 ms)
12:13:19
✓ returns job result if successful (28 ms)
12:13:19
✓ returns job error if failed (39 ms)
12:13:19
✓ correctly returns job status for concurrent writes (40 ms)
12:13:19 Consumer API
12:13:19
✓ returns undefined if no jobs are available (3 ms)
12:13:19
✓ returns jobs in priority order (3 ms)
12:13:19
✓ returns undefined if no jobs are available for the given allowList (2 ms)
12:13:19
✓ returns a job if it is in the allowList (3 ms)
12:13:19
✓ returns the most important job if it is in the allowList (6 ms)
12:13:19
✓ returns any job if filter is empty (9 ms)
12:13:19
✓ returns a new job when reporting job success (10 ms)
12:13:19
✓ returns a new job when reporting permanent error (5 ms)
12:13:19
✓ returns a new job when reporting retry-able error (2 ms)
12:13:19
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
12:13:19
✓ returns a new job if job is already in progress elsewhere (1061 ms)
12:13:19
✓ avoids sending the same job to a new agent after a restart (1005 ms)
12:13:19
✓ avoids sending a completed job to a new agent after a restart (1007 ms)
12:13:19
✓ tracks job result if in progress (3 ms)
12:13:19
✓ tracks job result even if job is in queue (3 ms)
12:13:19
✓ ignores reported job error if unknown job (1 ms)
12:13:19
✓ ignores job result if unknown job (2 ms)
12:13:19
✓ returns undefined if no jobs are available (2 ms)
12:13:19
✓ returns jobs in priority order (39 ms)
12:13:19
✓ returns undefined if no jobs are available for the given allowList (44 ms)
12:13:19
✓ returns a job if it is in the allowList (56 ms)
12:13:19
✓ returns the most important job if it is in the allowList (71 ms)
12:13:19
✓ returns any job if filter is empty (67 ms)
12:13:19
✓ returns a new job when reporting job success (40 ms)
12:13:19
✓ returns a new job when reporting permanent error (49 ms)
12:13:19
✓ returns a new job when reporting retry-able error (38 ms)
12:13:19
✓ returns a new job when reporting progress if current one is cancelled (44 ms)
12:13:19
✓ returns a new job if job is already in progress elsewhere (1116 ms)
12:13:19
✓ avoids sending the same job to a new agent after a restart (1056 ms)
12:13:19
✓ avoids sending a completed job to a new agent after a restart (1075 ms)
12:13:19
✓ tracks job result if in progress (67 ms)
12:13:19
✓ tracks job result even if job is in queue (62 ms)
12:13:19
✓ ignores reported job error if unknown job (3 ms)
12:13:19
✓ ignores job result if unknown job (2 ms)
12:13:19 Timeouts
12:13:19
✓ tracks in progress jobs (2 ms)
12:13:19
✓ re-enqueues jobs that time out (108 ms)
12:13:19
✓ cancel stale jobs that time out (234 ms)
12:13:19
✓ keeps the jobs in progress while it is alive (205 ms)
12:13:19
✓ tracks in progress jobs (33 ms)
12:13:19
✓ re-enqueues jobs that time out (132 ms)
12:13:19
✓ cancel stale jobs that time out (280 ms)
12:13:19
✓ keeps the jobs in progress while it is alive (229 ms)
12:13:19 Retries
12:13:19
✓ retries jobs (3 ms)
12:13:19
✓ retries up to a maximum number of times (3 ms)
12:13:19
✓ passing retry=false does not retry (2 ms)
12:13:19
✓ does not retry if job is stale (28 ms)
12:13:19
✓ retries jobs (27 ms)
12:13:19
✓ retries up to a maximum number of times (44 ms)
12:13:19
✓ passing retry=false does not retry (33 ms)
12:13:19
✓ does not retry if job is stale (85 ms)
12:13:19 Database management
12:13:19
✓ re-enqueues proof requests on start (3 ms)
12:13:19
✓ restores proof results on start (1 ms)
12:13:19
✓ only re-enqueues unfinished jobs (1 ms)
12:13:19
✓ saves job when enqueued (3 ms)
12:13:19
✓ does not retain job if database fails to save (2 ms)
12:13:19
✓ saves job result (1 ms)
12:13:19
✓ saves result even if database fails to save (2 ms)
12:13:19
✓ saves job error (3 ms)
12:13:19
✓ saves job error even if database fails to save (2 ms)
12:13:19
✓ does not save job result if job is unknown (2 ms)
12:13:19
✓ does not save job error if job is unknown (2 ms)
12:13:19
✓ cleans up old jobs periodically (261 ms)
12:13:19
✓ re-enqueues proof requests on start (56 ms)
12:13:19
✓ restores proof results on start (72 ms)
12:13:19
✓ only re-enqueues unfinished jobs (72 ms)
12:13:19
✓ saves job when enqueued (26 ms)
12:13:19
✓ does not retain job if database fails to save (7 ms)
12:13:19
✓ saves job result (41 ms)
12:13:19
✓ saves result even if database fails to save (28 ms)
12:13:19
✓ saves job error (36 ms)
12:13:19
✓ saves job error even if database fails to save (50 ms)
12:13:19
✓ does not save job result if job is unknown (9 ms)
12:13:19
✓ does not save job error if job is unknown (6 ms)
12:13:19
✓ cleans up old jobs periodically (413 ms)
12:13:19
12:13:19
Test Suites: 1 passed, 1 total
12:13:19
Tests: 96 passed, 96 total
12:13:19
Snapshots: 0 total
12:13:19
Time: 13.282 s
12:13:19
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
12:13:19
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?