Parent Log:
http://ci.aztec-labs.com/0cfbb3292f8dcdfc
Command: f30d64a7ce604682 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/7d1abcb337009939ee47da8445b5fee80f39fb58
Env: REF_NAME=gh-readonly-queue/next/pr-15156-df137f9440a970af2eb6e2ee94c48bf3bf47e207 CURRENT_VERSION=0.87.6 CI_FULL=0
Date: Fri Jun 20 10:06:43 UTC 2025
System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15156_arm64_a1-fast
Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
10:06:45 [10:06:45.980]
INFO:
prover-client:proving-broker Proving Broker started
10:06:45 [10:06:45.985]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dba89cb9cdb6268c epochNumber=42
{"provingJobId":"1:BASE_PARITY:dba89cb9cdb6268c"}
10:06:45 [10:06:45.988]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:03910316a982498d epochNumber=1
{"provingJobId":"1:BASE_PARITY:03910316a982498d"}
10:06:46 [10:06:46.013]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.014]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2564c1f65ecf8259 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2564c1f65ecf8259"}
10:06:46 [10:06:46.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a8ec9398b8ecc34 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a8ec9398b8ecc34"}
10:06:46 [10:06:46.017]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.018]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:449255ea405fcfcc epochNumber=1
{"provingJobId":"1:BASE_PARITY:449255ea405fcfcc"}
10:06:46 [10:06:46.019]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:449255ea405fcfcc epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:449255ea405fcfcc"}
10:06:46 [10:06:46.021]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f34d42d502efaf7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f34d42d502efaf7"}
10:06:46 [10:06:46.023]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0f34d42d502efaf7 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0f34d42d502efaf7"}
10:06:46 [10:06:46.023]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0f34d42d502efaf7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0f34d42d502efaf7"}
10:06:46 [10:06:46.024]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0f34d42d502efaf7 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0f34d42d502efaf7"}
10:06:46 [10:06:46.025]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.025]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc0a6c078c944453 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc0a6c078c944453"}
10:06:46 [10:06:46.044]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.045]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.046]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c0f941ab3958df1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c0f941ab3958df1"}
10:06:46 [10:06:46.046]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2c0f941ab3958df1
{"provingJobId":"1:BASE_PARITY:2c0f941ab3958df1"}
10:06:46 [10:06:46.047]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2c0f941ab3958df1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2c0f941ab3958df1"}
10:06:46 [10:06:46.047]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2c0f941ab3958df1 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:2c0f941ab3958df1"}
10:06:46 [10:06:46.048]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.049]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c50de964594f8f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c50de964594f8f5"}
10:06:46 [10:06:46.050]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5c50de964594f8f5
{"provingJobId":"1:BASE_PARITY:5c50de964594f8f5"}
10:06:46 [10:06:46.050]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5c50de964594f8f5 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:5c50de964594f8f5"}
10:06:46 [10:06:46.051]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.052]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bee4a0898dfb549b epochNumber=1
{"provingJobId":"1:BASE_PARITY:bee4a0898dfb549b"}
10:06:46 [10:06:46.052]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bee4a0898dfb549b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bee4a0898dfb549b"}
10:06:46 [10:06:46.053]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bee4a0898dfb549b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:bee4a0898dfb549b"}
10:06:46 [10:06:46.054]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:abc975829250bc4c epochNumber=1
{"provingJobId":"1:BASE_PARITY:abc975829250bc4c"}
10:06:46 [10:06:46.055]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:abc975829250bc4c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:abc975829250bc4c"}
10:06:46 [10:06:46.055]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:abc975829250bc4c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:abc975829250bc4c"}
10:06:46 [10:06:46.056]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b5686b33f607ec68 epochNumber=0
{"provingJobId":"1:BASE_PARITY:b5686b33f607ec68"}
10:06:46 [10:06:46.057]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b5686b33f607ec68 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b5686b33f607ec68"}
10:06:46 [10:06:46.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c60e0a50c7fce313 epochNumber=0
{"provingJobId":"1:BASE_PARITY:c60e0a50c7fce313"}
10:06:46 [10:06:46.058]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b5686b33f607ec68 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b5686b33f607ec68"}
10:06:46 [10:06:46.058]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:af6597bdffb0af4e epochNumber=0
{"provingJobId":"1:BASE_PARITY:af6597bdffb0af4e"}
10:06:46 [10:06:46.059]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.060]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.060]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1dfc8a51a7fc6ea3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1dfc8a51a7fc6ea3"}
10:06:46 [10:06:46.061]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62309afafe077052 epochNumber=1
{"provingJobId":"1:BASE_PARITY:62309afafe077052"}
10:06:46 [10:06:46.062]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.062]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c4f96ef632b9cae epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c4f96ef632b9cae"}
10:06:46 [10:06:46.068]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.068]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0947cd4d565e596 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b0947cd4d565e596"}
10:06:46 [10:06:46.068]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a4b8c0c1419b65f epochNumber=1
{"provingJobId":"1:BASE_PARITY:5a4b8c0c1419b65f"}
10:06:46 [10:06:46.069]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d78fd0e890ac7dca epochNumber=2
{"provingJobId":"1:BASE_PARITY:d78fd0e890ac7dca"}
10:06:46 [10:06:46.069]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9f08759480b36b06 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9f08759480b36b06"}
10:06:46 [10:06:46.070]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.070]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:03f1fee96c661605 epochNumber=1
{"provingJobId":"1:BASE_PARITY:03f1fee96c661605"}
10:06:46 [10:06:46.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:af337288989dfcde epochNumber=1
{"provingJobId":"1:BASE_PARITY:af337288989dfcde"}
10:06:46 [10:06:46.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:078318e1850af6c2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:078318e1850af6c2"}
10:06:46 [10:06:46.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:153437df6e1a8e10 epochNumber=1
{"provingJobId":"1:BASE_PARITY:153437df6e1a8e10"}
10:06:46 [10:06:46.072]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.072]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6216178e4ab764f4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6216178e4ab764f4"}
10:06:46 [10:06:46.073]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dba62640ec1c213e epochNumber=1
{"provingJobId":"1:BASE_PARITY:dba62640ec1c213e"}
10:06:46 [10:06:46.073]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6767332018b139b epochNumber=2
{"provingJobId":"1:BASE_PARITY:e6767332018b139b"}
10:06:46 [10:06:46.073]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:19de4e3f011a44f7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:19de4e3f011a44f7"}
10:06:46 [10:06:46.074]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.075]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b658bb59f68c3711 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b658bb59f68c3711"}
10:06:46 [10:06:46.075]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a78c394e68fa4a75 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a78c394e68fa4a75"}
10:06:46 [10:06:46.075]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b658bb59f68c3711 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b658bb59f68c3711"}
10:06:46 [10:06:46.076]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.077]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:65126bfa2c522376 epochNumber=1
{"provingJobId":"1:BASE_PARITY:65126bfa2c522376"}
10:06:46 [10:06:46.077]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:662c59909565e51c epochNumber=1
{"provingJobId":"1:BASE_PARITY:662c59909565e51c"}
10:06:46 [10:06:46.078]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:65126bfa2c522376 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:65126bfa2c522376"}
10:06:46 [10:06:46.078]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.079]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:13686699e1d1c5ee epochNumber=1
{"provingJobId":"1:BASE_PARITY:13686699e1d1c5ee"}
10:06:46 [10:06:46.079]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62085f6a5031a9d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:62085f6a5031a9d6"}
10:06:46 [10:06:46.080]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:13686699e1d1c5ee type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:13686699e1d1c5ee"}
10:06:46 [10:06:46.080]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.081]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f7b906addee9a058 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f7b906addee9a058"}
10:06:46 [10:06:46.081]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:f7b906addee9a058
{"provingJobId":"1:BASE_PARITY:f7b906addee9a058"}
10:06:46 [10:06:46.081]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f7b906addee9a058 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:f7b906addee9a058"}
10:06:46 [10:06:46.082]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ab7d847f713a5bf epochNumber=1
{"provingJobId":"1:BASE_PARITY:9ab7d847f713a5bf"}
10:06:46 [10:06:46.082]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f7b906addee9a058 has already been completed
{"provingJobId":"1:BASE_PARITY:f7b906addee9a058"}
10:06:46 [10:06:46.083]
INFO:
prover-client:proving-broker Proving Broker started
10:06:46 [10:06:46.084]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ec12167afe83cc4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ec12167afe83cc4"}
10:06:46 [10:06:46.084]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1dfc50f9aadc451 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c1dfc50f9aadc451"}
10:06:47 [10:06:47.135]
INFO:
prover-client:proving-broker Proving Broker started
10:06:47 [10:06:47.135]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2ec12167afe83cc4 settled=false
{"provingJobId":"1:BASE_PARITY:2ec12167afe83cc4","status":"pending"}
10:06:47 [10:06:47.136]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c1dfc50f9aadc451 settled=false
{"provingJobId":"1:BASE_PARITY:c1dfc50f9aadc451","status":"pending"}
10:06:47 [10:06:47.136]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2ec12167afe83cc4 type=BASE_PARITY startedAt=1750414006084 older agent has taken job
{"provingJobId":"1:BASE_PARITY:2ec12167afe83cc4"}
10:06:47 [10:06:47.137]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2ec12167afe83cc4 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:2ec12167afe83cc4"}
10:06:47 [10:06:47.138]
INFO:
prover-client:proving-broker Proving Broker started
10:06:47 [10:06:47.138]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09fb5fd1b3eaa481 epochNumber=1
{"provingJobId":"1:BASE_PARITY:09fb5fd1b3eaa481"}
10:06:47 [10:06:47.138]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc6d190c29f25557 epochNumber=2
{"provingJobId":"1:BASE_PARITY:bc6d190c29f25557"}
10:06:48 [10:06:48.139]
INFO:
prover-client:proving-broker Proving Broker started
10:06:48 [10:06:48.139]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:09fb5fd1b3eaa481 settled=false
{"provingJobId":"1:BASE_PARITY:09fb5fd1b3eaa481","status":"pending"}
10:06:48 [10:06:48.139]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bc6d190c29f25557 settled=false
{"provingJobId":"1:BASE_PARITY:bc6d190c29f25557","status":"pending"}
10:06:48 [10:06:48.140]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:09fb5fd1b3eaa481 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:09fb5fd1b3eaa481"}
10:06:48 [10:06:48.141]
INFO:
prover-client:proving-broker Proving Broker started
10:06:48 [10:06:48.141]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebac0d8a06911ae3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ebac0d8a06911ae3"}
10:06:48 [10:06:48.142]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b5a42eccfb36c8c2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b5a42eccfb36c8c2"}
10:06:49 [10:06:49.142]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.143]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ebac0d8a06911ae3 settled=false
{"provingJobId":"1:BASE_PARITY:ebac0d8a06911ae3","status":"pending"}
10:06:49 [10:06:49.143]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b5a42eccfb36c8c2 settled=false
{"provingJobId":"1:BASE_PARITY:b5a42eccfb36c8c2","status":"pending"}
10:06:49 [10:06:49.144]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ebac0d8a06911ae3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ebac0d8a06911ae3"}
10:06:49 [10:06:49.144]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ebac0d8a06911ae3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ebac0d8a06911ae3"}
10:06:49 [10:06:49.145]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.146]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:67787a5f7329e4c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:67787a5f7329e4c0"}
10:06:49 [10:06:49.146]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bbdab8cf2da60126 epochNumber=2
{"provingJobId":"1:BASE_PARITY:bbdab8cf2da60126"}
10:06:49 [10:06:49.146]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:67787a5f7329e4c0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:67787a5f7329e4c0"}
10:06:49 [10:06:49.147]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bbdab8cf2da60126 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:bbdab8cf2da60126"}
10:06:49 [10:06:49.148]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.148]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:733307d37ed4d8b5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:733307d37ed4d8b5"}
10:06:49 [10:06:49.148]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51e6569d579fa8d3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:51e6569d579fa8d3"}
10:06:49 [10:06:49.149]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:733307d37ed4d8b5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:733307d37ed4d8b5"}
10:06:49 [10:06:49.149]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:733307d37ed4d8b5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:733307d37ed4d8b5"}
10:06:49 [10:06:49.149]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:51e6569d579fa8d3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:51e6569d579fa8d3"}
10:06:49 [10:06:49.150]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.150]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b7a8461b5eae06fa err=test error
{"provingJoId":"1:BASE_PARITY:b7a8461b5eae06fa"}
10:06:49 [10:06:49.151]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.152]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:72817040b03f0d69 not found
{"provingJobId":"1:BASE_PARITY:72817040b03f0d69"}
10:06:49 [10:06:49.153]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.153]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c7e0ad8041c12f5f epochNumber=1
{"provingJobId":"1:BASE_PARITY:c7e0ad8041c12f5f"}
10:06:49 [10:06:49.154]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.155]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:23258f024ab37c9e epochNumber=1
{"provingJobId":"1:BASE_PARITY:23258f024ab37c9e"}
10:06:49 [10:06:49.256]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:23258f024ab37c9e timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:23258f024ab37c9e"}
10:06:49 [10:06:49.263]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.264]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2365d4b0f056619b epochNumber=1
{"provingJobId":"1:BASE_PARITY:2365d4b0f056619b"}
10:06:49 [10:06:49.364]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2365d4b0f056619b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2365d4b0f056619b"}
10:06:49 [10:06:49.370]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7627945e70b63677 epochNumber=10
{"provingJobId":"1:BASE_PARITY:7627945e70b63677"}
10:06:49 [10:06:49.390]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:49 [10:06:49.496]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.497]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:242a3192e3f4bbb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:242a3192e3f4bbb1"}
10:06:49 [10:06:49.673]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:242a3192e3f4bbb1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:242a3192e3f4bbb1"}
10:06:49 [10:06:49.699]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.700]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:396c0c0308ddf5f1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:396c0c0308ddf5f1"}
10:06:49 [10:06:49.700]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:396c0c0308ddf5f1 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:396c0c0308ddf5f1"}
10:06:49 [10:06:49.701]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.702]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6992091fc65a7907 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6992091fc65a7907"}
10:06:49 [10:06:49.702]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6992091fc65a7907 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:6992091fc65a7907"}
10:06:49 [10:06:49.703]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6992091fc65a7907 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:6992091fc65a7907"}
10:06:49 [10:06:49.704]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.704]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0946cc6397c0ebe6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0946cc6397c0ebe6"}
10:06:49 [10:06:49.705]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0946cc6397c0ebe6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0946cc6397c0ebe6"}
10:06:49 [10:06:49.705]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.706]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b43ce7dc102867b6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b43ce7dc102867b6"}
10:06:49 [10:06:49.706]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b43ce7dc102867b6 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:b43ce7dc102867b6"}
10:06:49 [10:06:49.707]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:13370ae19e271309 epochNumber=3
{"provingJobId":"1:BASE_PARITY:13370ae19e271309"}
10:06:49 [10:06:49.731]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:49 [10:06:49.732]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b43ce7dc102867b6 err=test error
{"provingJoId":"1:BASE_PARITY:b43ce7dc102867b6"}
10:06:49 [10:06:49.733]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.734]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ae9890ca6397b615 settled=false
{"provingJobId":"1:BASE_PARITY:ae9890ca6397b615","status":"pending"}
10:06:49 [10:06:49.734]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:61fe65fcb281d741 settled=false
{"provingJobId":"1:BASE_PARITY:61fe65fcb281d741","status":"pending"}
10:06:49 [10:06:49.736]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.736]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7ee77212c466d2c3 settled=true
{"provingJobId":"1:BASE_PARITY:7ee77212c466d2c3","status":"fulfilled"}
10:06:49 [10:06:49.736]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:2f83522e9ee76c70 settled=true
{"provingJobId":"2:BASE_PARITY:2f83522e9ee76c70","status":"fulfilled"}
10:06:49 [10:06:49.737]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.737]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f78212869a20ff6c settled=true
{"provingJobId":"1:BASE_PARITY:f78212869a20ff6c","status":"fulfilled"}
10:06:49 [10:06:49.738]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:213207bdb2ba0986 settled=false
{"provingJobId":"1:BASE_PARITY:213207bdb2ba0986","status":"pending"}
10:06:49 [10:06:49.739]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.739]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:92627b7af986e78f epochNumber=1
{"provingJobId":"1:BASE_PARITY:92627b7af986e78f"}
10:06:49 [10:06:49.741]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.741]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8487a7cb7aa0733e epochNumber=1
{"provingJobId":"1:BASE_PARITY:8487a7cb7aa0733e"}
10:06:49 [10:06:49.742]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:8487a7cb7aa0733e: Error: db error: Error: db error
10:06:49 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:8487a7cb7aa0733e"}
10:06:49 [10:06:49.743]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.743]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:869df5fd8f16f7f0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:869df5fd8f16f7f0"}
10:06:49 [10:06:49.744]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:869df5fd8f16f7f0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:869df5fd8f16f7f0"}
10:06:49 [10:06:49.744]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:869df5fd8f16f7f0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:869df5fd8f16f7f0"}
10:06:49 [10:06:49.745]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.745]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f7bd1341c54673c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f7bd1341c54673c5"}
10:06:49 [10:06:49.746]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f7bd1341c54673c5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f7bd1341c54673c5"}
10:06:49 [10:06:49.746]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f7bd1341c54673c5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f7bd1341c54673c5"}
10:06:49 [10:06:49.746]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:f7bd1341c54673c5: Error: db error
10:06:49 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:f7bd1341c54673c5"}
10:06:49 [10:06:49.747]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.748]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c77f05166342a3d1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c77f05166342a3d1"}
10:06:49 [10:06:49.748]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c77f05166342a3d1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c77f05166342a3d1"}
10:06:49 [10:06:49.748]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c77f05166342a3d1 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c77f05166342a3d1"}
10:06:49 [10:06:49.749]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.750]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd1e1c310e0188fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd1e1c310e0188fe"}
10:06:49 [10:06:49.750]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fd1e1c310e0188fe type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fd1e1c310e0188fe"}
10:06:49 [10:06:49.750]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fd1e1c310e0188fe type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fd1e1c310e0188fe"}
10:06:49 [10:06:49.751]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:fd1e1c310e0188fe jobErr=test error: Error: db error
10:06:49 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:fd1e1c310e0188fe"}
10:06:49 [10:06:49.752]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.752]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f767e57d50fe498b not found
{"provingJobId":"1:BASE_PARITY:f767e57d50fe498b"}
10:06:49 [10:06:49.755]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.756]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b252779c41d48d1c err=test error
{"provingJoId":"1:BASE_PARITY:b252779c41d48d1c"}
10:06:49 [10:06:49.757]
INFO:
prover-client:proving-broker Proving Broker started
10:06:49 [10:06:49.768]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e2cb09205cc179b epochNumber=1
{"provingJobId":"1:BASE_PARITY:6e2cb09205cc179b"}
10:06:49 [10:06:49.769]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6e2cb09205cc179b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6e2cb09205cc179b"}
10:06:49 [10:06:49.769]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6e2cb09205cc179b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6e2cb09205cc179b"}
10:06:49 [10:06:49.778]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:47f6571d5e826632 epochNumber=2
{"provingJobId":"2:BASE_PARITY:47f6571d5e826632"}
10:06:49 [10:06:49.779]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:47f6571d5e826632 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:47f6571d5e826632"}
10:06:49 [10:06:49.779]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:47f6571d5e826632 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:47f6571d5e826632"}
10:06:49 [10:06:49.840]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:ff9b359d0c194515 epochNumber=3
{"provingJobId":"3:BASE_PARITY:ff9b359d0c194515"}
10:06:49 [10:06:49.858]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:49 [10:06:49.901]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:591e15a767f88e51 epochNumber=4
{"provingJobId":"4:BASE_PARITY:591e15a767f88e51"}
10:06:49 [10:06:49.908]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:49 [10:06:49.963]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:50fe79bf940788b7 epochNumber=5
{"provingJobId":"5:BASE_PARITY:50fe79bf940788b7"}
10:06:49 [10:06:49.984]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:50 [10:06:50.016]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.016]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:92c372684521a866 epochNumber=42
{"provingJobId":"1:BASE_PARITY:92c372684521a866"}
10:06:50 [10:06:50.017]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testWd6jFi/42 with map size 1073741824
10:06:50 [10:06:50.020]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.041]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:4771cd9f5f08c2fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:4771cd9f5f08c2fd"}
10:06:50 [10:06:50.044]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.044]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4d4b4516895e81ed epochNumber=1
{"provingJobId":"1:BASE_PARITY:4d4b4516895e81ed"}
10:06:50 [10:06:50.045]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testylZyIf/1 with map size 1073741824
10:06:50 [10:06:50.046]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.061]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3c9e44f31dff6aa epochNumber=1
{"provingJobId":"1:BASE_PARITY:b3c9e44f31dff6aa"}
10:06:50 [10:06:50.070]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4958d2efda71326 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e4958d2efda71326"}
10:06:50 [10:06:50.071]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAwXwB3/1 with map size 1073741824
10:06:50 [10:06:50.073]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.091]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e4958d2efda71326 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e4958d2efda71326"}
10:06:50 [10:06:50.094]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.094]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0ae04e64d11d050 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b0ae04e64d11d050"}
10:06:50 [10:06:50.095]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmo5qgl/1 with map size 1073741824
10:06:50 [10:06:50.096]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.115]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b0ae04e64d11d050 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b0ae04e64d11d050"}
10:06:50 [10:06:50.116]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b0ae04e64d11d050 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b0ae04e64d11d050"}
10:06:50 [10:06:50.124]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b0ae04e64d11d050 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b0ae04e64d11d050"}
10:06:50 [10:06:50.129]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.130]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:742016bff4c66958 epochNumber=1
{"provingJobId":"1:BASE_PARITY:742016bff4c66958"}
10:06:50 [10:06:50.130]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1Xcb3s/1 with map size 1073741824
10:06:50 [10:06:50.133]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.155]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.157]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.157]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fbcc3fb8f5ed7a55 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fbcc3fb8f5ed7a55"}
10:06:50 [10:06:50.158]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJoJcRv/1 with map size 1073741824
10:06:50 [10:06:50.161]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.178]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:fbcc3fb8f5ed7a55
{"provingJobId":"1:BASE_PARITY:fbcc3fb8f5ed7a55"}
10:06:50 [10:06:50.178]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fbcc3fb8f5ed7a55 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fbcc3fb8f5ed7a55"}
10:06:50 [10:06:50.178]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fbcc3fb8f5ed7a55 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:fbcc3fb8f5ed7a55"}
10:06:50 [10:06:50.186]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.187]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:26023ade8ab79df8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:26023ade8ab79df8"}
10:06:50 [10:06:50.188]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2L3nuI/1 with map size 1073741824
10:06:50 [10:06:50.191]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.207]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:26023ade8ab79df8
{"provingJobId":"1:BASE_PARITY:26023ade8ab79df8"}
10:06:50 [10:06:50.208]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:26023ade8ab79df8 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:26023ade8ab79df8"}
10:06:50 [10:06:50.216]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.217]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:668221d6b78750ca epochNumber=1
{"provingJobId":"1:BASE_PARITY:668221d6b78750ca"}
10:06:50 [10:06:50.217]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHS09fT/1 with map size 1073741824
10:06:50 [10:06:50.220]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.236]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:668221d6b78750ca type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:668221d6b78750ca"}
10:06:50 [10:06:50.236]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:668221d6b78750ca type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:668221d6b78750ca"}
10:06:50 [10:06:50.243]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.243]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31d272170af290c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:31d272170af290c3"}
10:06:50 [10:06:50.244]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgpawy9/1 with map size 1073741824
10:06:50 [10:06:50.245]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.261]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:31d272170af290c3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:31d272170af290c3"}
10:06:50 [10:06:50.261]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:31d272170af290c3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:31d272170af290c3"}
10:06:50 [10:06:50.267]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.268]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6025285c8028083e epochNumber=0
{"provingJobId":"1:BASE_PARITY:6025285c8028083e"}
10:06:50 [10:06:50.268]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testwFYKVt/0 with map size 1073741824
10:06:50 [10:06:50.269]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.285]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6025285c8028083e epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6025285c8028083e"}
10:06:50 [10:06:50.285]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4bd7af21135a92b0 epochNumber=0
{"provingJobId":"1:BASE_PARITY:4bd7af21135a92b0"}
10:06:50 [10:06:50.285]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6025285c8028083e epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6025285c8028083e"}
10:06:50 [10:06:50.286]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4003fec123f28e9 epochNumber=0
{"provingJobId":"1:BASE_PARITY:a4003fec123f28e9"}
10:06:50 [10:06:50.297]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.301]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.301]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8cce662e709dbd12 epochNumber=2
{"provingJobId":"1:BASE_PARITY:8cce662e709dbd12"}
10:06:50 [10:06:50.303]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testWjTOfz/2 with map size 1073741824
10:06:50 [10:06:50.305]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.322]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5e622135d115dd04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5e622135d115dd04"}
10:06:50 [10:06:50.322]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWjTOfz/1 with map size 1073741824
10:06:50 [10:06:50.323]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.340]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.341]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6ce42e70297d12ef epochNumber=1
{"provingJobId":"1:BASE_PARITY:6ce42e70297d12ef"}
10:06:50 [10:06:50.341]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOpOInF/1 with map size 1073741824
10:06:50 [10:06:50.343]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.358]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.358]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f3ce5f90c39c0805 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f3ce5f90c39c0805"}
10:06:50 [10:06:50.359]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXYLgjg/1 with map size 1073741824
10:06:50 [10:06:50.360]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.374]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ac1f4d96a8f12a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3ac1f4d96a8f12a3"}
10:06:50 [10:06:50.379]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:566d026e2f96adcd epochNumber=2
{"provingJobId":"1:BASE_PARITY:566d026e2f96adcd"}
10:06:50 [10:06:50.380]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXYLgjg/2 with map size 1073741824
10:06:50 [10:06:50.382]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:96f674bc901993c6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:96f674bc901993c6"}
10:06:50 [10:06:50.411]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.411]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:56ef1799259e0515 epochNumber=1
{"provingJobId":"1:BASE_PARITY:56ef1799259e0515"}
10:06:50 [10:06:50.412]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test86Tz1y/1 with map size 1073741824
10:06:50 [10:06:50.413]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.432]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce8f77afa6c271f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce8f77afa6c271f8"}
10:06:50 [10:06:50.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f8c099d3f006c25b epochNumber=2
{"provingJobId":"1:BASE_PARITY:f8c099d3f006c25b"}
10:06:50 [10:06:50.440]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test86Tz1y/2 with map size 1073741824
10:06:50 [10:06:50.441]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.455]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8856b58c5ce5d851 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8856b58c5ce5d851"}
10:06:50 [10:06:50.462]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.462]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:56047a4e427e14b8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:56047a4e427e14b8"}
10:06:50 [10:06:50.463]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2egnlh/1 with map size 1073741824
10:06:50 [10:06:50.464]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.478]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bff479d9b2e45558 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bff479d9b2e45558"}
10:06:50 [10:06:50.483]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e1ce3c1bcf0533f epochNumber=2
{"provingJobId":"1:BASE_PARITY:0e1ce3c1bcf0533f"}
10:06:50 [10:06:50.484]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test2egnlh/2 with map size 1073741824
10:06:50 [10:06:50.486]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.501]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ed3612324d369f12 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ed3612324d369f12"}
10:06:50 [10:06:50.509]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.510]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd8382e77026a1d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd8382e77026a1d3"}
10:06:50 [10:06:50.511]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testka9FAX/1 with map size 1073741824
10:06:50 [10:06:50.513]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.529]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:41bef60e417a4e5e epochNumber=1
{"provingJobId":"1:BASE_PARITY:41bef60e417a4e5e"}
10:06:50 [10:06:50.534]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fd8382e77026a1d3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fd8382e77026a1d3"}
10:06:50 [10:06:50.541]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.542]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:901e5f5beb3a508f epochNumber=1
{"provingJobId":"1:BASE_PARITY:901e5f5beb3a508f"}
10:06:50 [10:06:50.542]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzgoNPm/1 with map size 1073741824
10:06:50 [10:06:50.544]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.558]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4320267a2bb131de epochNumber=1
{"provingJobId":"1:BASE_PARITY:4320267a2bb131de"}
10:06:50 [10:06:50.564]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:901e5f5beb3a508f type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:901e5f5beb3a508f"}
10:06:50 [10:06:50.570]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.571]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d04e54a576cfd1fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:d04e54a576cfd1fa"}
10:06:50 [10:06:50.571]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdsCqXc/1 with map size 1073741824
10:06:50 [10:06:50.574]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.588]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e2f6adcaeaceb28 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e2f6adcaeaceb28"}
10:06:50 [10:06:50.593]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d04e54a576cfd1fa type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:d04e54a576cfd1fa"}
10:06:50 [10:06:50.596]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.596]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54fbeef42db4e1f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:54fbeef42db4e1f5"}
10:06:50 [10:06:50.597]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl2U0f8/1 with map size 1073741824
10:06:50 [10:06:50.598]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.611]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:54fbeef42db4e1f5
{"provingJobId":"1:BASE_PARITY:54fbeef42db4e1f5"}
10:06:50 [10:06:50.611]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:54fbeef42db4e1f5 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:54fbeef42db4e1f5"}
10:06:50 [10:06:50.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1740c0361012dcd1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1740c0361012dcd1"}
10:06:50 [10:06:50.621]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:54fbeef42db4e1f5 has already been completed
{"provingJobId":"1:BASE_PARITY:54fbeef42db4e1f5"}
10:06:50 [10:06:50.624]
INFO:
prover-client:proving-broker Proving Broker started
10:06:50 [10:06:50.624]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d43fadb74b46ea9f epochNumber=1
{"provingJobId":"1:BASE_PARITY:d43fadb74b46ea9f"}
10:06:50 [10:06:50.625]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSFvMEB/1 with map size 1073741824
10:06:50 [10:06:50.626]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:50 [10:06:50.640]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:185ed4c849cc0c27 epochNumber=2
{"provingJobId":"1:BASE_PARITY:185ed4c849cc0c27"}
10:06:50 [10:06:50.641]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSFvMEB/2 with map size 1073741824
10:06:50 [10:06:50.642]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:51 [10:06:51.707]
INFO:
prover-client:proving-broker Proving Broker started
10:06:51 [10:06:51.711]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d43fadb74b46ea9f settled=false
{"provingJobId":"1:BASE_PARITY:d43fadb74b46ea9f","status":"pending"}
10:06:51 [10:06:51.712]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:185ed4c849cc0c27 settled=false
{"provingJobId":"1:BASE_PARITY:185ed4c849cc0c27","status":"pending"}
10:06:51 [10:06:51.713]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d43fadb74b46ea9f type=BASE_PARITY startedAt=1750414010655 older agent has taken job
{"provingJobId":"1:BASE_PARITY:d43fadb74b46ea9f"}
10:06:51 [10:06:51.713]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d43fadb74b46ea9f type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:d43fadb74b46ea9f"}
10:06:51 [10:06:51.717]
INFO:
prover-client:proving-broker Proving Broker started
10:06:51 [10:06:51.718]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8599c5677614399 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8599c5677614399"}
10:06:51 [10:06:51.720]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsIgjka/1 with map size 1073741824
10:06:51 [10:06:51.723]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:51 [10:06:51.742]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:44b64dd89d3f01d0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:44b64dd89d3f01d0"}
10:06:51 [10:06:51.746]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testsIgjka/2 with map size 1073741824
10:06:51 [10:06:51.749]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:52 [10:06:52.768]
INFO:
prover-client:proving-broker Proving Broker started
10:06:52 [10:06:52.769]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d8599c5677614399 settled=false
{"provingJobId":"1:BASE_PARITY:d8599c5677614399","status":"pending"}
10:06:52 [10:06:52.771]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:44b64dd89d3f01d0 settled=false
{"provingJobId":"1:BASE_PARITY:44b64dd89d3f01d0","status":"pending"}
10:06:52 [10:06:52.771]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d8599c5677614399 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:d8599c5677614399"}
10:06:52 [10:06:52.774]
INFO:
prover-client:proving-broker Proving Broker started
10:06:52 [10:06:52.775]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc20a956ea1b22c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc20a956ea1b22c3"}
10:06:52 [10:06:52.775]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSDt1EM/1 with map size 1073741824
10:06:52 [10:06:52.777]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:52 [10:06:52.793]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb505b09416be043 epochNumber=2
{"provingJobId":"1:BASE_PARITY:bb505b09416be043"}
10:06:52 [10:06:52.793]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSDt1EM/2 with map size 1073741824
10:06:52 [10:06:52.795]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.809]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.811]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fc20a956ea1b22c3 settled=false
{"provingJobId":"1:BASE_PARITY:fc20a956ea1b22c3","status":"pending"}
10:06:53 [10:06:53.813]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bb505b09416be043 settled=false
{"provingJobId":"1:BASE_PARITY:bb505b09416be043","status":"pending"}
10:06:53 [10:06:53.814]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fc20a956ea1b22c3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fc20a956ea1b22c3"}
10:06:53 [10:06:53.814]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fc20a956ea1b22c3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fc20a956ea1b22c3"}
10:06:53 [10:06:53.822]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.823]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5279232b9aa56a04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5279232b9aa56a04"}
10:06:53 [10:06:53.823]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvPb2N0/1 with map size 1073741824
10:06:53 [10:06:53.824]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.837]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:65320e33faca4029 epochNumber=2
{"provingJobId":"1:BASE_PARITY:65320e33faca4029"}
10:06:53 [10:06:53.838]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testvPb2N0/2 with map size 1073741824
10:06:53 [10:06:53.839]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.855]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5279232b9aa56a04 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5279232b9aa56a04"}
10:06:53 [10:06:53.860]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:65320e33faca4029 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:65320e33faca4029"}
10:06:53 [10:06:53.868]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.869]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:542bcf4f2c484960 epochNumber=1
{"provingJobId":"1:BASE_PARITY:542bcf4f2c484960"}
10:06:53 [10:06:53.870]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrzJ2xq/1 with map size 1073741824
10:06:53 [10:06:53.871]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.885]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9887082e0736cac epochNumber=2
{"provingJobId":"1:BASE_PARITY:f9887082e0736cac"}
10:06:53 [10:06:53.886]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrzJ2xq/2 with map size 1073741824
10:06:53 [10:06:53.887]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.901]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:542bcf4f2c484960 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:542bcf4f2c484960"}
10:06:53 [10:06:53.901]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:542bcf4f2c484960 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:542bcf4f2c484960"}
10:06:53 [10:06:53.907]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f9887082e0736cac type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f9887082e0736cac"}
10:06:53 [10:06:53.915]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.916]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8f48553eff464d29 err=test error
{"provingJoId":"1:BASE_PARITY:8f48553eff464d29"}
10:06:53 [10:06:53.918]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.919]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6026b7da69910fba not found
{"provingJobId":"1:BASE_PARITY:6026b7da69910fba"}
10:06:53 [10:06:53.922]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:86dd0c50687b5770 epochNumber=1
{"provingJobId":"1:BASE_PARITY:86dd0c50687b5770"}
10:06:53 [10:06:53.925]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9FUIPe/1 with map size 1073741824
10:06:53 [10:06:53.926]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:53 [10:06:53.943]
INFO:
prover-client:proving-broker Proving Broker started
10:06:53 [10:06:53.943]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:69dd4eff5d87247a epochNumber=1
{"provingJobId":"1:BASE_PARITY:69dd4eff5d87247a"}
10:06:53 [10:06:53.944]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgjMAlv/1 with map size 1073741824
10:06:53 [10:06:53.945]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.070]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:69dd4eff5d87247a timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:69dd4eff5d87247a"}
10:06:54 [10:06:54.075]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.075]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eba3604babebb5f1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:eba3604babebb5f1"}
10:06:54 [10:06:54.076]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testC1kcbg/1 with map size 1073741824
10:06:54 [10:06:54.077]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.204]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eba3604babebb5f1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:eba3604babebb5f1"}
10:06:54 [10:06:54.211]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:63f17c289af0e570 epochNumber=10
{"provingJobId":"1:BASE_PARITY:63f17c289af0e570"}
10:06:54 [10:06:54.211]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testC1kcbg/10 with map size 1073741824
10:06:54 [10:06:54.213]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.254]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:54 [10:06:54.254]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:06:54 [10:06:54.358]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.359]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91413bb4ad41dcd9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:91413bb4ad41dcd9"}
10:06:54 [10:06:54.359]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDFHp8R/1 with map size 1073741824
10:06:54 [10:06:54.360]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.536]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:91413bb4ad41dcd9 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:91413bb4ad41dcd9"}
10:06:54 [10:06:54.579]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.580]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e8ae605bde62bb42 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e8ae605bde62bb42"}
10:06:54 [10:06:54.580]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVQQpUp/1 with map size 1073741824
10:06:54 [10:06:54.581]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.598]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e8ae605bde62bb42 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e8ae605bde62bb42"}
10:06:54 [10:06:54.601]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.601]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8de02cdc9195130d epochNumber=1
{"provingJobId":"1:BASE_PARITY:8de02cdc9195130d"}
10:06:54 [10:06:54.602]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlngl4M/1 with map size 1073741824
10:06:54 [10:06:54.603]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.618]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8de02cdc9195130d type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:8de02cdc9195130d"}
10:06:54 [10:06:54.619]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8de02cdc9195130d type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:8de02cdc9195130d"}
10:06:54 [10:06:54.626]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.627]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:88c1ae669769fd22 epochNumber=1
{"provingJobId":"1:BASE_PARITY:88c1ae669769fd22"}
10:06:54 [10:06:54.627]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test16PNQd/1 with map size 1073741824
10:06:54 [10:06:54.629]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.643]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:88c1ae669769fd22 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:88c1ae669769fd22"}
10:06:54 [10:06:54.651]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.652]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25b5887e8cd3cf5a epochNumber=1
{"provingJobId":"1:BASE_PARITY:25b5887e8cd3cf5a"}
10:06:54 [10:06:54.653]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testds52vn/1 with map size 1073741824
10:06:54 [10:06:54.654]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.668]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:25b5887e8cd3cf5a type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:25b5887e8cd3cf5a"}
10:06:54 [10:06:54.668]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:541e94f1f32d6986 epochNumber=3
{"provingJobId":"1:BASE_PARITY:541e94f1f32d6986"}
10:06:54 [10:06:54.669]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testds52vn/3 with map size 1073741824
10:06:54 [10:06:54.670]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.701]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:54 [10:06:54.702]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:06:54 [10:06:54.709]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:25b5887e8cd3cf5a err=test error
{"provingJoId":"1:BASE_PARITY:25b5887e8cd3cf5a"}
10:06:54 [10:06:54.712]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0nblp4/1 with map size 1073741824
10:06:54 [10:06:54.713]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.727]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0nblp4/2 with map size 1073741824
10:06:54 [10:06:54.729]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.742]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.743]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bf79872d272ae5b0 settled=false
{"provingJobId":"1:BASE_PARITY:bf79872d272ae5b0","status":"pending"}
10:06:54 [10:06:54.744]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d2d49aa327f47571 settled=false
{"provingJobId":"1:BASE_PARITY:d2d49aa327f47571","status":"pending"}
10:06:54 [10:06:54.749]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test43q5in/1 with map size 1073741824
10:06:54 [10:06:54.751]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.765]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test43q5in/2 with map size 1073741824
10:06:54 [10:06:54.766]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.789]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.790]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:58187476cd916f7c settled=true
{"provingJobId":"1:BASE_PARITY:58187476cd916f7c","status":"fulfilled"}
10:06:54 [10:06:54.792]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:0c2306485ce0904e settled=true
{"provingJobId":"2:BASE_PARITY:0c2306485ce0904e","status":"fulfilled"}
10:06:54 [10:06:54.796]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjGBFJh/1 with map size 1073741824
10:06:54 [10:06:54.798]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.817]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testjGBFJh/2 with map size 1073741824
10:06:54 [10:06:54.818]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.832]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.833]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:45bd023c4285e204 settled=true
{"provingJobId":"1:BASE_PARITY:45bd023c4285e204","status":"fulfilled"}
10:06:54 [10:06:54.836]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d3d749e4e1b25e9d settled=false
{"provingJobId":"1:BASE_PARITY:d3d749e4e1b25e9d","status":"pending"}
10:06:54 [10:06:54.840]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.840]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5e7501fef8443b61 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5e7501fef8443b61"}
10:06:54 [10:06:54.841]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testatPfQ2/1 with map size 1073741824
10:06:54 [10:06:54.842]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.858]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.859]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7fd4be259e660a27 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7fd4be259e660a27"}
10:06:54 [10:06:54.859]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:7fd4be259e660a27: Error: db error: Error: db error
10:06:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:7fd4be259e660a27"}
10:06:54 [10:06:54.862]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.863]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f04c9de916777410 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f04c9de916777410"}
10:06:54 [10:06:54.863]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHjxfVo/1 with map size 1073741824
10:06:54 [10:06:54.865]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.880]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f04c9de916777410 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f04c9de916777410"}
10:06:54 [10:06:54.880]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f04c9de916777410 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f04c9de916777410"}
10:06:54 [10:06:54.887]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.888]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0af3ce5528dcd13 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0af3ce5528dcd13"}
10:06:54 [10:06:54.888]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpAfUW5/1 with map size 1073741824
10:06:54 [10:06:54.890]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.907]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e0af3ce5528dcd13 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e0af3ce5528dcd13"}
10:06:54 [10:06:54.908]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e0af3ce5528dcd13 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e0af3ce5528dcd13"}
10:06:54 [10:06:54.909]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:e0af3ce5528dcd13: Error: db error
10:06:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:e0af3ce5528dcd13"}
10:06:54 [10:06:54.912]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.913]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:909e0d1a849b3bf1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:909e0d1a849b3bf1"}
10:06:54 [10:06:54.913]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5rJtWR/1 with map size 1073741824
10:06:54 [10:06:54.914]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.931]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:909e0d1a849b3bf1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:909e0d1a849b3bf1"}
10:06:54 [10:06:54.931]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:909e0d1a849b3bf1 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:909e0d1a849b3bf1"}
10:06:54 [10:06:54.940]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c548bb7163798572 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c548bb7163798572"}
10:06:54 [10:06:54.942]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7Hlmz0/1 with map size 1073741824
10:06:54 [10:06:54.943]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.957]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c548bb7163798572 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c548bb7163798572"}
10:06:54 [10:06:54.958]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c548bb7163798572 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c548bb7163798572"}
10:06:54 [10:06:54.958]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:c548bb7163798572 jobErr=test error: Error: db error
10:06:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:c548bb7163798572"}
10:06:54 [10:06:54.962]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.962]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8983193abdaee1e0 not found
{"provingJobId":"1:BASE_PARITY:8983193abdaee1e0"}
10:06:54 [10:06:54.965]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.966]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:65ffeec93672246c err=test error
{"provingJoId":"1:BASE_PARITY:65ffeec93672246c"}
10:06:54 [10:06:54.968]
INFO:
prover-client:proving-broker Proving Broker started
10:06:54 [10:06:54.979]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d4cc378f45b8d004 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d4cc378f45b8d004"}
10:06:54 [10:06:54.980]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtOdpfB/1 with map size 1073741824
10:06:54 [10:06:54.982]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:54 [10:06:54.996]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d4cc378f45b8d004 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d4cc378f45b8d004"}
10:06:54 [10:06:54.997]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d4cc378f45b8d004 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d4cc378f45b8d004"}
10:06:55 [10:06:55.011]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:fffcb0d05804f460 epochNumber=2
{"provingJobId":"2:BASE_PARITY:fffcb0d05804f460"}
10:06:55 [10:06:55.012]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtOdpfB/2 with map size 1073741824
10:06:55 [10:06:55.013]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:55 [10:06:55.028]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:fffcb0d05804f460 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:fffcb0d05804f460"}
10:06:55 [10:06:55.028]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:fffcb0d05804f460 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:fffcb0d05804f460"}
10:06:55 [10:06:55.095]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:e146d9fe317f303d epochNumber=3
{"provingJobId":"3:BASE_PARITY:e146d9fe317f303d"}
10:06:55 [10:06:55.096]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testtOdpfB/3 with map size 1073741824
10:06:55 [10:06:55.097]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:55 [10:06:55.121]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:55 [10:06:55.121]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:06:55 [10:06:55.175]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:42c10cb43b00d0fa epochNumber=4
{"provingJobId":"4:BASE_PARITY:42c10cb43b00d0fa"}
10:06:55 [10:06:55.176]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testtOdpfB/4 with map size 1073741824
10:06:55 [10:06:55.177]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:55 [10:06:55.198]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:55 [10:06:55.198]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
10:06:55 [10:06:55.253]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:021b05e4b9e44229 epochNumber=5
{"provingJobId":"5:BASE_PARITY:021b05e4b9e44229"}
10:06:55 [10:06:55.253]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testtOdpfB/5 with map size 1073741824
10:06:55 [10:06:55.255]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:06:55 [10:06:55.276]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:06:55 [10:06:55.276]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
10:06:55
PASS src/proving_broker/proving_broker.test.ts (
11.23 s)
10:06:55 ProvingBroker
10:06:55 Producer API
10:06:55
✓ refuses stale jobs (35 ms)
10:06:55
✓ enqueues jobs (4 ms)
10:06:55
✓ ignores duplicate jobs (3 ms)
10:06:55
✓ reports correct status when enqueuing repeat jobs (3 ms)
10:06:55
✓ throws an error in case of duplicate job IDs (18 ms)
10:06:55
✓ returns not-found status for non-existing jobs (2 ms)
10:06:55
✓ cancels jobs in queue (3 ms)
10:06:55
✓ cancels jobs in-progress (2 ms)
10:06:55
✓ returns job result if successful (3 ms)
10:06:55
✓ returns job error if failed (2 ms)
10:06:55
✓ correctly returns job status for concurrent writes (2 ms)
10:06:55
✓ refuses stale jobs (29 ms)
10:06:55
✓ enqueues jobs (25 ms)
10:06:55
✓ ignores duplicate jobs (24 ms)
10:06:55
✓ reports correct status when enqueuing repeat jobs (32 ms)
10:06:55
✓ throws an error in case of duplicate job IDs (28 ms)
10:06:55
✓ returns not-found status for non-existing jobs (2 ms)
10:06:55
✓ cancels jobs in queue (29 ms)
10:06:55
✓ cancels jobs in-progress (30 ms)
10:06:55
✓ returns job result if successful (27 ms)
10:06:55
✓ returns job error if failed (24 ms)
10:06:55
✓ correctly returns job status for concurrent writes (29 ms)
10:06:55 Consumer API
10:06:55
✓ returns undefined if no jobs are available (1 ms)
10:06:55
✓ returns jobs in priority order (1 ms)
10:06:55
✓ returns undefined if no jobs are available for the given allowList (5 ms)
10:06:55
✓ returns a job if it is in the allowList (3 ms)
10:06:55
✓ returns the most important job if it is in the allowList (2 ms)
10:06:55
✓ returns any job if filter is empty (2 ms)
10:06:55
✓ returns a new job when reporting job success (2 ms)
10:06:55
✓ returns a new job when reporting permanent error (2 ms)
10:06:55
✓ returns a new job when reporting retry-able error (2 ms)
10:06:55
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
10:06:55
✓ returns a new job if job is already in progress elsewhere (1054 ms)
10:06:55
✓ avoids sending the same job to a new agent after a restart (1003 ms)
10:06:55
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
10:06:55
✓ tracks job result if in progress (2 ms)
10:06:55
✓ tracks job result even if job is in queue (2 ms)
10:06:55
✓ ignores reported job error if unknown job (1 ms)
10:06:55
✓ ignores job result if unknown job (1 ms)
10:06:55
✓ returns undefined if no jobs are available (2 ms)
10:06:55
✓ returns jobs in priority order (41 ms)
10:06:55
✓ returns undefined if no jobs are available for the given allowList (18 ms)
10:06:55
✓ returns a job if it is in the allowList (53 ms)
10:06:55
✓ returns the most important job if it is in the allowList (51 ms)
10:06:55
✓ returns any job if filter is empty (47 ms)
10:06:55
✓ returns a new job when reporting job success (32 ms)
10:06:55
✓ returns a new job when reporting permanent error (29 ms)
10:06:55
✓ returns a new job when reporting retry-able error (25 ms)
10:06:55
✓ returns a new job when reporting progress if current one is cancelled (28 ms)
10:06:55
✓ returns a new job if job is already in progress elsewhere (1091 ms)
10:06:55
✓ avoids sending the same job to a new agent after a restart (1058 ms)
10:06:55
✓ avoids sending a completed job to a new agent after a restart (1047 ms)
10:06:55
✓ tracks job result if in progress (46 ms)
10:06:55
✓ tracks job result even if job is in queue (46 ms)
10:06:55
✓ ignores reported job error if unknown job (2 ms)
10:06:55
✓ ignores job result if unknown job (3 ms)
10:06:55 Timeouts
10:06:55
✓ tracks in progress jobs (2 ms)
10:06:55
✓ re-enqueues jobs that time out (109 ms)
10:06:55
✓ cancel stale jobs that time out (233 ms)
10:06:55
✓ keeps the jobs in progress while it is alive (203 ms)
10:06:55
✓ tracks in progress jobs (21 ms)
10:06:55
✓ re-enqueues jobs that time out (131 ms)
10:06:55
✓ cancel stale jobs that time out (282 ms)
10:06:55
✓ keeps the jobs in progress while it is alive (221 ms)
10:06:55 Retries
10:06:55
✓ retries jobs (2 ms)
10:06:55
✓ retries up to a maximum number of times (2 ms)
10:06:55
✓ passing retry=false does not retry (2 ms)
10:06:55
✓ does not retry if job is stale (28 ms)
10:06:55
✓ retries jobs (22 ms)
10:06:55
✓ retries up to a maximum number of times (26 ms)
10:06:55
✓ passing retry=false does not retry (24 ms)
10:06:55
✓ does not retry if job is stale (61 ms)
10:06:55 Database management
10:06:55
✓ re-enqueues proof requests on start (2 ms)
10:06:55
✓ restores proof results on start (2 ms)
10:06:55
✓ only re-enqueues unfinished jobs (1 ms)
10:06:55
✓ saves job when enqueued (2 ms)
10:06:55
✓ does not retain job if database fails to save (2 ms)
10:06:55
✓ saves job result (1 ms)
10:06:55
✓ saves result even if database fails to save (2 ms)
10:06:55
✓ saves job error (2 ms)
10:06:55
✓ saves job error even if database fails to save (2 ms)
10:06:55
✓ does not save job result if job is unknown (4 ms)
10:06:55
✓ does not save job error if job is unknown (2 ms)
10:06:55
✓ cleans up old jobs periodically (257 ms)
10:06:55
✓ re-enqueues proof requests on start (36 ms)
10:06:55
✓ restores proof results on start (47 ms)
10:06:55
✓ only re-enqueues unfinished jobs (44 ms)
10:06:55
✓ saves job when enqueued (18 ms)
10:06:55
✓ does not retain job if database fails to save (4 ms)
10:06:55
✓ saves job result (24 ms)
10:06:55
✓ saves result even if database fails to save (24 ms)
10:06:55
✓ saves job error (27 ms)
10:06:55
✓ saves job error even if database fails to save (22 ms)
10:06:55
✓ does not save job result if job is unknown (3 ms)
10:06:55
✓ does not save job error if job is unknown (3 ms)
10:06:55
✓ cleans up old jobs periodically (354 ms)
10:06:55
10:06:55
Test Suites: 1 passed, 1 total
10:06:55
Tests: 96 passed, 96 total
10:06:55
Snapshots: 0 total
10:06:55
Time: 11.301 s
10:06:55
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
10:06:55
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?