Parent Log:
http://ci.aztec-labs.com/f820f650f8909c52
Command: bcc565b00bd8bb22 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/bcb41f0bbab679d1b5d50e6b59b4af0faeeb9b48
Env: REF_NAME=gh-readonly-queue/next/pr-15138-3c671e0822075208f705135771771f5a47e9652e CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Wed Jun 18 16:17:55 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15138_amd64_x1-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
16:18:00 [16:18:00.248]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5eb3197fd63416b4 epochNumber=42
{"provingJobId":"1:BASE_PARITY:5eb3197fd63416b4"}
16:18:00 [16:18:00.257]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:5747dd5ae0392a23 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5747dd5ae0392a23"}
16:18:00 [16:18:00.289]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.290]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cbdde8ae2df23365 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cbdde8ae2df23365"}
16:18:00 [16:18:00.291]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:671c4611d490ab56 epochNumber=1
{"provingJobId":"1:BASE_PARITY:671c4611d490ab56"}
16:18:00 [16:18:00.293]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.294]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fb3a7e55ea20bb4f epochNumber=1
{"provingJobId":"1:BASE_PARITY:fb3a7e55ea20bb4f"}
16:18:00 [16:18:00.295]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:fb3a7e55ea20bb4f epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:fb3a7e55ea20bb4f"}
16:18:00 [16:18:00.297]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.298]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a39447399696d14 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a39447399696d14"}
16:18:00 [16:18:00.299]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3a39447399696d14 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3a39447399696d14"}
16:18:00 [16:18:00.300]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3a39447399696d14 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3a39447399696d14"}
16:18:00 [16:18:00.300]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3a39447399696d14 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3a39447399696d14"}
16:18:00 [16:18:00.302]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.302]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cff0c744b3d59bed epochNumber=1
{"provingJobId":"1:BASE_PARITY:cff0c744b3d59bed"}
16:18:00 [16:18:00.327]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.328]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.329]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5214ccdf1148205d epochNumber=1
{"provingJobId":"1:BASE_PARITY:5214ccdf1148205d"}
16:18:00 [16:18:00.330]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5214ccdf1148205d
{"provingJobId":"1:BASE_PARITY:5214ccdf1148205d"}
16:18:00 [16:18:00.330]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5214ccdf1148205d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5214ccdf1148205d"}
16:18:00 [16:18:00.330]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5214ccdf1148205d type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:5214ccdf1148205d"}
16:18:00 [16:18:00.332]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31426df1f6645725 epochNumber=1
{"provingJobId":"1:BASE_PARITY:31426df1f6645725"}
16:18:00 [16:18:00.333]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:31426df1f6645725
{"provingJobId":"1:BASE_PARITY:31426df1f6645725"}
16:18:00 [16:18:00.333]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:31426df1f6645725 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:31426df1f6645725"}
16:18:00 [16:18:00.335]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.336]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:08e83a7d837775eb epochNumber=1
{"provingJobId":"1:BASE_PARITY:08e83a7d837775eb"}
16:18:00 [16:18:00.336]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:08e83a7d837775eb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:08e83a7d837775eb"}
16:18:00 [16:18:00.336]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:08e83a7d837775eb type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:08e83a7d837775eb"}
16:18:00 [16:18:00.338]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.338]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:72c2388f74d5f45e epochNumber=1
{"provingJobId":"1:BASE_PARITY:72c2388f74d5f45e"}
16:18:00 [16:18:00.339]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:72c2388f74d5f45e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:72c2388f74d5f45e"}
16:18:00 [16:18:00.339]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:72c2388f74d5f45e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:72c2388f74d5f45e"}
16:18:00 [16:18:00.340]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:65abed27313e30b3 epochNumber=0
{"provingJobId":"1:BASE_PARITY:65abed27313e30b3"}
16:18:00 [16:18:00.341]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:65abed27313e30b3 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:65abed27313e30b3"}
16:18:00 [16:18:00.341]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5919e9d2db8a2c31 epochNumber=0
{"provingJobId":"1:BASE_PARITY:5919e9d2db8a2c31"}
16:18:00 [16:18:00.341]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:65abed27313e30b3 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:65abed27313e30b3"}
16:18:00 [16:18:00.341]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e97ca69514f076a9 epochNumber=0
{"provingJobId":"1:BASE_PARITY:e97ca69514f076a9"}
16:18:00 [16:18:00.343]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.344]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.344]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e909a3f7094cd91c epochNumber=2
{"provingJobId":"1:BASE_PARITY:e909a3f7094cd91c"}
16:18:00 [16:18:00.344]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f20fa83fe7d06f2c epochNumber=1
{"provingJobId":"1:BASE_PARITY:f20fa83fe7d06f2c"}
16:18:00 [16:18:00.346]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.346]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4394651fe36ea85 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e4394651fe36ea85"}
16:18:00 [16:18:00.353]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.354]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0cae891207c0183 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0cae891207c0183"}
16:18:00 [16:18:00.354]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc588a240bfcc195 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc588a240bfcc195"}
16:18:00 [16:18:00.355]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0530ac82bf3ad80c epochNumber=2
{"provingJobId":"1:BASE_PARITY:0530ac82bf3ad80c"}
16:18:00 [16:18:00.355]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5145195b6d1bf42d epochNumber=1
{"provingJobId":"1:BASE_PARITY:5145195b6d1bf42d"}
16:18:00 [16:18:00.356]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.357]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:35a343ddaab0f286 epochNumber=1
{"provingJobId":"1:BASE_PARITY:35a343ddaab0f286"}
16:18:00 [16:18:00.357]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:969feb1452196cf8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:969feb1452196cf8"}
16:18:00 [16:18:00.358]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c5ee20ee964cd0a epochNumber=2
{"provingJobId":"1:BASE_PARITY:6c5ee20ee964cd0a"}
16:18:00 [16:18:00.358]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e83fc2798596af76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e83fc2798596af76"}
16:18:00 [16:18:00.359]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0cfd7b58240381d2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0cfd7b58240381d2"}
16:18:00 [16:18:00.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6dd6e69fed4a0a64 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6dd6e69fed4a0a64"}
16:18:00 [16:18:00.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:03d9ba37a1f2de89 epochNumber=2
{"provingJobId":"1:BASE_PARITY:03d9ba37a1f2de89"}
16:18:00 [16:18:00.361]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f5aa16a9bcca5687 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f5aa16a9bcca5687"}
16:18:00 [16:18:00.362]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f0d649c15d0a29b epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f0d649c15d0a29b"}
16:18:00 [16:18:00.363]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:380e809294b6fef9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:380e809294b6fef9"}
16:18:00 [16:18:00.363]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4f0d649c15d0a29b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4f0d649c15d0a29b"}
16:18:00 [16:18:00.365]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.365]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b16f776437dc7ab5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b16f776437dc7ab5"}
16:18:00 [16:18:00.366]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d011955dc2523ab6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d011955dc2523ab6"}
16:18:00 [16:18:00.366]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b16f776437dc7ab5 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:b16f776437dc7ab5"}
16:18:00 [16:18:00.367]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.368]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cba8e4637d140087 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cba8e4637d140087"}
16:18:00 [16:18:00.368]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a60d6db2a44fab3d epochNumber=1
{"provingJobId":"1:BASE_PARITY:a60d6db2a44fab3d"}
16:18:00 [16:18:00.368]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:cba8e4637d140087 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:cba8e4637d140087"}
16:18:00 [16:18:00.370]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.370]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a3cd259626af1ef7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a3cd259626af1ef7"}
16:18:00 [16:18:00.370]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:a3cd259626af1ef7
{"provingJobId":"1:BASE_PARITY:a3cd259626af1ef7"}
16:18:00 [16:18:00.371]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3cd259626af1ef7 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:a3cd259626af1ef7"}
16:18:00 [16:18:00.371]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:20c551b3f8b20d36 epochNumber=1
{"provingJobId":"1:BASE_PARITY:20c551b3f8b20d36"}
16:18:00 [16:18:00.371]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a3cd259626af1ef7 has already been completed
{"provingJobId":"1:BASE_PARITY:a3cd259626af1ef7"}
16:18:00 [16:18:00.372]
INFO:
prover-client:proving-broker Proving Broker started
16:18:00 [16:18:00.373]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7adfc3d5ffb521cc epochNumber=1
{"provingJobId":"1:BASE_PARITY:7adfc3d5ffb521cc"}
16:18:00 [16:18:00.373]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5da444cb5ca9de56 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5da444cb5ca9de56"}
16:18:01 [16:18:01.425]
INFO:
prover-client:proving-broker Proving Broker started
16:18:01 [16:18:01.426]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7adfc3d5ffb521cc settled=false
{"provingJobId":"1:BASE_PARITY:7adfc3d5ffb521cc","status":"pending"}
16:18:01 [16:18:01.427]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5da444cb5ca9de56 settled=false
{"provingJobId":"1:BASE_PARITY:5da444cb5ca9de56","status":"pending"}
16:18:01 [16:18:01.428]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7adfc3d5ffb521cc type=BASE_PARITY startedAt=1750263480373 older agent has taken job
{"provingJobId":"1:BASE_PARITY:7adfc3d5ffb521cc"}
16:18:01 [16:18:01.428]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7adfc3d5ffb521cc type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:7adfc3d5ffb521cc"}
16:18:01 [16:18:01.429]
INFO:
prover-client:proving-broker Proving Broker started
16:18:01 [16:18:01.430]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:61e626310916180a epochNumber=1
{"provingJobId":"1:BASE_PARITY:61e626310916180a"}
16:18:01 [16:18:01.431]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c04f5fdc08ec9cc epochNumber=2
{"provingJobId":"1:BASE_PARITY:0c04f5fdc08ec9cc"}
16:18:02 [16:18:02.431]
INFO:
prover-client:proving-broker Proving Broker started
16:18:02 [16:18:02.432]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:61e626310916180a settled=false
{"provingJobId":"1:BASE_PARITY:61e626310916180a","status":"pending"}
16:18:02 [16:18:02.432]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0c04f5fdc08ec9cc settled=false
{"provingJobId":"1:BASE_PARITY:0c04f5fdc08ec9cc","status":"pending"}
16:18:02 [16:18:02.432]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:61e626310916180a type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:61e626310916180a"}
16:18:02 [16:18:02.434]
INFO:
prover-client:proving-broker Proving Broker started
16:18:02 [16:18:02.434]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:58bee63b043f62fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:58bee63b043f62fa"}
16:18:02 [16:18:02.434]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:184dceb3ed102358 epochNumber=2
{"provingJobId":"1:BASE_PARITY:184dceb3ed102358"}
16:18:03 [16:18:03.435]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.436]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:58bee63b043f62fa settled=false
{"provingJobId":"1:BASE_PARITY:58bee63b043f62fa","status":"pending"}
16:18:03 [16:18:03.436]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:184dceb3ed102358 settled=false
{"provingJobId":"1:BASE_PARITY:184dceb3ed102358","status":"pending"}
16:18:03 [16:18:03.437]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:58bee63b043f62fa type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:58bee63b043f62fa"}
16:18:03 [16:18:03.437]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:58bee63b043f62fa type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:58bee63b043f62fa"}
16:18:03 [16:18:03.438]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:327e187e1df9d80c epochNumber=1
{"provingJobId":"1:BASE_PARITY:327e187e1df9d80c"}
16:18:03 [16:18:03.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e64fd918935d8780 epochNumber=2
{"provingJobId":"1:BASE_PARITY:e64fd918935d8780"}
16:18:03 [16:18:03.439]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:327e187e1df9d80c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:327e187e1df9d80c"}
16:18:03 [16:18:03.440]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e64fd918935d8780 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e64fd918935d8780"}
16:18:03 [16:18:03.441]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc937f2818a03c2d epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc937f2818a03c2d"}
16:18:03 [16:18:03.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:08ea3fa222bce5f4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:08ea3fa222bce5f4"}
16:18:03 [16:18:03.442]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dc937f2818a03c2d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:dc937f2818a03c2d"}
16:18:03 [16:18:03.442]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:dc937f2818a03c2d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:dc937f2818a03c2d"}
16:18:03 [16:18:03.442]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:08ea3fa222bce5f4 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:08ea3fa222bce5f4"}
16:18:03 [16:18:03.443]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.448]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2bb1be60690a4d15 err=test error
{"provingJoId":"1:BASE_PARITY:2bb1be60690a4d15"}
16:18:03 [16:18:03.451]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.453]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2d0f5007c50c4d38 not found
{"provingJobId":"1:BASE_PARITY:2d0f5007c50c4d38"}
16:18:03 [16:18:03.455]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.456]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7f4ba1ca0d90b95 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7f4ba1ca0d90b95"}
16:18:03 [16:18:03.457]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.458]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0077c79662d43b4d epochNumber=1
{"provingJobId":"1:BASE_PARITY:0077c79662d43b4d"}
16:18:03 [16:18:03.561]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0077c79662d43b4d timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:0077c79662d43b4d"}
16:18:03 [16:18:03.568]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.568]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ded350f4dfa9c0f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ded350f4dfa9c0f8"}
16:18:03 [16:18:03.671]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ded350f4dfa9c0f8 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:ded350f4dfa9c0f8"}
16:18:03 [16:18:03.676]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3e037ec60cfb413 epochNumber=10
{"provingJobId":"1:BASE_PARITY:d3e037ec60cfb413"}
16:18:03 [16:18:03.697]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:03 [16:18:03.802]
INFO:
prover-client:proving-broker Proving Broker started
16:18:03 [16:18:03.803]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b9c257358931fec4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b9c257358931fec4"}
16:18:03 [16:18:03.956]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b9c257358931fec4 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b9c257358931fec4"}
16:18:04 [16:18:04.006]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.007]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dca0fd4ac177c711 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dca0fd4ac177c711"}
16:18:04 [16:18:04.008]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:dca0fd4ac177c711 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:dca0fd4ac177c711"}
16:18:04 [16:18:04.009]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.010]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c2972aad6423d551 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c2972aad6423d551"}
16:18:04 [16:18:04.010]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c2972aad6423d551 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c2972aad6423d551"}
16:18:04 [16:18:04.011]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c2972aad6423d551 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:c2972aad6423d551"}
16:18:04 [16:18:04.012]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.012]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b1421610a09808e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b1421610a09808e6"}
16:18:04 [16:18:04.013]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b1421610a09808e6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:b1421610a09808e6"}
16:18:04 [16:18:04.014]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.014]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9f8da9fc0dd62425 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9f8da9fc0dd62425"}
16:18:04 [16:18:04.014]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9f8da9fc0dd62425 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:9f8da9fc0dd62425"}
16:18:04 [16:18:04.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a353e83d838b7d4a epochNumber=3
{"provingJobId":"1:BASE_PARITY:a353e83d838b7d4a"}
16:18:04 [16:18:04.038]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:04 [16:18:04.040]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:9f8da9fc0dd62425 err=test error
{"provingJoId":"1:BASE_PARITY:9f8da9fc0dd62425"}
16:18:04 [16:18:04.042]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.043]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cfc01a8a0bc261d7 settled=false
{"provingJobId":"1:BASE_PARITY:cfc01a8a0bc261d7","status":"pending"}
16:18:04 [16:18:04.043]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9d37ef65facf5e84 settled=false
{"provingJobId":"1:BASE_PARITY:9d37ef65facf5e84","status":"pending"}
16:18:04 [16:18:04.046]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.046]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6014192ec371ace1 settled=true
{"provingJobId":"1:BASE_PARITY:6014192ec371ace1","status":"fulfilled"}
16:18:04 [16:18:04.046]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:deb3f2df553faf0a settled=true
{"provingJobId":"2:BASE_PARITY:deb3f2df553faf0a","status":"fulfilled"}
16:18:04 [16:18:04.047]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.047]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b3d0f166c05fad79 settled=true
{"provingJobId":"1:BASE_PARITY:b3d0f166c05fad79","status":"fulfilled"}
16:18:04 [16:18:04.048]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:86509103943d6b75 settled=false
{"provingJobId":"1:BASE_PARITY:86509103943d6b75","status":"pending"}
16:18:04 [16:18:04.049]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.050]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa04d9ce6c4624d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa04d9ce6c4624d3"}
16:18:04 [16:18:04.052]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.053]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d36ed66cd6a2116 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1d36ed66cd6a2116"}
16:18:04 [16:18:04.054]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:1d36ed66cd6a2116: Error: db error: Error: db error
16:18:04 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:1d36ed66cd6a2116"}
16:18:04 [16:18:04.055]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c11222016568e80b epochNumber=1
{"provingJobId":"1:BASE_PARITY:c11222016568e80b"}
16:18:04 [16:18:04.056]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c11222016568e80b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c11222016568e80b"}
16:18:04 [16:18:04.056]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c11222016568e80b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c11222016568e80b"}
16:18:04 [16:18:04.057]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.058]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eecd273b23d8975b epochNumber=1
{"provingJobId":"1:BASE_PARITY:eecd273b23d8975b"}
16:18:04 [16:18:04.058]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eecd273b23d8975b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:eecd273b23d8975b"}
16:18:04 [16:18:04.058]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:eecd273b23d8975b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:eecd273b23d8975b"}
16:18:04 [16:18:04.061]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:eecd273b23d8975b: Error: db error
16:18:04 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:eecd273b23d8975b"}
16:18:04 [16:18:04.062]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.063]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e3a2af107fe74aa epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e3a2af107fe74aa"}
16:18:04 [16:18:04.063]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0e3a2af107fe74aa type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0e3a2af107fe74aa"}
16:18:04 [16:18:04.063]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0e3a2af107fe74aa type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0e3a2af107fe74aa"}
16:18:04 [16:18:04.065]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.066]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:521d6ac84c44b09f epochNumber=1
{"provingJobId":"1:BASE_PARITY:521d6ac84c44b09f"}
16:18:04 [16:18:04.066]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:521d6ac84c44b09f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:521d6ac84c44b09f"}
16:18:04 [16:18:04.067]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:521d6ac84c44b09f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:521d6ac84c44b09f"}
16:18:04 [16:18:04.067]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:521d6ac84c44b09f jobErr=test error: Error: db error
16:18:04 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:521d6ac84c44b09f"}
16:18:04 [16:18:04.068]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.069]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7ef8baf5415dfe0a not found
{"provingJobId":"1:BASE_PARITY:7ef8baf5415dfe0a"}
16:18:04 [16:18:04.076]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.077]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:47fdab1ddf9e3bc4 err=test error
{"provingJoId":"1:BASE_PARITY:47fdab1ddf9e3bc4"}
16:18:04 [16:18:04.078]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.088]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b06947e66b0a1ab2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b06947e66b0a1ab2"}
16:18:04 [16:18:04.089]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b06947e66b0a1ab2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b06947e66b0a1ab2"}
16:18:04 [16:18:04.090]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b06947e66b0a1ab2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b06947e66b0a1ab2"}
16:18:04 [16:18:04.101]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:404e5c0a5b9c6fbf epochNumber=2
{"provingJobId":"2:BASE_PARITY:404e5c0a5b9c6fbf"}
16:18:04 [16:18:04.102]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:404e5c0a5b9c6fbf type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:404e5c0a5b9c6fbf"}
16:18:04 [16:18:04.103]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:404e5c0a5b9c6fbf type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:404e5c0a5b9c6fbf"}
16:18:04 [16:18:04.164]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:684a4e1d4d67f05b epochNumber=3
{"provingJobId":"3:BASE_PARITY:684a4e1d4d67f05b"}
16:18:04 [16:18:04.180]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:04 [16:18:04.227]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:ef1b0f34e7ed0a58 epochNumber=4
{"provingJobId":"4:BASE_PARITY:ef1b0f34e7ed0a58"}
16:18:04 [16:18:04.232]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:04 [16:18:04.288]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:92cc7b3b4082c53c epochNumber=5
{"provingJobId":"5:BASE_PARITY:92cc7b3b4082c53c"}
16:18:04 [16:18:04.312]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:04 [16:18:04.343]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.344]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d386571db2b635c5 epochNumber=42
{"provingJobId":"1:BASE_PARITY:d386571db2b635c5"}
16:18:04 [16:18:04.345]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-test1OhrnP/42 with map size 1073741824
16:18:04 [16:18:04.349]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.379]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:1e9cec02faae0b75 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1e9cec02faae0b75"}
16:18:04 [16:18:04.385]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.386]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec563eebd3370a8b epochNumber=1
{"provingJobId":"1:BASE_PARITY:ec563eebd3370a8b"}
16:18:04 [16:18:04.389]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDwiwjy/1 with map size 1073741824
16:18:04 [16:18:04.393]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.415]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b75d886949ed9be epochNumber=1
{"provingJobId":"1:BASE_PARITY:7b75d886949ed9be"}
16:18:04 [16:18:04.429]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.429]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e7eac5a56d4b4d04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e7eac5a56d4b4d04"}
16:18:04 [16:18:04.431]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkPYZii/1 with map size 1073741824
16:18:04 [16:18:04.433]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.459]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e7eac5a56d4b4d04 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e7eac5a56d4b4d04"}
16:18:04 [16:18:04.462]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.463]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f78bc0cc7fa944f6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f78bc0cc7fa944f6"}
16:18:04 [16:18:04.464]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfqxCM4/1 with map size 1073741824
16:18:04 [16:18:04.465]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.483]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f78bc0cc7fa944f6 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f78bc0cc7fa944f6"}
16:18:04 [16:18:04.484]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f78bc0cc7fa944f6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f78bc0cc7fa944f6"}
16:18:04 [16:18:04.491]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f78bc0cc7fa944f6 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f78bc0cc7fa944f6"}
16:18:04 [16:18:04.497]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.497]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1037477a0d9a77c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1037477a0d9a77c2"}
16:18:04 [16:18:04.498]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdYw6HZ/1 with map size 1073741824
16:18:04 [16:18:04.500]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.526]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.531]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.532]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dfe32fd9704b31a2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dfe32fd9704b31a2"}
16:18:04 [16:18:04.533]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDyPWYb/1 with map size 1073741824
16:18:04 [16:18:04.535]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.555]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:dfe32fd9704b31a2
{"provingJobId":"1:BASE_PARITY:dfe32fd9704b31a2"}
16:18:04 [16:18:04.556]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dfe32fd9704b31a2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:dfe32fd9704b31a2"}
16:18:04 [16:18:04.556]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dfe32fd9704b31a2 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:dfe32fd9704b31a2"}
16:18:04 [16:18:04.566]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.566]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91fa185887390aa5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:91fa185887390aa5"}
16:18:04 [16:18:04.567]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFPVZDy/1 with map size 1073741824
16:18:04 [16:18:04.569]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.588]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:91fa185887390aa5
{"provingJobId":"1:BASE_PARITY:91fa185887390aa5"}
16:18:04 [16:18:04.588]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:91fa185887390aa5 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:91fa185887390aa5"}
16:18:04 [16:18:04.598]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.599]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:128eecaa9997cc77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:128eecaa9997cc77"}
16:18:04 [16:18:04.601]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKkWpgE/1 with map size 1073741824
16:18:04 [16:18:04.605]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.624]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:128eecaa9997cc77 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:128eecaa9997cc77"}
16:18:04 [16:18:04.625]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:128eecaa9997cc77 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:128eecaa9997cc77"}
16:18:04 [16:18:04.637]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.638]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:616abc50461787b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:616abc50461787b0"}
16:18:04 [16:18:04.639]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlsASqp/1 with map size 1073741824
16:18:04 [16:18:04.641]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.660]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:616abc50461787b0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:616abc50461787b0"}
16:18:04 [16:18:04.661]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:616abc50461787b0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:616abc50461787b0"}
16:18:04 [16:18:04.670]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.671]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e8540eb2e00077a0 epochNumber=0
{"provingJobId":"1:BASE_PARITY:e8540eb2e00077a0"}
16:18:04 [16:18:04.671]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-test1L6pMi/0 with map size 1073741824
16:18:04 [16:18:04.673]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.693]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e8540eb2e00077a0 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e8540eb2e00077a0"}
16:18:04 [16:18:04.694]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8efcf6bf09857b97 epochNumber=0
{"provingJobId":"1:BASE_PARITY:8efcf6bf09857b97"}
16:18:04 [16:18:04.695]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e8540eb2e00077a0 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e8540eb2e00077a0"}
16:18:04 [16:18:04.695]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c850b3d869407c44 epochNumber=0
{"provingJobId":"1:BASE_PARITY:c850b3d869407c44"}
16:18:04 [16:18:04.711]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.713]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.714]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:daf0bbda547ccd21 epochNumber=2
{"provingJobId":"1:BASE_PARITY:daf0bbda547ccd21"}
16:18:04 [16:18:04.715]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5cCoKU/2 with map size 1073741824
16:18:04 [16:18:04.717]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.733]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d44ab5deffe1aafc epochNumber=1
{"provingJobId":"1:BASE_PARITY:d44ab5deffe1aafc"}
16:18:04 [16:18:04.734]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5cCoKU/1 with map size 1073741824
16:18:04 [16:18:04.737]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.760]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.761]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c060cda233143e5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c060cda233143e5"}
16:18:04 [16:18:04.762]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testH1rhBz/1 with map size 1073741824
16:18:04 [16:18:04.764]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.790]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.791]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dabc678dbcea8a97 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dabc678dbcea8a97"}
16:18:04 [16:18:04.792]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVeumCG/1 with map size 1073741824
16:18:04 [16:18:04.794]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.804]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ff74007cecbf05c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ff74007cecbf05c5"}
16:18:04 [16:18:04.807]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:99876e0e7de637b9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:99876e0e7de637b9"}
16:18:04 [16:18:04.808]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVeumCG/2 with map size 1073741824
16:18:04 [16:18:04.810]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.822]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:06ef4cd8f03766d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:06ef4cd8f03766d5"}
16:18:04 [16:18:04.829]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.830]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e64096ff3ca7f42c epochNumber=1
{"provingJobId":"1:BASE_PARITY:e64096ff3ca7f42c"}
16:18:04 [16:18:04.831]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testH7mOdF/1 with map size 1073741824
16:18:04 [16:18:04.833]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.844]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:673263b10990767e epochNumber=1
{"provingJobId":"1:BASE_PARITY:673263b10990767e"}
16:18:04 [16:18:04.848]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:995756516b176905 epochNumber=2
{"provingJobId":"1:BASE_PARITY:995756516b176905"}
16:18:04 [16:18:04.849]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testH7mOdF/2 with map size 1073741824
16:18:04 [16:18:04.852]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.866]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f835a6ad989b793f epochNumber=1
{"provingJobId":"1:BASE_PARITY:f835a6ad989b793f"}
16:18:04 [16:18:04.876]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.877]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8444305c42b5c8a1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8444305c42b5c8a1"}
16:18:04 [16:18:04.877]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8VHmCU/1 with map size 1073741824
16:18:04 [16:18:04.879]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.897]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a1b1779e38906e85 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a1b1779e38906e85"}
16:18:04 [16:18:04.903]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:76ecd4ffab11b3b4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:76ecd4ffab11b3b4"}
16:18:04 [16:18:04.904]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8VHmCU/2 with map size 1073741824
16:18:04 [16:18:04.906]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.922]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce21cf4c31c1413c epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce21cf4c31c1413c"}
16:18:04 [16:18:04.932]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.933]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b8934dea87461bc9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b8934dea87461bc9"}
16:18:04 [16:18:04.934]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testciMRYv/1 with map size 1073741824
16:18:04 [16:18:04.936]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.952]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b4951c054ee5bb21 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b4951c054ee5bb21"}
16:18:04 [16:18:04.959]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b8934dea87461bc9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b8934dea87461bc9"}
16:18:04 [16:18:04.969]
INFO:
prover-client:proving-broker Proving Broker started
16:18:04 [16:18:04.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b74c60c9c48a19ef epochNumber=1
{"provingJobId":"1:BASE_PARITY:b74c60c9c48a19ef"}
16:18:04 [16:18:04.971]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAJ8mdD/1 with map size 1073741824
16:18:04 [16:18:04.973]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:04 [16:18:04.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:10a661039e1b65df epochNumber=1
{"provingJobId":"1:BASE_PARITY:10a661039e1b65df"}
16:18:04 [16:18:04.996]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b74c60c9c48a19ef type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:b74c60c9c48a19ef"}
16:18:05 [16:18:05.006]
INFO:
prover-client:proving-broker Proving Broker started
16:18:05 [16:18:05.006]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a3afdcebf570ee4e epochNumber=1
{"provingJobId":"1:BASE_PARITY:a3afdcebf570ee4e"}
16:18:05 [16:18:05.009]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKrIdnc/1 with map size 1073741824
16:18:05 [16:18:05.010]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:05 [16:18:05.026]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ed0ab7483afc4dd epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ed0ab7483afc4dd"}
16:18:05 [16:18:05.031]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a3afdcebf570ee4e type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:a3afdcebf570ee4e"}
16:18:05 [16:18:05.035]
INFO:
prover-client:proving-broker Proving Broker started
16:18:05 [16:18:05.035]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:398cfd113a3b730f epochNumber=1
{"provingJobId":"1:BASE_PARITY:398cfd113a3b730f"}
16:18:05 [16:18:05.036]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4sJYjc/1 with map size 1073741824
16:18:05 [16:18:05.038]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:05 [16:18:05.055]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:398cfd113a3b730f
{"provingJobId":"1:BASE_PARITY:398cfd113a3b730f"}
16:18:05 [16:18:05.056]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:398cfd113a3b730f type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:398cfd113a3b730f"}
16:18:05 [16:18:05.062]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c517ca570a17c963 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c517ca570a17c963"}
16:18:05 [16:18:05.068]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:398cfd113a3b730f has already been completed
{"provingJobId":"1:BASE_PARITY:398cfd113a3b730f"}
16:18:05 [16:18:05.072]
INFO:
prover-client:proving-broker Proving Broker started
16:18:05 [16:18:05.073]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:78e7ff98fbf0942a epochNumber=1
{"provingJobId":"1:BASE_PARITY:78e7ff98fbf0942a"}
16:18:05 [16:18:05.074]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkrFB8c/1 with map size 1073741824
16:18:05 [16:18:05.080]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:05 [16:18:05.095]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:44f1acf21f1a3ac0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:44f1acf21f1a3ac0"}
16:18:05 [16:18:05.096]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkrFB8c/2 with map size 1073741824
16:18:05 [16:18:05.097]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:06 [16:18:06.164]
INFO:
prover-client:proving-broker Proving Broker started
16:18:06 [16:18:06.176]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:78e7ff98fbf0942a settled=false
{"provingJobId":"1:BASE_PARITY:78e7ff98fbf0942a","status":"pending"}
16:18:06 [16:18:06.178]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:44f1acf21f1a3ac0 settled=false
{"provingJobId":"1:BASE_PARITY:44f1acf21f1a3ac0","status":"pending"}
16:18:06 [16:18:06.180]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:78e7ff98fbf0942a type=BASE_PARITY startedAt=1750263485113 older agent has taken job
{"provingJobId":"1:BASE_PARITY:78e7ff98fbf0942a"}
16:18:06 [16:18:06.180]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:78e7ff98fbf0942a type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:78e7ff98fbf0942a"}
16:18:06 [16:18:06.185]
INFO:
prover-client:proving-broker Proving Broker started
16:18:06 [16:18:06.185]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e9a95a1a8aab00a epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e9a95a1a8aab00a"}
16:18:06 [16:18:06.186]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testamlqJl/1 with map size 1073741824
16:18:06 [16:18:06.188]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:06 [16:18:06.207]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7a69bf2ef092614 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b7a69bf2ef092614"}
16:18:06 [16:18:06.208]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testamlqJl/2 with map size 1073741824
16:18:06 [16:18:06.210]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:07 [16:18:07.227]
INFO:
prover-client:proving-broker Proving Broker started
16:18:07 [16:18:07.230]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0e9a95a1a8aab00a settled=false
{"provingJobId":"1:BASE_PARITY:0e9a95a1a8aab00a","status":"pending"}
16:18:07 [16:18:07.232]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b7a69bf2ef092614 settled=false
{"provingJobId":"1:BASE_PARITY:b7a69bf2ef092614","status":"pending"}
16:18:07 [16:18:07.234]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0e9a95a1a8aab00a type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:0e9a95a1a8aab00a"}
16:18:07 [16:18:07.245]
INFO:
prover-client:proving-broker Proving Broker started
16:18:07 [16:18:07.248]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:572fc7f288b8cbba epochNumber=1
{"provingJobId":"1:BASE_PARITY:572fc7f288b8cbba"}
16:18:07 [16:18:07.249]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpSw3OY/1 with map size 1073741824
16:18:07 [16:18:07.255]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:07 [16:18:07.277]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:27599043c0ccfbbe epochNumber=2
{"provingJobId":"1:BASE_PARITY:27599043c0ccfbbe"}
16:18:07 [16:18:07.278]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpSw3OY/2 with map size 1073741824
16:18:07 [16:18:07.281]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.303]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.306]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:572fc7f288b8cbba settled=false
{"provingJobId":"1:BASE_PARITY:572fc7f288b8cbba","status":"pending"}
16:18:08 [16:18:08.308]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:27599043c0ccfbbe settled=false
{"provingJobId":"1:BASE_PARITY:27599043c0ccfbbe","status":"pending"}
16:18:08 [16:18:08.309]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:572fc7f288b8cbba type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:572fc7f288b8cbba"}
16:18:08 [16:18:08.310]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:572fc7f288b8cbba type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:572fc7f288b8cbba"}
16:18:08 [16:18:08.321]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.322]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:85fdbc99eec42fe3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:85fdbc99eec42fe3"}
16:18:08 [16:18:08.323]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3Y4kiT/1 with map size 1073741824
16:18:08 [16:18:08.325]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.348]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8cdc823c6925b5bf epochNumber=2
{"provingJobId":"1:BASE_PARITY:8cdc823c6925b5bf"}
16:18:08 [16:18:08.349]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3Y4kiT/2 with map size 1073741824
16:18:08 [16:18:08.351]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.374]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:85fdbc99eec42fe3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:85fdbc99eec42fe3"}
16:18:08 [16:18:08.381]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8cdc823c6925b5bf type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8cdc823c6925b5bf"}
16:18:08 [16:18:08.392]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.393]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2eb5f2755afa7925 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2eb5f2755afa7925"}
16:18:08 [16:18:08.394]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testh9u37K/1 with map size 1073741824
16:18:08 [16:18:08.398]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.415]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec9140a24d004aa7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ec9140a24d004aa7"}
16:18:08 [16:18:08.417]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testh9u37K/2 with map size 1073741824
16:18:08 [16:18:08.419]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.438]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2eb5f2755afa7925 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2eb5f2755afa7925"}
16:18:08 [16:18:08.439]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2eb5f2755afa7925 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2eb5f2755afa7925"}
16:18:08 [16:18:08.449]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ec9140a24d004aa7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ec9140a24d004aa7"}
16:18:08 [16:18:08.464]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.466]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e3f567a729de9111 err=test error
{"provingJoId":"1:BASE_PARITY:e3f567a729de9111"}
16:18:08 [16:18:08.471]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.472]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3ccd0adcaf66355e not found
{"provingJobId":"1:BASE_PARITY:3ccd0adcaf66355e"}
16:18:08 [16:18:08.478]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.479]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bed2c9878cb5b0d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bed2c9878cb5b0d6"}
16:18:08 [16:18:08.480]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwmLQEf/1 with map size 1073741824
16:18:08 [16:18:08.483]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.507]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.507]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:880807b61e35f1ee epochNumber=1
{"provingJobId":"1:BASE_PARITY:880807b61e35f1ee"}
16:18:08 [16:18:08.508]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8k0YcE/1 with map size 1073741824
16:18:08 [16:18:08.510]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.638]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:880807b61e35f1ee timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:880807b61e35f1ee"}
16:18:08 [16:18:08.645]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.646]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6ce6d3288c28e035 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6ce6d3288c28e035"}
16:18:08 [16:18:08.647]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7AhERW/1 with map size 1073741824
16:18:08 [16:18:08.650]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.773]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6ce6d3288c28e035 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:6ce6d3288c28e035"}
16:18:08 [16:18:08.779]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:da7942ffd4b0ed3d epochNumber=10
{"provingJobId":"1:BASE_PARITY:da7942ffd4b0ed3d"}
16:18:08 [16:18:08.780]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test7AhERW/10 with map size 1073741824
16:18:08 [16:18:08.782]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:08 [16:18:08.799]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:08 [16:18:08.800]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
16:18:08 [16:18:08.929]
INFO:
prover-client:proving-broker Proving Broker started
16:18:08 [16:18:08.930]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a003f69086a651fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:a003f69086a651fa"}
16:18:08 [16:18:08.931]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testh4jloa/1 with map size 1073741824
16:18:08 [16:18:08.933]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.108]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a003f69086a651fa timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:a003f69086a651fa"}
16:18:09 [16:18:09.156]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.157]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f7727f0bbe015a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f7727f0bbe015a3"}
16:18:09 [16:18:09.158]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testy88ps1/1 with map size 1073741824
16:18:09 [16:18:09.160]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.182]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7f7727f0bbe015a3 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:7f7727f0bbe015a3"}
16:18:09 [16:18:09.187]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.189]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8cffdd3a89b45439 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8cffdd3a89b45439"}
16:18:09 [16:18:09.190]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoxD8Oz/1 with map size 1073741824
16:18:09 [16:18:09.192]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.209]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8cffdd3a89b45439 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:8cffdd3a89b45439"}
16:18:09 [16:18:09.210]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8cffdd3a89b45439 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:8cffdd3a89b45439"}
16:18:09 [16:18:09.219]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.220]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d2204a99c9e1f8ae epochNumber=1
{"provingJobId":"1:BASE_PARITY:d2204a99c9e1f8ae"}
16:18:09 [16:18:09.223]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1JcTkC/1 with map size 1073741824
16:18:09 [16:18:09.225]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.242]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d2204a99c9e1f8ae type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d2204a99c9e1f8ae"}
16:18:09 [16:18:09.253]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ed89138a0ced09fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:ed89138a0ced09fd"}
16:18:09 [16:18:09.255]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJyjO2g/1 with map size 1073741824
16:18:09 [16:18:09.257]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.273]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ed89138a0ced09fd type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:ed89138a0ced09fd"}
16:18:09 [16:18:09.274]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36b767221e55b55d epochNumber=3
{"provingJobId":"1:BASE_PARITY:36b767221e55b55d"}
16:18:09 [16:18:09.275]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testJyjO2g/3 with map size 1073741824
16:18:09 [16:18:09.277]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.304]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:09 [16:18:09.304]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
16:18:09 [16:18:09.319]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ed89138a0ced09fd err=test error
{"provingJoId":"1:BASE_PARITY:ed89138a0ced09fd"}
16:18:09 [16:18:09.325]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCvuVNS/1 with map size 1073741824
16:18:09 [16:18:09.328]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.345]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCvuVNS/2 with map size 1073741824
16:18:09 [16:18:09.348]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.381]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.384]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5857a55158889465 settled=false
{"provingJobId":"1:BASE_PARITY:5857a55158889465","status":"pending"}
16:18:09 [16:18:09.385]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:10cc60519c72eaeb settled=false
{"provingJobId":"1:BASE_PARITY:10cc60519c72eaeb","status":"pending"}
16:18:09 [16:18:09.394]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0IscuJ/1 with map size 1073741824
16:18:09 [16:18:09.398]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.418]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0IscuJ/2 with map size 1073741824
16:18:09 [16:18:09.424]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.473]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.479]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:029cc2bb8d301109 settled=true
{"provingJobId":"1:BASE_PARITY:029cc2bb8d301109","status":"fulfilled"}
16:18:09 [16:18:09.481]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:9826f8b5ec034688 settled=true
{"provingJobId":"2:BASE_PARITY:9826f8b5ec034688","status":"fulfilled"}
16:18:09 [16:18:09.490]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPKsV2r/1 with map size 1073741824
16:18:09 [16:18:09.493]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.523]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPKsV2r/2 with map size 1073741824
16:18:09 [16:18:09.528]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.548]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.550]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2400b465f9bc693a settled=true
{"provingJobId":"1:BASE_PARITY:2400b465f9bc693a","status":"fulfilled"}
16:18:09 [16:18:09.553]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:48be15ab6193a6d0 settled=false
{"provingJobId":"1:BASE_PARITY:48be15ab6193a6d0","status":"pending"}
16:18:09 [16:18:09.560]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.562]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c4474681c34a3eae epochNumber=1
{"provingJobId":"1:BASE_PARITY:c4474681c34a3eae"}
16:18:09 [16:18:09.563]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPQ3s3k/1 with map size 1073741824
16:18:09 [16:18:09.577]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.601]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.602]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5477b10fd3390d64 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5477b10fd3390d64"}
16:18:09 [16:18:09.603]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:5477b10fd3390d64: Error: db error: Error: db error
16:18:09 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:5477b10fd3390d64"}
16:18:09 [16:18:09.608]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.609]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8fd98ebefbf732d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8fd98ebefbf732d6"}
16:18:09 [16:18:09.610]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJRVvI5/1 with map size 1073741824
16:18:09 [16:18:09.614]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.635]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8fd98ebefbf732d6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8fd98ebefbf732d6"}
16:18:09 [16:18:09.636]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8fd98ebefbf732d6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8fd98ebefbf732d6"}
16:18:09 [16:18:09.648]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc80581494219a80 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc80581494219a80"}
16:18:09 [16:18:09.651]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAiXN1y/1 with map size 1073741824
16:18:09 [16:18:09.654]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.681]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fc80581494219a80 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fc80581494219a80"}
16:18:09 [16:18:09.681]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fc80581494219a80 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fc80581494219a80"}
16:18:09 [16:18:09.682]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:fc80581494219a80: Error: db error
16:18:09 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:fc80581494219a80"}
16:18:09 [16:18:09.687]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.688]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:70e7c213ab79a8db epochNumber=1
{"provingJobId":"1:BASE_PARITY:70e7c213ab79a8db"}
16:18:09 [16:18:09.688]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3uwMKQ/1 with map size 1073741824
16:18:09 [16:18:09.693]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.721]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:70e7c213ab79a8db type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:70e7c213ab79a8db"}
16:18:09 [16:18:09.722]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:70e7c213ab79a8db type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:70e7c213ab79a8db"}
16:18:09 [16:18:09.736]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.736]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d323b2d5151ed89 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d323b2d5151ed89"}
16:18:09 [16:18:09.737]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3dC7UZ/1 with map size 1073741824
16:18:09 [16:18:09.743]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.776]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0d323b2d5151ed89 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0d323b2d5151ed89"}
16:18:09 [16:18:09.776]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0d323b2d5151ed89 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0d323b2d5151ed89"}
16:18:09 [16:18:09.777]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:0d323b2d5151ed89 jobErr=test error: Error: db error
16:18:09 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:0d323b2d5151ed89"}
16:18:09 [16:18:09.785]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.785]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:484b9ceeb3fe43ef not found
{"provingJobId":"1:BASE_PARITY:484b9ceeb3fe43ef"}
16:18:09 [16:18:09.792]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.793]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2abe8078e5c50516 err=test error
{"provingJoId":"1:BASE_PARITY:2abe8078e5c50516"}
16:18:09 [16:18:09.797]
INFO:
prover-client:proving-broker Proving Broker started
16:18:09 [16:18:09.809]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c2b58b4f6a2701e epochNumber=1
{"provingJobId":"1:BASE_PARITY:0c2b58b4f6a2701e"}
16:18:09 [16:18:09.810]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNAXAYp/1 with map size 1073741824
16:18:09 [16:18:09.815]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.837]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0c2b58b4f6a2701e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0c2b58b4f6a2701e"}
16:18:09 [16:18:09.837]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0c2b58b4f6a2701e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0c2b58b4f6a2701e"}
16:18:09 [16:18:09.856]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:b5f21c3177ecf01a epochNumber=2
{"provingJobId":"2:BASE_PARITY:b5f21c3177ecf01a"}
16:18:09 [16:18:09.858]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNAXAYp/2 with map size 1073741824
16:18:09 [16:18:09.870]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:09 [16:18:09.894]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:b5f21c3177ecf01a type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:b5f21c3177ecf01a"}
16:18:09 [16:18:09.895]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:b5f21c3177ecf01a type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:b5f21c3177ecf01a"}
16:18:09 [16:18:09.964]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:bbfabee47df00d1e epochNumber=3
{"provingJobId":"3:BASE_PARITY:bbfabee47df00d1e"}
16:18:09 [16:18:09.977]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testNAXAYp/3 with map size 1073741824
16:18:09 [16:18:09.981]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:10 [16:18:10.009]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:10 [16:18:10.009]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
16:18:10 [16:18:10.065]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:ccc23d0ebda81f33 epochNumber=4
{"provingJobId":"4:BASE_PARITY:ccc23d0ebda81f33"}
16:18:10 [16:18:10.067]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testNAXAYp/4 with map size 1073741824
16:18:10 [16:18:10.071]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:10 [16:18:10.092]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:10 [16:18:10.093]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
16:18:10 [16:18:10.151]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:c9944e2b994e9777 epochNumber=5
{"provingJobId":"5:BASE_PARITY:c9944e2b994e9777"}
16:18:10 [16:18:10.153]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testNAXAYp/5 with map size 1073741824
16:18:10 [16:18:10.168]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
16:18:10 [16:18:10.197]
INFO:
prover-client:proving-broker Cleaned up jobs=1
16:18:10 [16:18:10.198]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
16:18:10
PASS src/proving_broker/proving_broker.test.ts (
13.212 s)
16:18:10 ProvingBroker
16:18:10 Producer API
16:18:10
✓ refuses stale jobs (42 ms)
16:18:10
✓ enqueues jobs (4 ms)
16:18:10
✓ ignores duplicate jobs (4 ms)
16:18:10
✓ reports correct status when enqueuing repeat jobs (4 ms)
16:18:10
✓ throws an error in case of duplicate job IDs (24 ms)
16:18:10
✓ returns not-found status for non-existing jobs (2 ms)
16:18:10
✓ cancels jobs in queue (3 ms)
16:18:10
✓ cancels jobs in-progress (2 ms)
16:18:10
✓ returns job result if successful (3 ms)
16:18:10
✓ returns job error if failed (2 ms)
16:18:10
✓ correctly returns job status for concurrent writes (3 ms)
16:18:10
✓ refuses stale jobs (41 ms)
16:18:10
✓ enqueues jobs (44 ms)
16:18:10
✓ ignores duplicate jobs (32 ms)
16:18:10
✓ reports correct status when enqueuing repeat jobs (31 ms)
16:18:10
✓ throws an error in case of duplicate job IDs (30 ms)
16:18:10
✓ returns not-found status for non-existing jobs (5 ms)
16:18:10
✓ cancels jobs in queue (36 ms)
16:18:10
✓ cancels jobs in-progress (32 ms)
16:18:10
✓ returns job result if successful (38 ms)
16:18:10
✓ returns job error if failed (33 ms)
16:18:10
✓ correctly returns job status for concurrent writes (41 ms)
16:18:10 Consumer API
16:18:10
✓ returns undefined if no jobs are available (1 ms)
16:18:10
✓ returns jobs in priority order (2 ms)
16:18:10
✓ returns undefined if no jobs are available for the given allowList (7 ms)
16:18:10
✓ returns a job if it is in the allowList (3 ms)
16:18:10
✓ returns the most important job if it is in the allowList (2 ms)
16:18:10
✓ returns any job if filter is empty (2 ms)
16:18:10
✓ returns a new job when reporting job success (3 ms)
16:18:10
✓ returns a new job when reporting permanent error (3 ms)
16:18:10
✓ returns a new job when reporting retry-able error (2 ms)
16:18:10
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
16:18:10
✓ returns a new job if job is already in progress elsewhere (1057 ms)
16:18:10
✓ avoids sending the same job to a new agent after a restart (1004 ms)
16:18:10
✓ avoids sending a completed job to a new agent after a restart (1005 ms)
16:18:10
✓ tracks job result if in progress (2 ms)
16:18:10
✓ tracks job result even if job is in queue (2 ms)
16:18:10
✓ ignores reported job error if unknown job (7 ms)
16:18:10
✓ ignores job result if unknown job (2 ms)
16:18:10
✓ returns undefined if no jobs are available (3 ms)
16:18:10
✓ returns jobs in priority order (46 ms)
16:18:10
✓ returns undefined if no jobs are available for the given allowList (29 ms)
16:18:10
✓ returns a job if it is in the allowList (38 ms)
16:18:10
✓ returns the most important job if it is in the allowList (46 ms)
16:18:10
✓ returns any job if filter is empty (56 ms)
16:18:10
✓ returns a new job when reporting job success (35 ms)
16:18:10
✓ returns a new job when reporting permanent error (36 ms)
16:18:10
✓ returns a new job when reporting retry-able error (30 ms)
16:18:10
✓ returns a new job when reporting progress if current one is cancelled (37 ms)
16:18:10
✓ returns a new job if job is already in progress elsewhere (1111 ms)
16:18:10
✓ avoids sending the same job to a new agent after a restart (1054 ms)
16:18:10
✓ avoids sending a completed job to a new agent after a restart (1081 ms)
16:18:10
✓ tracks job result if in progress (71 ms)
16:18:10
✓ tracks job result even if job is in queue (70 ms)
16:18:10
✓ ignores reported job error if unknown job (5 ms)
16:18:10
✓ ignores job result if unknown job (7 ms)
16:18:10 Timeouts
16:18:10
✓ tracks in progress jobs (3 ms)
16:18:10
✓ re-enqueues jobs that time out (110 ms)
16:18:10
✓ cancel stale jobs that time out (234 ms)
16:18:10
✓ keeps the jobs in progress while it is alive (204 ms)
16:18:10
✓ tracks in progress jobs (29 ms)
16:18:10
✓ re-enqueues jobs that time out (137 ms)
16:18:10
✓ cancel stale jobs that time out (283 ms)
16:18:10
✓ keeps the jobs in progress while it is alive (226 ms)
16:18:10 Retries
16:18:10
✓ retries jobs (3 ms)
16:18:10
✓ retries up to a maximum number of times (2 ms)
16:18:10
✓ passing retry=false does not retry (1 ms)
16:18:10
✓ does not retry if job is stale (28 ms)
16:18:10
✓ retries jobs (31 ms)
16:18:10
✓ retries up to a maximum number of times (32 ms)
16:18:10
✓ passing retry=false does not retry (33 ms)
16:18:10
✓ does not retry if job is stale (70 ms)
16:18:10 Database management
16:18:10
✓ re-enqueues proof requests on start (4 ms)
16:18:10
✓ restores proof results on start (2 ms)
16:18:10
✓ only re-enqueues unfinished jobs (1 ms)
16:18:10
✓ saves job when enqueued (4 ms)
16:18:10
✓ does not retain job if database fails to save (2 ms)
16:18:10
✓ saves job result (2 ms)
16:18:10
✓ saves result even if database fails to save (4 ms)
16:18:10
✓ saves job error (4 ms)
16:18:10
✓ saves job error even if database fails to save (3 ms)
16:18:10
✓ does not save job result if job is unknown (7 ms)
16:18:10
✓ does not save job error if job is unknown (1 ms)
16:18:10
✓ cleans up old jobs periodically (262 ms)
16:18:10
✓ re-enqueues proof requests on start (67 ms)
16:18:10
✓ restores proof results on start (95 ms)
16:18:10
✓ only re-enqueues unfinished jobs (69 ms)
16:18:10
✓ saves job when enqueued (42 ms)
16:18:10
✓ does not retain job if database fails to save (7 ms)
16:18:10
✓ saves job result (40 ms)
16:18:10
✓ saves result even if database fails to save (38 ms)
16:18:10
✓ saves job error (47 ms)
16:18:10
✓ saves job error even if database fails to save (51 ms)
16:18:10
✓ does not save job result if job is unknown (3 ms)
16:18:10
✓ does not save job error if job is unknown (7 ms)
16:18:10
✓ cleans up old jobs periodically (452 ms)
16:18:10
16:18:10
Test Suites: 1 passed, 1 total
16:18:10
Tests: 96 passed, 96 total
16:18:10
Snapshots: 0 total
16:18:10
Time: 13.316 s
16:18:10
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
16:18:10
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?