Parent Log:
http://ci.aztec-labs.com/d6ab7b7597a8cce2
Command: 19ef62d9450eeb9e yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/eb7990c89ac679ce7fdfc2e985670cbf85c04be2
Env: REF_NAME=gh-readonly-queue/next/pr-14918-b70544b6d50e3ecb6598361f1990bd51745c7b55 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Tue Jun 17 09:02:48 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-14918_amd64_x3-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
09:02:53 [09:02:53.257]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.264]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4accbdb5b378179c epochNumber=42
{"provingJobId":"1:BASE_PARITY:4accbdb5b378179c"}
09:02:53 [09:02:53.267]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:c1af273c8b941705 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c1af273c8b941705"}
09:02:53 [09:02:53.300]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.301]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d3da3e7c7397a76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3d3da3e7c7397a76"}
09:02:53 [09:02:53.302]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ba34c39c40dd36aa epochNumber=1
{"provingJobId":"1:BASE_PARITY:ba34c39c40dd36aa"}
09:02:53 [09:02:53.304]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.305]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6aef8724634c847d epochNumber=1
{"provingJobId":"1:BASE_PARITY:6aef8724634c847d"}
09:02:53 [09:02:53.306]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6aef8724634c847d epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6aef8724634c847d"}
09:02:53 [09:02:53.308]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.308]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9dde17628439f51b epochNumber=1
{"provingJobId":"1:BASE_PARITY:9dde17628439f51b"}
09:02:53 [09:02:53.309]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9dde17628439f51b epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9dde17628439f51b"}
09:02:53 [09:02:53.310]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9dde17628439f51b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9dde17628439f51b"}
09:02:53 [09:02:53.310]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9dde17628439f51b epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9dde17628439f51b"}
09:02:53 [09:02:53.311]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.311]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:600a435ef826e3b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:600a435ef826e3b4"}
09:02:53 [09:02:53.332]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.334]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.334]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1a034c0407b37812 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1a034c0407b37812"}
09:02:53 [09:02:53.335]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:1a034c0407b37812
{"provingJobId":"1:BASE_PARITY:1a034c0407b37812"}
09:02:53 [09:02:53.336]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1a034c0407b37812 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:1a034c0407b37812"}
09:02:53 [09:02:53.336]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1a034c0407b37812 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:1a034c0407b37812"}
09:02:53 [09:02:53.337]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.338]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebd936217612529d epochNumber=1
{"provingJobId":"1:BASE_PARITY:ebd936217612529d"}
09:02:53 [09:02:53.338]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ebd936217612529d
{"provingJobId":"1:BASE_PARITY:ebd936217612529d"}
09:02:53 [09:02:53.338]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ebd936217612529d type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:ebd936217612529d"}
09:02:53 [09:02:53.340]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d00355164257a7d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d00355164257a7d5"}
09:02:53 [09:02:53.341]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d00355164257a7d5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d00355164257a7d5"}
09:02:53 [09:02:53.341]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d00355164257a7d5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d00355164257a7d5"}
09:02:53 [09:02:53.350]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.351]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0bfd3c1d133248fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:0bfd3c1d133248fe"}
09:02:53 [09:02:53.351]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0bfd3c1d133248fe type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0bfd3c1d133248fe"}
09:02:53 [09:02:53.351]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0bfd3c1d133248fe type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0bfd3c1d133248fe"}
09:02:53 [09:02:53.352]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.353]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bdf2d46c9eed1514 epochNumber=0
{"provingJobId":"1:BASE_PARITY:bdf2d46c9eed1514"}
09:02:53 [09:02:53.353]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:bdf2d46c9eed1514 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:bdf2d46c9eed1514"}
09:02:53 [09:02:53.353]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5391552cf06df1bc epochNumber=0
{"provingJobId":"1:BASE_PARITY:5391552cf06df1bc"}
09:02:53 [09:02:53.353]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:bdf2d46c9eed1514 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:bdf2d46c9eed1514"}
09:02:53 [09:02:53.353]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e92e86876055f049 epochNumber=0
{"provingJobId":"1:BASE_PARITY:e92e86876055f049"}
09:02:53 [09:02:53.354]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.356]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.356]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f90ccd7eb879a533 epochNumber=2
{"provingJobId":"1:BASE_PARITY:f90ccd7eb879a533"}
09:02:53 [09:02:53.356]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4fb95143d95eba94 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4fb95143d95eba94"}
09:02:53 [09:02:53.358]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.358]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:530412bd9a266f0b epochNumber=1
{"provingJobId":"1:BASE_PARITY:530412bd9a266f0b"}
09:02:53 [09:02:53.359]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.359]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f8bef5109c6858f4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f8bef5109c6858f4"}
09:02:53 [09:02:53.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7e41b4dd87d37073 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e41b4dd87d37073"}
09:02:53 [09:02:53.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd75b837b4f542f6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:bd75b837b4f542f6"}
09:02:53 [09:02:53.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cdc2f7ad482bf607 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cdc2f7ad482bf607"}
09:02:53 [09:02:53.361]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1e02cb5ec0c7fdff epochNumber=1
{"provingJobId":"1:BASE_PARITY:1e02cb5ec0c7fdff"}
09:02:53 [09:02:53.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:542896cc9fa1dd5b epochNumber=1
{"provingJobId":"1:BASE_PARITY:542896cc9fa1dd5b"}
09:02:53 [09:02:53.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:079961676c02a3ce epochNumber=2
{"provingJobId":"1:BASE_PARITY:079961676c02a3ce"}
09:02:53 [09:02:53.363]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9c97fee57d0b1017 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9c97fee57d0b1017"}
09:02:53 [09:02:53.364]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.364]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e85780b65be40f1c epochNumber=1
{"provingJobId":"1:BASE_PARITY:e85780b65be40f1c"}
09:02:53 [09:02:53.364]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:35d6f8dbe1c6ca07 epochNumber=1
{"provingJobId":"1:BASE_PARITY:35d6f8dbe1c6ca07"}
09:02:53 [09:02:53.365]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e4cbdf37a987264 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3e4cbdf37a987264"}
09:02:53 [09:02:53.365]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:52c62164112911d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:52c62164112911d9"}
09:02:53 [09:02:53.366]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.366]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8813b81b3aa1a422 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8813b81b3aa1a422"}
09:02:53 [09:02:53.366]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:598fcbbc73cdaa67 epochNumber=1
{"provingJobId":"1:BASE_PARITY:598fcbbc73cdaa67"}
09:02:53 [09:02:53.367]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8813b81b3aa1a422 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8813b81b3aa1a422"}
09:02:53 [09:02:53.368]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.368]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c70d99b814c86756 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c70d99b814c86756"}
09:02:53 [09:02:53.369]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:281b0d55a6cd6fa0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:281b0d55a6cd6fa0"}
09:02:53 [09:02:53.369]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c70d99b814c86756 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:c70d99b814c86756"}
09:02:53 [09:02:53.370]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.370]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c0286349f2e0cc2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c0286349f2e0cc2"}
09:02:53 [09:02:53.371]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0327f819c79660e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0327f819c79660e3"}
09:02:53 [09:02:53.371]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:5c0286349f2e0cc2 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:5c0286349f2e0cc2"}
09:02:53 [09:02:53.372]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.372]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c0b8c49efa860016 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c0b8c49efa860016"}
09:02:53 [09:02:53.373]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c0b8c49efa860016
{"provingJobId":"1:BASE_PARITY:c0b8c49efa860016"}
09:02:53 [09:02:53.373]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c0b8c49efa860016 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c0b8c49efa860016"}
09:02:53 [09:02:53.374]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdebe0e75e158cf7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdebe0e75e158cf7"}
09:02:53 [09:02:53.374]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c0b8c49efa860016 has already been completed
{"provingJobId":"1:BASE_PARITY:c0b8c49efa860016"}
09:02:53 [09:02:53.375]
INFO:
prover-client:proving-broker Proving Broker started
09:02:53 [09:02:53.376]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f5eda55ac4d59f5e epochNumber=1
{"provingJobId":"1:BASE_PARITY:f5eda55ac4d59f5e"}
09:02:53 [09:02:53.376]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c7ec5ef9f56521a epochNumber=2
{"provingJobId":"1:BASE_PARITY:1c7ec5ef9f56521a"}
09:02:54 [09:02:54.428]
INFO:
prover-client:proving-broker Proving Broker started
09:02:54 [09:02:54.429]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f5eda55ac4d59f5e settled=false
{"provingJobId":"1:BASE_PARITY:f5eda55ac4d59f5e","status":"pending"}
09:02:54 [09:02:54.429]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1c7ec5ef9f56521a settled=false
{"provingJobId":"1:BASE_PARITY:1c7ec5ef9f56521a","status":"pending"}
09:02:54 [09:02:54.430]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f5eda55ac4d59f5e type=BASE_PARITY startedAt=1750150973376 older agent has taken job
{"provingJobId":"1:BASE_PARITY:f5eda55ac4d59f5e"}
09:02:54 [09:02:54.430]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f5eda55ac4d59f5e type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:f5eda55ac4d59f5e"}
09:02:54 [09:02:54.432]
INFO:
prover-client:proving-broker Proving Broker started
09:02:54 [09:02:54.432]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9891f9e060ea27c9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9891f9e060ea27c9"}
09:02:54 [09:02:54.433]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f486faaa3af03d66 epochNumber=2
{"provingJobId":"1:BASE_PARITY:f486faaa3af03d66"}
09:02:55 [09:02:55.435]
INFO:
prover-client:proving-broker Proving Broker started
09:02:55 [09:02:55.435]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9891f9e060ea27c9 settled=false
{"provingJobId":"1:BASE_PARITY:9891f9e060ea27c9","status":"pending"}
09:02:55 [09:02:55.436]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f486faaa3af03d66 settled=false
{"provingJobId":"1:BASE_PARITY:f486faaa3af03d66","status":"pending"}
09:02:55 [09:02:55.436]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9891f9e060ea27c9 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:9891f9e060ea27c9"}
09:02:55 [09:02:55.438]
INFO:
prover-client:proving-broker Proving Broker started
09:02:55 [09:02:55.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c89d5e59783a01fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:c89d5e59783a01fc"}
09:02:55 [09:02:55.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d89469c1a83764df epochNumber=2
{"provingJobId":"1:BASE_PARITY:d89469c1a83764df"}
09:02:56 [09:02:56.440]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.441]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c89d5e59783a01fc settled=false
{"provingJobId":"1:BASE_PARITY:c89d5e59783a01fc","status":"pending"}
09:02:56 [09:02:56.441]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d89469c1a83764df settled=false
{"provingJobId":"1:BASE_PARITY:d89469c1a83764df","status":"pending"}
09:02:56 [09:02:56.442]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c89d5e59783a01fc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c89d5e59783a01fc"}
09:02:56 [09:02:56.442]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c89d5e59783a01fc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c89d5e59783a01fc"}
09:02:56 [09:02:56.444]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b69f5581e9abcc4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b69f5581e9abcc4"}
09:02:56 [09:02:56.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6afa30f68082c155 epochNumber=2
{"provingJobId":"1:BASE_PARITY:6afa30f68082c155"}
09:02:56 [09:02:56.446]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8b69f5581e9abcc4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8b69f5581e9abcc4"}
09:02:56 [09:02:56.446]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6afa30f68082c155 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6afa30f68082c155"}
09:02:56 [09:02:56.447]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.448]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5f9fba24071926bf epochNumber=1
{"provingJobId":"1:BASE_PARITY:5f9fba24071926bf"}
09:02:56 [09:02:56.448]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60a6c4cd76b8752a epochNumber=2
{"provingJobId":"1:BASE_PARITY:60a6c4cd76b8752a"}
09:02:56 [09:02:56.449]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5f9fba24071926bf type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5f9fba24071926bf"}
09:02:56 [09:02:56.449]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5f9fba24071926bf type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5f9fba24071926bf"}
09:02:56 [09:02:56.449]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:60a6c4cd76b8752a type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:60a6c4cd76b8752a"}
09:02:56 [09:02:56.450]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.451]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:4b88d9f43d24d782 err=test error
{"provingJoId":"1:BASE_PARITY:4b88d9f43d24d782"}
09:02:56 [09:02:56.451]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.452]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0824b21c4fa72909 not found
{"provingJobId":"1:BASE_PARITY:0824b21c4fa72909"}
09:02:56 [09:02:56.453]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.454]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e660655d575a1b40 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e660655d575a1b40"}
09:02:56 [09:02:56.455]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.455]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:19d374c6efd1a9ff epochNumber=1
{"provingJobId":"1:BASE_PARITY:19d374c6efd1a9ff"}
09:02:56 [09:02:56.557]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:19d374c6efd1a9ff timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:19d374c6efd1a9ff"}
09:02:56 [09:02:56.564]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.565]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdcf93f29ff4eb45 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdcf93f29ff4eb45"}
09:02:56 [09:02:56.667]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fdcf93f29ff4eb45 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:fdcf93f29ff4eb45"}
09:02:56 [09:02:56.675]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a685a9da4aae205b epochNumber=10
{"provingJobId":"1:BASE_PARITY:a685a9da4aae205b"}
09:02:56 [09:02:56.693]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:02:56 [09:02:56.802]
INFO:
prover-client:proving-broker Proving Broker started
09:02:56 [09:02:56.803]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6598fb37e1f61102 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6598fb37e1f61102"}
09:02:56 [09:02:56.956]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6598fb37e1f61102 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:6598fb37e1f61102"}
09:02:57 [09:02:57.007]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.008]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:688d0680dfbf1feb epochNumber=1
{"provingJobId":"1:BASE_PARITY:688d0680dfbf1feb"}
09:02:57 [09:02:57.009]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:688d0680dfbf1feb type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:688d0680dfbf1feb"}
09:02:57 [09:02:57.010]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.011]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:84695d71da5c862b epochNumber=1
{"provingJobId":"1:BASE_PARITY:84695d71da5c862b"}
09:02:57 [09:02:57.011]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:84695d71da5c862b type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:84695d71da5c862b"}
09:02:57 [09:02:57.012]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:84695d71da5c862b type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:84695d71da5c862b"}
09:02:57 [09:02:57.013]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.013]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e878f20d755f92df epochNumber=1
{"provingJobId":"1:BASE_PARITY:e878f20d755f92df"}
09:02:57 [09:02:57.014]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e878f20d755f92df type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e878f20d755f92df"}
09:02:57 [09:02:57.015]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b447413f20961cfe epochNumber=1
{"provingJobId":"1:BASE_PARITY:b447413f20961cfe"}
09:02:57 [09:02:57.016]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b447413f20961cfe type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:b447413f20961cfe"}
09:02:57 [09:02:57.017]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ad8c7ff21ed00a8 epochNumber=3
{"provingJobId":"1:BASE_PARITY:9ad8c7ff21ed00a8"}
09:02:57 [09:02:57.040]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:02:57 [09:02:57.042]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b447413f20961cfe err=test error
{"provingJoId":"1:BASE_PARITY:b447413f20961cfe"}
09:02:57 [09:02:57.044]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.045]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:078d3b810635f1be settled=false
{"provingJobId":"1:BASE_PARITY:078d3b810635f1be","status":"pending"}
09:02:57 [09:02:57.045]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:949662da8040907c settled=false
{"provingJobId":"1:BASE_PARITY:949662da8040907c","status":"pending"}
09:02:57 [09:02:57.048]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.048]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:aee95cc1a54861bb settled=true
{"provingJobId":"1:BASE_PARITY:aee95cc1a54861bb","status":"fulfilled"}
09:02:57 [09:02:57.048]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:fdfd17e9e7f525e7 settled=true
{"provingJobId":"2:BASE_PARITY:fdfd17e9e7f525e7","status":"fulfilled"}
09:02:57 [09:02:57.050]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.050]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7e8219f998552f2a settled=true
{"provingJobId":"1:BASE_PARITY:7e8219f998552f2a","status":"fulfilled"}
09:02:57 [09:02:57.051]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9210b7d56554af58 settled=false
{"provingJobId":"1:BASE_PARITY:9210b7d56554af58","status":"pending"}
09:02:57 [09:02:57.052]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.054]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:153b1d8b4694752a epochNumber=1
{"provingJobId":"1:BASE_PARITY:153b1d8b4694752a"}
09:02:57 [09:02:57.057]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d45ed73c9297012b epochNumber=1
{"provingJobId":"1:BASE_PARITY:d45ed73c9297012b"}
09:02:57 [09:02:57.058]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:d45ed73c9297012b: Error: db error: Error: db error
09:02:57 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:d45ed73c9297012b"}
09:02:57 [09:02:57.060]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.060]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7565b92f335b3ad0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7565b92f335b3ad0"}
09:02:57 [09:02:57.061]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7565b92f335b3ad0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7565b92f335b3ad0"}
09:02:57 [09:02:57.061]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7565b92f335b3ad0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7565b92f335b3ad0"}
09:02:57 [09:02:57.062]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.063]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:48d0d2eab764f785 epochNumber=1
{"provingJobId":"1:BASE_PARITY:48d0d2eab764f785"}
09:02:57 [09:02:57.063]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:48d0d2eab764f785 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:48d0d2eab764f785"}
09:02:57 [09:02:57.064]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:48d0d2eab764f785 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:48d0d2eab764f785"}
09:02:57 [09:02:57.064]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:48d0d2eab764f785: Error: db error
09:02:57 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:48d0d2eab764f785"}
09:02:57 [09:02:57.065]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.066]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01ea0039a626b8eb epochNumber=1
{"provingJobId":"1:BASE_PARITY:01ea0039a626b8eb"}
09:02:57 [09:02:57.066]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:01ea0039a626b8eb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:01ea0039a626b8eb"}
09:02:57 [09:02:57.066]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:01ea0039a626b8eb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:01ea0039a626b8eb"}
09:02:57 [09:02:57.067]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.068]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a9af9d0f5d1d489 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a9af9d0f5d1d489"}
09:02:57 [09:02:57.068]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3a9af9d0f5d1d489 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3a9af9d0f5d1d489"}
09:02:57 [09:02:57.068]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3a9af9d0f5d1d489 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3a9af9d0f5d1d489"}
09:02:57 [09:02:57.069]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:3a9af9d0f5d1d489 jobErr=test error: Error: db error
09:02:57 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:3a9af9d0f5d1d489"}
09:02:57 [09:02:57.070]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.070]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:74fac8ce8828f1b7 not found
{"provingJobId":"1:BASE_PARITY:74fac8ce8828f1b7"}
09:02:57 [09:02:57.071]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.072]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:6a0a812658646c24 err=test error
{"provingJoId":"1:BASE_PARITY:6a0a812658646c24"}
09:02:57 [09:02:57.073]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.083]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8afb4807ea33dd71 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8afb4807ea33dd71"}
09:02:57 [09:02:57.084]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8afb4807ea33dd71 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8afb4807ea33dd71"}
09:02:57 [09:02:57.084]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8afb4807ea33dd71 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8afb4807ea33dd71"}
09:02:57 [09:02:57.095]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:d0fa92e6b243b3a5 epochNumber=2
{"provingJobId":"2:BASE_PARITY:d0fa92e6b243b3a5"}
09:02:57 [09:02:57.096]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:d0fa92e6b243b3a5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:d0fa92e6b243b3a5"}
09:02:57 [09:02:57.096]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:d0fa92e6b243b3a5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:d0fa92e6b243b3a5"}
09:02:57 [09:02:57.156]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:d23aaeade782a263 epochNumber=3
{"provingJobId":"3:BASE_PARITY:d23aaeade782a263"}
09:02:57 [09:02:57.174]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:02:57 [09:02:57.218]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:703c6a0de864dcd2 epochNumber=4
{"provingJobId":"4:BASE_PARITY:703c6a0de864dcd2"}
09:02:57 [09:02:57.225]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:02:57 [09:02:57.280]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:7bb6c6fe912d52ad epochNumber=5
{"provingJobId":"5:BASE_PARITY:7bb6c6fe912d52ad"}
09:02:57 [09:02:57.303]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:02:57 [09:02:57.336]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.337]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:769cdbae33386e44 epochNumber=42
{"provingJobId":"1:BASE_PARITY:769cdbae33386e44"}
09:02:57 [09:02:57.338]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testxPley2/42 with map size 1073741824
09:02:57 [09:02:57.342]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.369]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:652839b7daeac8c8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:652839b7daeac8c8"}
09:02:57 [09:02:57.374]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.375]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc741be01c056515 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bc741be01c056515"}
09:02:57 [09:02:57.376]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrPxzXl/1 with map size 1073741824
09:02:57 [09:02:57.378]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.397]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:37b5b514b658eeba epochNumber=1
{"provingJobId":"1:BASE_PARITY:37b5b514b658eeba"}
09:02:57 [09:02:57.408]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.409]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cce4b6af0b41b3e5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cce4b6af0b41b3e5"}
09:02:57 [09:02:57.410]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzys29C/1 with map size 1073741824
09:02:57 [09:02:57.413]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.431]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:cce4b6af0b41b3e5 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:cce4b6af0b41b3e5"}
09:02:57 [09:02:57.435]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.436]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3bf343558a584676 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3bf343558a584676"}
09:02:57 [09:02:57.438]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBYGSnV/1 with map size 1073741824
09:02:57 [09:02:57.441]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.460]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3bf343558a584676 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3bf343558a584676"}
09:02:57 [09:02:57.460]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3bf343558a584676 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3bf343558a584676"}
09:02:57 [09:02:57.467]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3bf343558a584676 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3bf343558a584676"}
09:02:57 [09:02:57.471]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.472]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c622a847a749694 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1c622a847a749694"}
09:02:57 [09:02:57.473]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw4lUtH/1 with map size 1073741824
09:02:57 [09:02:57.475]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.497]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.502]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.502]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0087e4d5376806d0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0087e4d5376806d0"}
09:02:57 [09:02:57.503]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyHAihY/1 with map size 1073741824
09:02:57 [09:02:57.505]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.522]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:0087e4d5376806d0
{"provingJobId":"1:BASE_PARITY:0087e4d5376806d0"}
09:02:57 [09:02:57.522]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0087e4d5376806d0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0087e4d5376806d0"}
09:02:57 [09:02:57.523]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0087e4d5376806d0 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:0087e4d5376806d0"}
09:02:57 [09:02:57.533]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.534]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:221732847b37b5e8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:221732847b37b5e8"}
09:02:57 [09:02:57.534]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testihQtJa/1 with map size 1073741824
09:02:57 [09:02:57.536]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.554]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:221732847b37b5e8
{"provingJobId":"1:BASE_PARITY:221732847b37b5e8"}
09:02:57 [09:02:57.555]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:221732847b37b5e8 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:221732847b37b5e8"}
09:02:57 [09:02:57.563]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.564]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d903401461123e2a epochNumber=1
{"provingJobId":"1:BASE_PARITY:d903401461123e2a"}
09:02:57 [09:02:57.565]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2lw2Ic/1 with map size 1073741824
09:02:57 [09:02:57.567]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.584]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d903401461123e2a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d903401461123e2a"}
09:02:57 [09:02:57.584]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d903401461123e2a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d903401461123e2a"}
09:02:57 [09:02:57.593]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.594]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7fa6fdcb6aaf22cb epochNumber=1
{"provingJobId":"1:BASE_PARITY:7fa6fdcb6aaf22cb"}
09:02:57 [09:02:57.595]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTdduqg/1 with map size 1073741824
09:02:57 [09:02:57.598]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.616]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7fa6fdcb6aaf22cb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7fa6fdcb6aaf22cb"}
09:02:57 [09:02:57.617]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7fa6fdcb6aaf22cb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7fa6fdcb6aaf22cb"}
09:02:57 [09:02:57.626]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.627]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1f38af62c27b6e6c epochNumber=0
{"provingJobId":"1:BASE_PARITY:1f38af62c27b6e6c"}
09:02:57 [09:02:57.627]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testsMjHfR/0 with map size 1073741824
09:02:57 [09:02:57.629]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.654]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f38af62c27b6e6c epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:1f38af62c27b6e6c"}
09:02:57 [09:02:57.656]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:627b5dc7c02cdea0 epochNumber=0
{"provingJobId":"1:BASE_PARITY:627b5dc7c02cdea0"}
09:02:57 [09:02:57.657]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:1f38af62c27b6e6c epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:1f38af62c27b6e6c"}
09:02:57 [09:02:57.657]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:751bfa5e95311867 epochNumber=0
{"provingJobId":"1:BASE_PARITY:751bfa5e95311867"}
09:02:57 [09:02:57.683]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.688]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.689]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eba4c3a9f9842e50 epochNumber=2
{"provingJobId":"1:BASE_PARITY:eba4c3a9f9842e50"}
09:02:57 [09:02:57.690]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiZGWuh/2 with map size 1073741824
09:02:57 [09:02:57.693]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.711]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0cbd7dd9681362f7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0cbd7dd9681362f7"}
09:02:57 [09:02:57.712]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiZGWuh/1 with map size 1073741824
09:02:57 [09:02:57.714]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.735]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.736]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c0601e80e048821 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0c0601e80e048821"}
09:02:57 [09:02:57.736]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWBBwxR/1 with map size 1073741824
09:02:57 [09:02:57.740]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.772]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.773]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:106ea9c5f79fb171 epochNumber=1
{"provingJobId":"1:BASE_PARITY:106ea9c5f79fb171"}
09:02:57 [09:02:57.775]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCuBYhP/1 with map size 1073741824
09:02:57 [09:02:57.778]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.802]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ae6be6b7ad4893b epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ae6be6b7ad4893b"}
09:02:57 [09:02:57.808]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e8225553e8556420 epochNumber=2
{"provingJobId":"1:BASE_PARITY:e8225553e8556420"}
09:02:57 [09:02:57.808]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testCuBYhP/2 with map size 1073741824
09:02:57 [09:02:57.810]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.827]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a618bb664b9bf604 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a618bb664b9bf604"}
09:02:57 [09:02:57.836]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.837]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d2096cd561065e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d2096cd561065e6"}
09:02:57 [09:02:57.838]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testok3TBi/1 with map size 1073741824
09:02:57 [09:02:57.839]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.856]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a383b45bc432174 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a383b45bc432174"}
09:02:57 [09:02:57.867]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fb16c6a273c13d5b epochNumber=2
{"provingJobId":"1:BASE_PARITY:fb16c6a273c13d5b"}
09:02:57 [09:02:57.870]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testok3TBi/2 with map size 1073741824
09:02:57 [09:02:57.875]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.895]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:458a0d074d6ffa6e epochNumber=1
{"provingJobId":"1:BASE_PARITY:458a0d074d6ffa6e"}
09:02:57 [09:02:57.907]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.908]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:52038cedddb013a4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:52038cedddb013a4"}
09:02:57 [09:02:57.909]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyd69S3/1 with map size 1073741824
09:02:57 [09:02:57.912]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.929]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:819aeb5e20553b9b epochNumber=1
{"provingJobId":"1:BASE_PARITY:819aeb5e20553b9b"}
09:02:57 [09:02:57.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:286b898f57cd3ec8 epochNumber=2
{"provingJobId":"1:BASE_PARITY:286b898f57cd3ec8"}
09:02:57 [09:02:57.937]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyd69S3/2 with map size 1073741824
09:02:57 [09:02:57.939]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:57 [09:02:57.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bbbdec0b05565476 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bbbdec0b05565476"}
09:02:57 [09:02:57.991]
INFO:
prover-client:proving-broker Proving Broker started
09:02:57 [09:02:57.992]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d08e70a752b6d4d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d08e70a752b6d4d3"}
09:02:57 [09:02:57.993]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6QjVq3/1 with map size 1073741824
09:02:57 [09:02:57.995]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:58 [09:02:58.016]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7025ce4f0432b8d epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7025ce4f0432b8d"}
09:02:58 [09:02:58.023]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d08e70a752b6d4d3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d08e70a752b6d4d3"}
09:02:58 [09:02:58.033]
INFO:
prover-client:proving-broker Proving Broker started
09:02:58 [09:02:58.034]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:06d70cdd6c267297 epochNumber=1
{"provingJobId":"1:BASE_PARITY:06d70cdd6c267297"}
09:02:58 [09:02:58.035]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZd9HAU/1 with map size 1073741824
09:02:58 [09:02:58.037]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:58 [09:02:58.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c728e885dd4938c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c728e885dd4938c0"}
09:02:58 [09:02:58.065]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:06d70cdd6c267297 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:06d70cdd6c267297"}
09:02:58 [09:02:58.083]
INFO:
prover-client:proving-broker Proving Broker started
09:02:58 [09:02:58.084]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f0d02ec013d19f31 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f0d02ec013d19f31"}
09:02:58 [09:02:58.086]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqjTDIV/1 with map size 1073741824
09:02:58 [09:02:58.089]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:58 [09:02:58.108]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55a1eb19c1af8f4e epochNumber=1
{"provingJobId":"1:BASE_PARITY:55a1eb19c1af8f4e"}
09:02:58 [09:02:58.114]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f0d02ec013d19f31 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:f0d02ec013d19f31"}
09:02:58 [09:02:58.117]
INFO:
prover-client:proving-broker Proving Broker started
09:02:58 [09:02:58.118]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:390621227dd0d3d1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:390621227dd0d3d1"}
09:02:58 [09:02:58.118]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4g2QQ3/1 with map size 1073741824
09:02:58 [09:02:58.121]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:58 [09:02:58.140]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:390621227dd0d3d1
{"provingJobId":"1:BASE_PARITY:390621227dd0d3d1"}
09:02:58 [09:02:58.141]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:390621227dd0d3d1 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:390621227dd0d3d1"}
09:02:58 [09:02:58.147]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:528b569f31029d84 epochNumber=1
{"provingJobId":"1:BASE_PARITY:528b569f31029d84"}
09:02:58 [09:02:58.153]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:390621227dd0d3d1 has already been completed
{"provingJobId":"1:BASE_PARITY:390621227dd0d3d1"}
09:02:58 [09:02:58.157]
INFO:
prover-client:proving-broker Proving Broker started
09:02:58 [09:02:58.158]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f60fbfc6b1df2263 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f60fbfc6b1df2263"}
09:02:58 [09:02:58.164]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfbRu4U/1 with map size 1073741824
09:02:58 [09:02:58.168]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:58 [09:02:58.190]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f5765c1a8ddc845 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7f5765c1a8ddc845"}
09:02:58 [09:02:58.191]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfbRu4U/2 with map size 1073741824
09:02:58 [09:02:58.193]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:59 [09:02:59.267]
INFO:
prover-client:proving-broker Proving Broker started
09:02:59 [09:02:59.280]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f60fbfc6b1df2263 settled=false
{"provingJobId":"1:BASE_PARITY:f60fbfc6b1df2263","status":"pending"}
09:02:59 [09:02:59.283]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7f5765c1a8ddc845 settled=false
{"provingJobId":"1:BASE_PARITY:7f5765c1a8ddc845","status":"pending"}
09:02:59 [09:02:59.284]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f60fbfc6b1df2263 type=BASE_PARITY startedAt=1750150978216 older agent has taken job
{"provingJobId":"1:BASE_PARITY:f60fbfc6b1df2263"}
09:02:59 [09:02:59.285]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f60fbfc6b1df2263 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:f60fbfc6b1df2263"}
09:02:59 [09:02:59.289]
INFO:
prover-client:proving-broker Proving Broker started
09:02:59 [09:02:59.290]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:934c0c078ecaa53f epochNumber=1
{"provingJobId":"1:BASE_PARITY:934c0c078ecaa53f"}
09:02:59 [09:02:59.291]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0xDmBI/1 with map size 1073741824
09:02:59 [09:02:59.294]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:02:59 [09:02:59.316]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:634fa200863c45ab epochNumber=2
{"provingJobId":"1:BASE_PARITY:634fa200863c45ab"}
09:02:59 [09:02:59.317]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test0xDmBI/2 with map size 1073741824
09:02:59 [09:02:59.319]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:00 [09:03:00.336]
INFO:
prover-client:proving-broker Proving Broker started
09:03:00 [09:03:00.338]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:934c0c078ecaa53f settled=false
{"provingJobId":"1:BASE_PARITY:934c0c078ecaa53f","status":"pending"}
09:03:00 [09:03:00.340]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:634fa200863c45ab settled=false
{"provingJobId":"1:BASE_PARITY:634fa200863c45ab","status":"pending"}
09:03:00 [09:03:00.340]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:934c0c078ecaa53f type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:934c0c078ecaa53f"}
09:03:00 [09:03:00.345]
INFO:
prover-client:proving-broker Proving Broker started
09:03:00 [09:03:00.346]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71bf83222e541d78 epochNumber=1
{"provingJobId":"1:BASE_PARITY:71bf83222e541d78"}
09:03:00 [09:03:00.346]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testH5Ydki/1 with map size 1073741824
09:03:00 [09:03:00.348]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:00 [09:03:00.364]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4a8447f2b879716 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a4a8447f2b879716"}
09:03:00 [09:03:00.365]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testH5Ydki/2 with map size 1073741824
09:03:00 [09:03:00.367]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.396]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.398]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:71bf83222e541d78 settled=false
{"provingJobId":"1:BASE_PARITY:71bf83222e541d78","status":"pending"}
09:03:01 [09:03:01.399]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a4a8447f2b879716 settled=false
{"provingJobId":"1:BASE_PARITY:a4a8447f2b879716","status":"pending"}
09:03:01 [09:03:01.400]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:71bf83222e541d78 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:71bf83222e541d78"}
09:03:01 [09:03:01.400]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:71bf83222e541d78 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:71bf83222e541d78"}
09:03:01 [09:03:01.412]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.413]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7784cdf79e10e465 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7784cdf79e10e465"}
09:03:01 [09:03:01.415]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtc4SVS/1 with map size 1073741824
09:03:01 [09:03:01.420]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.443]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:979644842e0c7387 epochNumber=2
{"provingJobId":"1:BASE_PARITY:979644842e0c7387"}
09:03:01 [09:03:01.444]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtc4SVS/2 with map size 1073741824
09:03:01 [09:03:01.448]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.476]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7784cdf79e10e465 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7784cdf79e10e465"}
09:03:01 [09:03:01.484]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:979644842e0c7387 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:979644842e0c7387"}
09:03:01 [09:03:01.496]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.497]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c95d93b425d7c475 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c95d93b425d7c475"}
09:03:01 [09:03:01.498]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4N9Eor/1 with map size 1073741824
09:03:01 [09:03:01.504]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.521]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:754fc855aa8d8116 epochNumber=2
{"provingJobId":"1:BASE_PARITY:754fc855aa8d8116"}
09:03:01 [09:03:01.522]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test4N9Eor/2 with map size 1073741824
09:03:01 [09:03:01.525]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.544]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c95d93b425d7c475 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c95d93b425d7c475"}
09:03:01 [09:03:01.545]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c95d93b425d7c475 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c95d93b425d7c475"}
09:03:01 [09:03:01.551]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:754fc855aa8d8116 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:754fc855aa8d8116"}
09:03:01 [09:03:01.562]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.563]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:555560c7fa6c87b3 err=test error
{"provingJoId":"1:BASE_PARITY:555560c7fa6c87b3"}
09:03:01 [09:03:01.566]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.567]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:832875dda8584247 not found
{"provingJobId":"1:BASE_PARITY:832875dda8584247"}
09:03:01 [09:03:01.573]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.574]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a2b601c1eb688be epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a2b601c1eb688be"}
09:03:01 [09:03:01.575]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkvBgOp/1 with map size 1073741824
09:03:01 [09:03:01.577]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.602]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.603]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f6636237cf349850 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f6636237cf349850"}
09:03:01 [09:03:01.604]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdSBFGC/1 with map size 1073741824
09:03:01 [09:03:01.606]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.756]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f6636237cf349850 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:f6636237cf349850"}
09:03:01 [09:03:01.768]
INFO:
prover-client:proving-broker Proving Broker started
09:03:01 [09:03:01.769]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:857f3fba4f17f255 epochNumber=1
{"provingJobId":"1:BASE_PARITY:857f3fba4f17f255"}
09:03:01 [09:03:01.771]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYxTDI9/1 with map size 1073741824
09:03:01 [09:03:01.773]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.895]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:857f3fba4f17f255 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:857f3fba4f17f255"}
09:03:01 [09:03:01.900]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:618a732d1d262586 epochNumber=10
{"provingJobId":"1:BASE_PARITY:618a732d1d262586"}
09:03:01 [09:03:01.901]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testYxTDI9/10 with map size 1073741824
09:03:01 [09:03:01.905]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:01 [09:03:01.947]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:03:01 [09:03:01.947]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:03:02 [09:03:02.067]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.069]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6fd0f69b8240c8ee epochNumber=1
{"provingJobId":"1:BASE_PARITY:6fd0f69b8240c8ee"}
09:03:02 [09:03:02.071]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2pxECa/1 with map size 1073741824
09:03:02 [09:03:02.074]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.250]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6fd0f69b8240c8ee timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:6fd0f69b8240c8ee"}
09:03:02 [09:03:02.306]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.307]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f0919ed53cb26c55 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f0919ed53cb26c55"}
09:03:02 [09:03:02.308]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testT13nrD/1 with map size 1073741824
09:03:02 [09:03:02.310]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.328]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f0919ed53cb26c55 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:f0919ed53cb26c55"}
09:03:02 [09:03:02.332]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae96283f6dd977d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae96283f6dd977d5"}
09:03:02 [09:03:02.334]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzEPRpB/1 with map size 1073741824
09:03:02 [09:03:02.336]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.358]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ae96283f6dd977d5 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:ae96283f6dd977d5"}
09:03:02 [09:03:02.359]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ae96283f6dd977d5 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:ae96283f6dd977d5"}
09:03:02 [09:03:02.372]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.373]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d87b94ee9005ccb2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d87b94ee9005ccb2"}
09:03:02 [09:03:02.374]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbR8rqd/1 with map size 1073741824
09:03:02 [09:03:02.377]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.397]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d87b94ee9005ccb2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d87b94ee9005ccb2"}
09:03:02 [09:03:02.408]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.409]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd61d33e3cff786d epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd61d33e3cff786d"}
09:03:02 [09:03:02.410]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testN3AyCv/1 with map size 1073741824
09:03:02 [09:03:02.413]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.432]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fd61d33e3cff786d type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:fd61d33e3cff786d"}
09:03:02 [09:03:02.433]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7acd3feca863d6bd epochNumber=3
{"provingJobId":"1:BASE_PARITY:7acd3feca863d6bd"}
09:03:02 [09:03:02.434]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testN3AyCv/3 with map size 1073741824
09:03:02 [09:03:02.440]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.485]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:03:02 [09:03:02.486]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:03:02 [09:03:02.494]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:fd61d33e3cff786d err=test error
{"provingJoId":"1:BASE_PARITY:fd61d33e3cff786d"}
09:03:02 [09:03:02.502]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzsS0UL/1 with map size 1073741824
09:03:02 [09:03:02.504]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.524]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzsS0UL/2 with map size 1073741824
09:03:02 [09:03:02.526]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.544]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.546]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:75d24587dfc6fad1 settled=false
{"provingJobId":"1:BASE_PARITY:75d24587dfc6fad1","status":"pending"}
09:03:02 [09:03:02.548]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:49e108d984eac9b0 settled=false
{"provingJobId":"1:BASE_PARITY:49e108d984eac9b0","status":"pending"}
09:03:02 [09:03:02.556]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg5Sno9/1 with map size 1073741824
09:03:02 [09:03:02.560]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.580]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testg5Sno9/2 with map size 1073741824
09:03:02 [09:03:02.583]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.611]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.614]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fc6ace14591dbc12 settled=true
{"provingJobId":"1:BASE_PARITY:fc6ace14591dbc12","status":"fulfilled"}
09:03:02 [09:03:02.617]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:277274f4e1448fca settled=true
{"provingJobId":"2:BASE_PARITY:277274f4e1448fca","status":"fulfilled"}
09:03:02 [09:03:02.623]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPan1Nm/1 with map size 1073741824
09:03:02 [09:03:02.625]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.658]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPan1Nm/2 with map size 1073741824
09:03:02 [09:03:02.662]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.682]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.685]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f6f3209885b2b4ac settled=true
{"provingJobId":"1:BASE_PARITY:f6f3209885b2b4ac","status":"fulfilled"}
09:03:02 [09:03:02.687]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:59ba9f6e05c1569b settled=false
{"provingJobId":"1:BASE_PARITY:59ba9f6e05c1569b","status":"pending"}
09:03:02 [09:03:02.693]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.694]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a455de20123e0de8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a455de20123e0de8"}
09:03:02 [09:03:02.696]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZe5wc7/1 with map size 1073741824
09:03:02 [09:03:02.700]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.731]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.732]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d3c66ca5469563b epochNumber=1
{"provingJobId":"1:BASE_PARITY:1d3c66ca5469563b"}
09:03:02 [09:03:02.733]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:1d3c66ca5469563b: Error: db error: Error: db error
09:03:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:1d3c66ca5469563b"}
09:03:02 [09:03:02.739]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.740]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b6ba4f00691061a7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b6ba4f00691061a7"}
09:03:02 [09:03:02.741]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJFKXsi/1 with map size 1073741824
09:03:02 [09:03:02.744]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.766]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b6ba4f00691061a7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b6ba4f00691061a7"}
09:03:02 [09:03:02.767]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b6ba4f00691061a7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b6ba4f00691061a7"}
09:03:02 [09:03:02.779]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.780]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b48e0c57d0ede78b epochNumber=1
{"provingJobId":"1:BASE_PARITY:b48e0c57d0ede78b"}
09:03:02 [09:03:02.781]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testifbV8c/1 with map size 1073741824
09:03:02 [09:03:02.784]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.808]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b48e0c57d0ede78b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b48e0c57d0ede78b"}
09:03:02 [09:03:02.808]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b48e0c57d0ede78b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b48e0c57d0ede78b"}
09:03:02 [09:03:02.809]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:b48e0c57d0ede78b: Error: db error
09:03:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:b48e0c57d0ede78b"}
09:03:02 [09:03:02.815]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.816]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:585c4d8b6b55cf2e epochNumber=1
{"provingJobId":"1:BASE_PARITY:585c4d8b6b55cf2e"}
09:03:02 [09:03:02.818]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5AYFBw/1 with map size 1073741824
09:03:02 [09:03:02.821]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.847]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:585c4d8b6b55cf2e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:585c4d8b6b55cf2e"}
09:03:02 [09:03:02.847]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:585c4d8b6b55cf2e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:585c4d8b6b55cf2e"}
09:03:02 [09:03:02.858]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.859]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01cefb97caf012ae epochNumber=1
{"provingJobId":"1:BASE_PARITY:01cefb97caf012ae"}
09:03:02 [09:03:02.861]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmJbPFs/1 with map size 1073741824
09:03:02 [09:03:02.863]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.880]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:01cefb97caf012ae type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:01cefb97caf012ae"}
09:03:02 [09:03:02.881]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:01cefb97caf012ae type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:01cefb97caf012ae"}
09:03:02 [09:03:02.882]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:01cefb97caf012ae jobErr=test error: Error: db error
09:03:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:01cefb97caf012ae"}
09:03:02 [09:03:02.888]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.888]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a00066d396d2f104 not found
{"provingJobId":"1:BASE_PARITY:a00066d396d2f104"}
09:03:02 [09:03:02.892]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.893]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:112c82a47e12ef8b err=test error
{"provingJoId":"1:BASE_PARITY:112c82a47e12ef8b"}
09:03:02 [09:03:02.898]
INFO:
prover-client:proving-broker Proving Broker started
09:03:02 [09:03:02.908]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2bf6cfb9b3d23a0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:2bf6cfb9b3d23a0f"}
09:03:02 [09:03:02.909]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHr5BQN/1 with map size 1073741824
09:03:02 [09:03:02.912]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.930]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2bf6cfb9b3d23a0f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2bf6cfb9b3d23a0f"}
09:03:02 [09:03:02.931]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2bf6cfb9b3d23a0f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2bf6cfb9b3d23a0f"}
09:03:02 [09:03:02.947]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:a18a38d9e93f20e0 epochNumber=2
{"provingJobId":"2:BASE_PARITY:a18a38d9e93f20e0"}
09:03:02 [09:03:02.948]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testHr5BQN/2 with map size 1073741824
09:03:02 [09:03:02.950]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:02 [09:03:02.972]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:a18a38d9e93f20e0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:a18a38d9e93f20e0"}
09:03:02 [09:03:02.972]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:a18a38d9e93f20e0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:a18a38d9e93f20e0"}
09:03:03 [09:03:03.039]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:ab00dd26569acd53 epochNumber=3
{"provingJobId":"3:BASE_PARITY:ab00dd26569acd53"}
09:03:03 [09:03:03.042]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testHr5BQN/3 with map size 1073741824
09:03:03 [09:03:03.044]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:03 [09:03:03.075]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:03:03 [09:03:03.076]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:03:03 [09:03:03.124]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:cb54dd1aa6ca7361 epochNumber=4
{"provingJobId":"4:BASE_PARITY:cb54dd1aa6ca7361"}
09:03:03 [09:03:03.126]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testHr5BQN/4 with map size 1073741824
09:03:03 [09:03:03.130]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:03 [09:03:03.181]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:03:03 [09:03:03.181]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
09:03:03 [09:03:03.217]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:69d77d40ab0664b2 epochNumber=5
{"provingJobId":"5:BASE_PARITY:69d77d40ab0664b2"}
09:03:03 [09:03:03.219]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testHr5BQN/5 with map size 1073741824
09:03:03 [09:03:03.222]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:03:03 [09:03:03.261]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:03:03 [09:03:03.262]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
09:03:03
PASS src/proving_broker/proving_broker.test.ts (
13.571 s)
09:03:03 ProvingBroker
09:03:03 Producer API
09:03:03
✓ refuses stale jobs (45 ms)
09:03:03
✓ enqueues jobs (4 ms)
09:03:03
✓ ignores duplicate jobs (4 ms)
09:03:03
✓ reports correct status when enqueuing repeat jobs (4 ms)
09:03:03
✓ throws an error in case of duplicate job IDs (20 ms)
09:03:03
✓ returns not-found status for non-existing jobs (1 ms)
09:03:03
✓ cancels jobs in queue (4 ms)
09:03:03
✓ cancels jobs in-progress (2 ms)
09:03:03
✓ returns job result if successful (2 ms)
09:03:03
✓ returns job error if failed (11 ms)
09:03:03
✓ correctly returns job status for concurrent writes (2 ms)
09:03:03
✓ refuses stale jobs (39 ms)
09:03:03
✓ enqueues jobs (33 ms)
09:03:03
✓ ignores duplicate jobs (27 ms)
09:03:03
✓ reports correct status when enqueuing repeat jobs (35 ms)
09:03:03
✓ throws an error in case of duplicate job IDs (26 ms)
09:03:03
✓ returns not-found status for non-existing jobs (3 ms)
09:03:03
✓ cancels jobs in queue (33 ms)
09:03:03
✓ cancels jobs in-progress (30 ms)
09:03:03
✓ returns job result if successful (29 ms)
09:03:03
✓ returns job error if failed (32 ms)
09:03:03
✓ correctly returns job status for concurrent writes (55 ms)
09:03:03 Consumer API
09:03:03
✓ returns undefined if no jobs are available (1 ms)
09:03:03
✓ returns jobs in priority order (2 ms)
09:03:03
✓ returns undefined if no jobs are available for the given allowList (1 ms)
09:03:03
✓ returns a job if it is in the allowList (2 ms)
09:03:03
✓ returns the most important job if it is in the allowList (2 ms)
09:03:03
✓ returns any job if filter is empty (2 ms)
09:03:03
✓ returns a new job when reporting job success (2 ms)
09:03:03
✓ returns a new job when reporting permanent error (2 ms)
09:03:03
✓ returns a new job when reporting retry-able error (2 ms)
09:03:03
✓ returns a new job when reporting progress if current one is cancelled (2 ms)
09:03:03
✓ returns a new job if job is already in progress elsewhere (1056 ms)
09:03:03
✓ avoids sending the same job to a new agent after a restart (1006 ms)
09:03:03
✓ avoids sending a completed job to a new agent after a restart (1005 ms)
09:03:03
✓ tracks job result if in progress (3 ms)
09:03:03
✓ tracks job result even if job is in queue (3 ms)
09:03:03
✓ ignores reported job error if unknown job (1 ms)
09:03:03
✓ ignores job result if unknown job (1 ms)
09:03:03
✓ returns undefined if no jobs are available (4 ms)
09:03:03
✓ returns jobs in priority order (48 ms)
09:03:03
✓ returns undefined if no jobs are available for the given allowList (31 ms)
09:03:03
✓ returns a job if it is in the allowList (68 ms)
09:03:03
✓ returns the most important job if it is in the allowList (68 ms)
09:03:03
✓ returns any job if filter is empty (81 ms)
09:03:03
✓ returns a new job when reporting job success (47 ms)
09:03:03
✓ returns a new job when reporting permanent error (49 ms)
09:03:03
✓ returns a new job when reporting retry-able error (34 ms)
09:03:03
✓ returns a new job when reporting progress if current one is cancelled (39 ms)
09:03:03
✓ returns a new job if job is already in progress elsewhere (1131 ms)
09:03:03
✓ avoids sending the same job to a new agent after a restart (1055 ms)
09:03:03
✓ avoids sending a completed job to a new agent after a restart (1066 ms)
09:03:03
✓ tracks job result if in progress (83 ms)
09:03:03
✓ tracks job result even if job is in queue (65 ms)
09:03:03
✓ ignores reported job error if unknown job (4 ms)
09:03:03
✓ ignores job result if unknown job (4 ms)
09:03:03 Timeouts
09:03:03
✓ tracks in progress jobs (1 ms)
09:03:03
✓ re-enqueues jobs that time out (109 ms)
09:03:03
✓ cancel stale jobs that time out (237 ms)
09:03:03
✓ keeps the jobs in progress while it is alive (204 ms)
09:03:03
✓ tracks in progress jobs (32 ms)
09:03:03
✓ re-enqueues jobs that time out (163 ms)
09:03:03
✓ cancel stale jobs that time out (297 ms)
09:03:03
✓ keeps the jobs in progress while it is alive (240 ms)
09:03:03 Retries
09:03:03
✓ retries jobs (3 ms)
09:03:03
✓ retries up to a maximum number of times (2 ms)
09:03:03
✓ passing retry=false does not retry (1 ms)
09:03:03
✓ does not retry if job is stale (28 ms)
09:03:03
✓ retries jobs (28 ms)
09:03:03
✓ retries up to a maximum number of times (38 ms)
09:03:03
✓ passing retry=false does not retry (36 ms)
09:03:03
✓ does not retry if job is stale (91 ms)
09:03:03 Database management
09:03:03
✓ re-enqueues proof requests on start (4 ms)
09:03:03
✓ restores proof results on start (2 ms)
09:03:03
✓ only re-enqueues unfinished jobs (2 ms)
09:03:03
✓ saves job when enqueued (4 ms)
09:03:03
✓ does not retain job if database fails to save (3 ms)
09:03:03
✓ saves job result (3 ms)
09:03:03
✓ saves result even if database fails to save (3 ms)
09:03:03
✓ saves job error (2 ms)
09:03:03
✓ saves job error even if database fails to save (2 ms)
09:03:03
✓ does not save job result if job is unknown (2 ms)
09:03:03
✓ does not save job error if job is unknown (1 ms)
09:03:03
✓ cleans up old jobs periodically (260 ms)
09:03:03
✓ re-enqueues proof requests on start (54 ms)
09:03:03
✓ restores proof results on start (67 ms)
09:03:03
✓ only re-enqueues unfinished jobs (70 ms)
09:03:03
✓ saves job when enqueued (38 ms)
09:03:03
✓ does not retain job if database fails to save (7 ms)
09:03:03
✓ saves job result (41 ms)
09:03:03
✓ saves result even if database fails to save (36 ms)
09:03:03
✓ saves job error (43 ms)
09:03:03
✓ saves job error even if database fails to save (29 ms)
09:03:03
✓ does not save job result if job is unknown (4 ms)
09:03:03
✓ does not save job error if job is unknown (4 ms)
09:03:03
✓ cleans up old jobs periodically (403 ms)
09:03:03
09:03:03
Test Suites: 1 passed, 1 total
09:03:03
Tests: 96 passed, 96 total
09:03:03
Snapshots: 0 total
09:03:03
Time: 13.688 s
09:03:03
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
09:03:03
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?