Parent Log:
http://ci.aztec-labs.com/2d6245e20328508a
Command: 3f1d1d87da1584d1 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/e742ab186f29d74148403272e50f75bb77cbb8a8
Env: REF_NAME=gh-readonly-queue/next/pr-15051-92d685c3a802eb31897b6f13d28b9ee428c73744 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Thu Jun 19 22:32:15 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15051_amd64_x4-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
22:32:19 [22:32:19.918]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.924]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:61dc6c2c31e43292 epochNumber=42
{"provingJobId":"1:BASE_PARITY:61dc6c2c31e43292"}
22:32:19 [22:32:19.928]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:666ea3d498b4c020 epochNumber=1
{"provingJobId":"1:BASE_PARITY:666ea3d498b4c020"}
22:32:19 [22:32:19.963]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.964]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:137100d2c5920bf9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:137100d2c5920bf9"}
22:32:19 [22:32:19.965]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9743ffb7b1f1c400 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9743ffb7b1f1c400"}
22:32:19 [22:32:19.966]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.967]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4220cb45068c2289 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4220cb45068c2289"}
22:32:19 [22:32:19.967]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4220cb45068c2289 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:4220cb45068c2289"}
22:32:19 [22:32:19.969]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f316b2325d1de1f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f316b2325d1de1f9"}
22:32:19 [22:32:19.970]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f316b2325d1de1f9 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f316b2325d1de1f9"}
22:32:19 [22:32:19.971]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f316b2325d1de1f9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f316b2325d1de1f9"}
22:32:19 [22:32:19.971]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f316b2325d1de1f9 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f316b2325d1de1f9"}
22:32:19 [22:32:19.973]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.973]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6692193a27461b86 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6692193a27461b86"}
22:32:19 [22:32:19.992]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.993]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.994]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0cf06eda931111a epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0cf06eda931111a"}
22:32:19 [22:32:19.994]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e0cf06eda931111a
{"provingJobId":"1:BASE_PARITY:e0cf06eda931111a"}
22:32:19 [22:32:19.995]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e0cf06eda931111a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e0cf06eda931111a"}
22:32:19 [22:32:19.995]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e0cf06eda931111a type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:e0cf06eda931111a"}
22:32:19 [22:32:19.996]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.997]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd8943ad01e0a851 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd8943ad01e0a851"}
22:32:19 [22:32:19.997]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:bd8943ad01e0a851
{"provingJobId":"1:BASE_PARITY:bd8943ad01e0a851"}
22:32:19 [22:32:19.998]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bd8943ad01e0a851 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:bd8943ad01e0a851"}
22:32:19 [22:32:19.999]
INFO:
prover-client:proving-broker Proving Broker started
22:32:19 [22:32:19.999]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68594a71bc934f69 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68594a71bc934f69"}
22:32:19 [22:32:20.000]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:68594a71bc934f69 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:68594a71bc934f69"}
22:32:19 [22:32:20.000]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:68594a71bc934f69 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:68594a71bc934f69"}
22:32:20 [22:32:20.001]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.002]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3c994bfe6e3961c epochNumber=1
{"provingJobId":"1:BASE_PARITY:b3c994bfe6e3961c"}
22:32:20 [22:32:20.002]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b3c994bfe6e3961c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b3c994bfe6e3961c"}
22:32:20 [22:32:20.002]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b3c994bfe6e3961c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:b3c994bfe6e3961c"}
22:32:20 [22:32:20.003]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.004]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:146b6b8d5226bd77 epochNumber=0
{"provingJobId":"1:BASE_PARITY:146b6b8d5226bd77"}
22:32:20 [22:32:20.004]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:146b6b8d5226bd77 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:146b6b8d5226bd77"}
22:32:20 [22:32:20.004]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8baabaa07b05a748 epochNumber=0
{"provingJobId":"1:BASE_PARITY:8baabaa07b05a748"}
22:32:20 [22:32:20.004]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:146b6b8d5226bd77 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:146b6b8d5226bd77"}
22:32:20 [22:32:20.005]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f7c8bf3d7984d318 epochNumber=0
{"provingJobId":"1:BASE_PARITY:f7c8bf3d7984d318"}
22:32:20 [22:32:20.006]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.014]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8867f61afc1dfa53 epochNumber=2
{"provingJobId":"1:BASE_PARITY:8867f61afc1dfa53"}
22:32:20 [22:32:20.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:864e78546d35c587 epochNumber=1
{"provingJobId":"1:BASE_PARITY:864e78546d35c587"}
22:32:20 [22:32:20.016]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.017]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5901c4e708bfee09 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5901c4e708bfee09"}
22:32:20 [22:32:20.018]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.018]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7702a3c9d423a7f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7702a3c9d423a7f9"}
22:32:20 [22:32:20.019]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:897baa6f4abcec43 epochNumber=1
{"provingJobId":"1:BASE_PARITY:897baa6f4abcec43"}
22:32:20 [22:32:20.019]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ea8d6491837d9a90 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ea8d6491837d9a90"}
22:32:20 [22:32:20.019]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55edcca87c534a01 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55edcca87c534a01"}
22:32:20 [22:32:20.021]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.021]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a00a7a804699dfa0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a00a7a804699dfa0"}
22:32:20 [22:32:20.021]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:efe82ae7783f6aa2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:efe82ae7783f6aa2"}
22:32:20 [22:32:20.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7ba2134bb6f4647 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d7ba2134bb6f4647"}
22:32:20 [22:32:20.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57e3808336ad8221 epochNumber=1
{"provingJobId":"1:BASE_PARITY:57e3808336ad8221"}
22:32:20 [22:32:20.023]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c98344851810eb24 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c98344851810eb24"}
22:32:20 [22:32:20.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0a45ead8945a322 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0a45ead8945a322"}
22:32:20 [22:32:20.024]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4ac22ff91aff395d epochNumber=2
{"provingJobId":"1:BASE_PARITY:4ac22ff91aff395d"}
22:32:20 [22:32:20.024]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b31e78e070cfa22a epochNumber=1
{"provingJobId":"1:BASE_PARITY:b31e78e070cfa22a"}
22:32:20 [22:32:20.025]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.026]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c4370f632e529f2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0c4370f632e529f2"}
22:32:20 [22:32:20.026]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0cbf1a35bf4d925 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0cbf1a35bf4d925"}
22:32:20 [22:32:20.026]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0c4370f632e529f2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0c4370f632e529f2"}
22:32:20 [22:32:20.028]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.028]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:58571c43a10e92b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:58571c43a10e92b4"}
22:32:20 [22:32:20.029]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1ef61353117e0e3b epochNumber=1
{"provingJobId":"1:BASE_PARITY:1ef61353117e0e3b"}
22:32:20 [22:32:20.029]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:58571c43a10e92b4 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:58571c43a10e92b4"}
22:32:20 [22:32:20.030]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.031]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25e8fbbac7a7f17e epochNumber=1
{"provingJobId":"1:BASE_PARITY:25e8fbbac7a7f17e"}
22:32:20 [22:32:20.031]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2e4b02ea4e856ae epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2e4b02ea4e856ae"}
22:32:20 [22:32:20.032]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:25e8fbbac7a7f17e type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:25e8fbbac7a7f17e"}
22:32:20 [22:32:20.033]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.033]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdfaffd905bd671a epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdfaffd905bd671a"}
22:32:20 [22:32:20.034]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:fdfaffd905bd671a
{"provingJobId":"1:BASE_PARITY:fdfaffd905bd671a"}
22:32:20 [22:32:20.034]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fdfaffd905bd671a type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:fdfaffd905bd671a"}
22:32:20 [22:32:20.035]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b06fb30295960cde epochNumber=1
{"provingJobId":"1:BASE_PARITY:b06fb30295960cde"}
22:32:20 [22:32:20.035]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fdfaffd905bd671a has already been completed
{"provingJobId":"1:BASE_PARITY:fdfaffd905bd671a"}
22:32:20 [22:32:20.036]
INFO:
prover-client:proving-broker Proving Broker started
22:32:20 [22:32:20.037]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cab9c872f64031d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cab9c872f64031d7"}
22:32:20 [22:32:20.037]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d9d46650cb77bbfe epochNumber=2
{"provingJobId":"1:BASE_PARITY:d9d46650cb77bbfe"}
22:32:21 [22:32:21.091]
INFO:
prover-client:proving-broker Proving Broker started
22:32:21 [22:32:21.091]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cab9c872f64031d7 settled=false
{"provingJobId":"1:BASE_PARITY:cab9c872f64031d7","status":"pending"}
22:32:21 [22:32:21.092]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d9d46650cb77bbfe settled=false
{"provingJobId":"1:BASE_PARITY:d9d46650cb77bbfe","status":"pending"}
22:32:21 [22:32:21.093]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cab9c872f64031d7 type=BASE_PARITY startedAt=1750372340037 older agent has taken job
{"provingJobId":"1:BASE_PARITY:cab9c872f64031d7"}
22:32:21 [22:32:21.093]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cab9c872f64031d7 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:cab9c872f64031d7"}
22:32:21 [22:32:21.094]
INFO:
prover-client:proving-broker Proving Broker started
22:32:21 [22:32:21.095]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9d65cdf947d16b20 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9d65cdf947d16b20"}
22:32:21 [22:32:21.095]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8a25f956960eb36c epochNumber=2
{"provingJobId":"1:BASE_PARITY:8a25f956960eb36c"}
22:32:22 [22:32:22.096]
INFO:
prover-client:proving-broker Proving Broker started
22:32:22 [22:32:22.097]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9d65cdf947d16b20 settled=false
{"provingJobId":"1:BASE_PARITY:9d65cdf947d16b20","status":"pending"}
22:32:22 [22:32:22.097]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8a25f956960eb36c settled=false
{"provingJobId":"1:BASE_PARITY:8a25f956960eb36c","status":"pending"}
22:32:22 [22:32:22.098]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9d65cdf947d16b20 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:9d65cdf947d16b20"}
22:32:22 [22:32:22.099]
INFO:
prover-client:proving-broker Proving Broker started
22:32:22 [22:32:22.099]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a64e7fbfd0fc9424 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a64e7fbfd0fc9424"}
22:32:22 [22:32:22.099]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:574c2c770d46d8e2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:574c2c770d46d8e2"}
22:32:23 [22:32:23.100]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.101]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a64e7fbfd0fc9424 settled=false
{"provingJobId":"1:BASE_PARITY:a64e7fbfd0fc9424","status":"pending"}
22:32:23 [22:32:23.101]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:574c2c770d46d8e2 settled=false
{"provingJobId":"1:BASE_PARITY:574c2c770d46d8e2","status":"pending"}
22:32:23 [22:32:23.101]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a64e7fbfd0fc9424 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a64e7fbfd0fc9424"}
22:32:23 [22:32:23.102]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a64e7fbfd0fc9424 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a64e7fbfd0fc9424"}
22:32:23 [22:32:23.103]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.103]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7e577c14a2c562a7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e577c14a2c562a7"}
22:32:23 [22:32:23.103]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b3b735abc71567c epochNumber=2
{"provingJobId":"1:BASE_PARITY:9b3b735abc71567c"}
22:32:23 [22:32:23.104]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7e577c14a2c562a7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7e577c14a2c562a7"}
22:32:23 [22:32:23.104]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9b3b735abc71567c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9b3b735abc71567c"}
22:32:23 [22:32:23.105]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.105]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0265d81be028636d epochNumber=1
{"provingJobId":"1:BASE_PARITY:0265d81be028636d"}
22:32:23 [22:32:23.106]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dae36ff212965228 epochNumber=2
{"provingJobId":"1:BASE_PARITY:dae36ff212965228"}
22:32:23 [22:32:23.106]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0265d81be028636d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0265d81be028636d"}
22:32:23 [22:32:23.106]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0265d81be028636d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0265d81be028636d"}
22:32:23 [22:32:23.106]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dae36ff212965228 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:dae36ff212965228"}
22:32:23 [22:32:23.107]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.107]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7d90598354083e7c err=test error
{"provingJoId":"1:BASE_PARITY:7d90598354083e7c"}
22:32:23 [22:32:23.108]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.109]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:03ca0ee30e431cc2 not found
{"provingJobId":"1:BASE_PARITY:03ca0ee30e431cc2"}
22:32:23 [22:32:23.110]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.110]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aadad81eeac283e8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aadad81eeac283e8"}
22:32:23 [22:32:23.111]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.111]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:46cb7d4c7464397e epochNumber=1
{"provingJobId":"1:BASE_PARITY:46cb7d4c7464397e"}
22:32:23 [22:32:23.213]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:46cb7d4c7464397e timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:46cb7d4c7464397e"}
22:32:23 [22:32:23.219]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.219]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0dc7035d5184b97b epochNumber=1
{"provingJobId":"1:BASE_PARITY:0dc7035d5184b97b"}
22:32:23 [22:32:23.320]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0dc7035d5184b97b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:0dc7035d5184b97b"}
22:32:23 [22:32:23.328]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc89aac5a038eb42 epochNumber=10
{"provingJobId":"1:BASE_PARITY:fc89aac5a038eb42"}
22:32:23 [22:32:23.347]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:23 [22:32:23.454]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.455]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:574459bcd6d54cde epochNumber=1
{"provingJobId":"1:BASE_PARITY:574459bcd6d54cde"}
22:32:23 [22:32:23.611]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:574459bcd6d54cde timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:574459bcd6d54cde"}
22:32:23 [22:32:23.657]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.658]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f4c4a732b45cb147 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f4c4a732b45cb147"}
22:32:23 [22:32:23.658]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f4c4a732b45cb147 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:f4c4a732b45cb147"}
22:32:23 [22:32:23.659]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:39c0eb82e5b20c24 epochNumber=1
{"provingJobId":"1:BASE_PARITY:39c0eb82e5b20c24"}
22:32:23 [22:32:23.660]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:39c0eb82e5b20c24 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:39c0eb82e5b20c24"}
22:32:23 [22:32:23.661]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:39c0eb82e5b20c24 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:39c0eb82e5b20c24"}
22:32:23 [22:32:23.662]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.662]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fefcad2ee2167446 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fefcad2ee2167446"}
22:32:23 [22:32:23.662]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fefcad2ee2167446 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fefcad2ee2167446"}
22:32:23 [22:32:23.663]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.663]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec6f2245415c4090 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ec6f2245415c4090"}
22:32:23 [22:32:23.664]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ec6f2245415c4090 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:ec6f2245415c4090"}
22:32:23 [22:32:23.664]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d60e4638291896f9 epochNumber=3
{"provingJobId":"1:BASE_PARITY:d60e4638291896f9"}
22:32:23 [22:32:23.689]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:23 [22:32:23.690]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ec6f2245415c4090 err=test error
{"provingJoId":"1:BASE_PARITY:ec6f2245415c4090"}
22:32:23 [22:32:23.692]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.692]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e29f9b372f69238b settled=false
{"provingJobId":"1:BASE_PARITY:e29f9b372f69238b","status":"pending"}
22:32:23 [22:32:23.693]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f4135862acec116b settled=false
{"provingJobId":"1:BASE_PARITY:f4135862acec116b","status":"pending"}
22:32:23 [22:32:23.695]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.696]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3492736f275fa135 settled=true
{"provingJobId":"1:BASE_PARITY:3492736f275fa135","status":"fulfilled"}
22:32:23 [22:32:23.696]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:120ee564c371fc9e settled=true
{"provingJobId":"2:BASE_PARITY:120ee564c371fc9e","status":"fulfilled"}
22:32:23 [22:32:23.698]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.698]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:08c2fe3cddc627a0 settled=true
{"provingJobId":"1:BASE_PARITY:08c2fe3cddc627a0","status":"fulfilled"}
22:32:23 [22:32:23.699]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:393e95e54b6dba00 settled=false
{"provingJobId":"1:BASE_PARITY:393e95e54b6dba00","status":"pending"}
22:32:23 [22:32:23.700]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.701]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c686bda3dea82864 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c686bda3dea82864"}
22:32:23 [22:32:23.703]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.703]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c560e0c19d554ce5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c560e0c19d554ce5"}
22:32:23 [22:32:23.704]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:c560e0c19d554ce5: Error: db error: Error: db error
22:32:23 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:c560e0c19d554ce5"}
22:32:23 [22:32:23.705]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.706]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25c5e5ada3b28764 epochNumber=1
{"provingJobId":"1:BASE_PARITY:25c5e5ada3b28764"}
22:32:23 [22:32:23.706]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:25c5e5ada3b28764 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:25c5e5ada3b28764"}
22:32:23 [22:32:23.706]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:25c5e5ada3b28764 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:25c5e5ada3b28764"}
22:32:23 [22:32:23.708]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.708]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f6cd63b37e4fe28 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2f6cd63b37e4fe28"}
22:32:23 [22:32:23.708]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2f6cd63b37e4fe28 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2f6cd63b37e4fe28"}
22:32:23 [22:32:23.709]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2f6cd63b37e4fe28 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2f6cd63b37e4fe28"}
22:32:23 [22:32:23.709]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:2f6cd63b37e4fe28: Error: db error
22:32:23 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:2f6cd63b37e4fe28"}
22:32:23 [22:32:23.710]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.711]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e282d8ca00dbdbe epochNumber=1
{"provingJobId":"1:BASE_PARITY:9e282d8ca00dbdbe"}
22:32:23 [22:32:23.711]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9e282d8ca00dbdbe type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9e282d8ca00dbdbe"}
22:32:23 [22:32:23.711]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9e282d8ca00dbdbe type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9e282d8ca00dbdbe"}
22:32:23 [22:32:23.713]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.713]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a8dd1e8aadfc66be epochNumber=1
{"provingJobId":"1:BASE_PARITY:a8dd1e8aadfc66be"}
22:32:23 [22:32:23.713]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a8dd1e8aadfc66be type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a8dd1e8aadfc66be"}
22:32:23 [22:32:23.714]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a8dd1e8aadfc66be type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a8dd1e8aadfc66be"}
22:32:23 [22:32:23.714]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:a8dd1e8aadfc66be jobErr=test error: Error: db error
22:32:23 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:a8dd1e8aadfc66be"}
22:32:23 [22:32:23.716]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.716]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:61c0fefa8ff1f7e5 not found
{"provingJobId":"1:BASE_PARITY:61c0fefa8ff1f7e5"}
22:32:23 [22:32:23.718]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.718]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d293c50180d8868e err=test error
{"provingJoId":"1:BASE_PARITY:d293c50180d8868e"}
22:32:23 [22:32:23.719]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.730]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:13a0709ebc588166 epochNumber=1
{"provingJobId":"1:BASE_PARITY:13a0709ebc588166"}
22:32:23 [22:32:23.730]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:13a0709ebc588166 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:13a0709ebc588166"}
22:32:23 [22:32:23.730]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:13a0709ebc588166 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:13a0709ebc588166"}
22:32:23 [22:32:23.741]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:2db83918248c3e74 epochNumber=2
{"provingJobId":"2:BASE_PARITY:2db83918248c3e74"}
22:32:23 [22:32:23.741]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:2db83918248c3e74 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:2db83918248c3e74"}
22:32:23 [22:32:23.741]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:2db83918248c3e74 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:2db83918248c3e74"}
22:32:23 [22:32:23.802]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:3bf4fee878d7452f epochNumber=3
{"provingJobId":"3:BASE_PARITY:3bf4fee878d7452f"}
22:32:23 [22:32:23.821]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:23 [22:32:23.864]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:be80fd9d05284391 epochNumber=4
{"provingJobId":"4:BASE_PARITY:be80fd9d05284391"}
22:32:23 [22:32:23.872]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:23 [22:32:23.925]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:58b3d3f4b5d9b518 epochNumber=5
{"provingJobId":"5:BASE_PARITY:58b3d3f4b5d9b518"}
22:32:23 [22:32:23.950]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:23 [22:32:23.982]
INFO:
prover-client:proving-broker Proving Broker started
22:32:23 [22:32:23.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1281f8613b8b4ad epochNumber=42
{"provingJobId":"1:BASE_PARITY:c1281f8613b8b4ad"}
22:32:23 [22:32:23.983]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testO8AKaC/42 with map size 1073741824
22:32:23 [22:32:23.987]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.012]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:afbc4bc18bcf2256 epochNumber=1
{"provingJobId":"1:BASE_PARITY:afbc4bc18bcf2256"}
22:32:24 [22:32:24.021]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f3fbfbc6dba531bd epochNumber=1
{"provingJobId":"1:BASE_PARITY:f3fbfbc6dba531bd"}
22:32:24 [22:32:24.024]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqAucba/1 with map size 1073741824
22:32:24 [22:32:24.030]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4be9225d29fd68d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4be9225d29fd68d3"}
22:32:24 [22:32:24.069]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.070]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd6ddb59bb5626b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd6ddb59bb5626b4"}
22:32:24 [22:32:24.072]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrj6HT6/1 with map size 1073741824
22:32:24 [22:32:24.075]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.099]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:bd6ddb59bb5626b4 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:bd6ddb59bb5626b4"}
22:32:24 [22:32:24.102]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.102]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce22294ceb622913 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce22294ceb622913"}
22:32:24 [22:32:24.103]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGLZllI/1 with map size 1073741824
22:32:24 [22:32:24.104]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.123]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ce22294ceb622913 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ce22294ceb622913"}
22:32:24 [22:32:24.124]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ce22294ceb622913 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ce22294ceb622913"}
22:32:24 [22:32:24.133]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ce22294ceb622913 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ce22294ceb622913"}
22:32:24 [22:32:24.138]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.138]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:33d0d7dde0a6710d epochNumber=1
{"provingJobId":"1:BASE_PARITY:33d0d7dde0a6710d"}
22:32:24 [22:32:24.139]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZUTPSG/1 with map size 1073741824
22:32:24 [22:32:24.146]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.169]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.172]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.173]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2aec9e12da197ef epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2aec9e12da197ef"}
22:32:24 [22:32:24.173]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwB2Hdo/1 with map size 1073741824
22:32:24 [22:32:24.176]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.193]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:f2aec9e12da197ef
{"provingJobId":"1:BASE_PARITY:f2aec9e12da197ef"}
22:32:24 [22:32:24.193]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f2aec9e12da197ef type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f2aec9e12da197ef"}
22:32:24 [22:32:24.194]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f2aec9e12da197ef type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:f2aec9e12da197ef"}
22:32:24 [22:32:24.204]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.204]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebf3eeb598967136 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ebf3eeb598967136"}
22:32:24 [22:32:24.205]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDojGuV/1 with map size 1073741824
22:32:24 [22:32:24.208]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.225]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ebf3eeb598967136
{"provingJobId":"1:BASE_PARITY:ebf3eeb598967136"}
22:32:24 [22:32:24.225]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ebf3eeb598967136 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:ebf3eeb598967136"}
22:32:24 [22:32:24.242]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.245]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:240879631d54f071 epochNumber=1
{"provingJobId":"1:BASE_PARITY:240879631d54f071"}
22:32:24 [22:32:24.248]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoYVYi0/1 with map size 1073741824
22:32:24 [22:32:24.256]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.274]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:240879631d54f071 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:240879631d54f071"}
22:32:24 [22:32:24.275]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:240879631d54f071 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:240879631d54f071"}
22:32:24 [22:32:24.284]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.284]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a3a0353461dbf2a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a3a0353461dbf2a0"}
22:32:24 [22:32:24.285]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUcfea4/1 with map size 1073741824
22:32:24 [22:32:24.287]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.305]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a3a0353461dbf2a0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a3a0353461dbf2a0"}
22:32:24 [22:32:24.306]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3a0353461dbf2a0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a3a0353461dbf2a0"}
22:32:24 [22:32:24.315]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.316]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c820242088ab3595 epochNumber=0
{"provingJobId":"1:BASE_PARITY:c820242088ab3595"}
22:32:24 [22:32:24.316]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testmtjef6/0 with map size 1073741824
22:32:24 [22:32:24.318]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.341]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c820242088ab3595 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c820242088ab3595"}
22:32:24 [22:32:24.342]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b5b8a8cf1dbb2a52 epochNumber=0
{"provingJobId":"1:BASE_PARITY:b5b8a8cf1dbb2a52"}
22:32:24 [22:32:24.343]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c820242088ab3595 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c820242088ab3595"}
22:32:24 [22:32:24.343]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1822ba01c310335f epochNumber=0
{"provingJobId":"1:BASE_PARITY:1822ba01c310335f"}
22:32:24 [22:32:24.362]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.364]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.365]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80885c03c747ceeb epochNumber=2
{"provingJobId":"1:BASE_PARITY:80885c03c747ceeb"}
22:32:24 [22:32:24.365]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testcO7ilg/2 with map size 1073741824
22:32:24 [22:32:24.367]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.382]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8f230f5c969a758a epochNumber=1
{"provingJobId":"1:BASE_PARITY:8f230f5c969a758a"}
22:32:24 [22:32:24.383]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcO7ilg/1 with map size 1073741824
22:32:24 [22:32:24.385]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.404]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.405]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4ee6f1f9a5b89ce4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4ee6f1f9a5b89ce4"}
22:32:24 [22:32:24.406]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdy5njF/1 with map size 1073741824
22:32:24 [22:32:24.407]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.428]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.430]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:70123c15713ccd2e epochNumber=1
{"provingJobId":"1:BASE_PARITY:70123c15713ccd2e"}
22:32:24 [22:32:24.431]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAmKkyK/1 with map size 1073741824
22:32:24 [22:32:24.435]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.458]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ecdbe34a8093bbd epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ecdbe34a8093bbd"}
22:32:24 [22:32:24.465]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb17658e89faf6f4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:cb17658e89faf6f4"}
22:32:24 [22:32:24.466]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAmKkyK/2 with map size 1073741824
22:32:24 [22:32:24.468]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.487]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dd2e05179756e919 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dd2e05179756e919"}
22:32:24 [22:32:24.496]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.497]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d31f1ca03e24fd2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3d31f1ca03e24fd2"}
22:32:24 [22:32:24.497]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testp6UEXX/1 with map size 1073741824
22:32:24 [22:32:24.499]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:283fd1542ab3d8cd epochNumber=1
{"provingJobId":"1:BASE_PARITY:283fd1542ab3d8cd"}
22:32:24 [22:32:24.521]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09da4eaed7eb2727 epochNumber=2
{"provingJobId":"1:BASE_PARITY:09da4eaed7eb2727"}
22:32:24 [22:32:24.522]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testp6UEXX/2 with map size 1073741824
22:32:24 [22:32:24.525]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.543]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9d2c5bee867e4e8e epochNumber=1
{"provingJobId":"1:BASE_PARITY:9d2c5bee867e4e8e"}
22:32:24 [22:32:24.558]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.559]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a0173cabc659fb1b epochNumber=1
{"provingJobId":"1:BASE_PARITY:a0173cabc659fb1b"}
22:32:24 [22:32:24.560]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqtFcwh/1 with map size 1073741824
22:32:24 [22:32:24.562]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.578]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c27b6be25f484978 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c27b6be25f484978"}
22:32:24 [22:32:24.583]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7cdfb00647ed6183 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7cdfb00647ed6183"}
22:32:24 [22:32:24.584]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqtFcwh/2 with map size 1073741824
22:32:24 [22:32:24.585]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.601]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:056db939bcd02edd epochNumber=1
{"provingJobId":"1:BASE_PARITY:056db939bcd02edd"}
22:32:24 [22:32:24.610]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.611]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:edd6b8d2896c5f58 epochNumber=1
{"provingJobId":"1:BASE_PARITY:edd6b8d2896c5f58"}
22:32:24 [22:32:24.611]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testatiz2J/1 with map size 1073741824
22:32:24 [22:32:24.613]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.634]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9963850620160cba epochNumber=1
{"provingJobId":"1:BASE_PARITY:9963850620160cba"}
22:32:24 [22:32:24.642]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:edd6b8d2896c5f58 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:edd6b8d2896c5f58"}
22:32:24 [22:32:24.655]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.656]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7f32d76534bc81d epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7f32d76534bc81d"}
22:32:24 [22:32:24.657]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw50g8e/1 with map size 1073741824
22:32:24 [22:32:24.659]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.677]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e1b790d3ac69c1c8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e1b790d3ac69c1c8"}
22:32:24 [22:32:24.682]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a7f32d76534bc81d type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:a7f32d76534bc81d"}
22:32:24 [22:32:24.692]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.693]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:448895ec0e70df99 epochNumber=1
{"provingJobId":"1:BASE_PARITY:448895ec0e70df99"}
22:32:24 [22:32:24.694]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbyxL8j/1 with map size 1073741824
22:32:24 [22:32:24.696]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.713]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:984410f40f4a6631 epochNumber=1
{"provingJobId":"1:BASE_PARITY:984410f40f4a6631"}
22:32:24 [22:32:24.719]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:448895ec0e70df99 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:448895ec0e70df99"}
22:32:24 [22:32:24.722]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:188c9b31353b0d24 epochNumber=1
{"provingJobId":"1:BASE_PARITY:188c9b31353b0d24"}
22:32:24 [22:32:24.724]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9wti4R/1 with map size 1073741824
22:32:24 [22:32:24.725]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.741]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:188c9b31353b0d24
{"provingJobId":"1:BASE_PARITY:188c9b31353b0d24"}
22:32:24 [22:32:24.742]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:188c9b31353b0d24 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:188c9b31353b0d24"}
22:32:24 [22:32:24.748]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac6501babcdb230a epochNumber=1
{"provingJobId":"1:BASE_PARITY:ac6501babcdb230a"}
22:32:24 [22:32:24.753]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:188c9b31353b0d24 has already been completed
{"provingJobId":"1:BASE_PARITY:188c9b31353b0d24"}
22:32:24 [22:32:24.758]
INFO:
prover-client:proving-broker Proving Broker started
22:32:24 [22:32:24.759]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09960e2089544c01 epochNumber=1
{"provingJobId":"1:BASE_PARITY:09960e2089544c01"}
22:32:24 [22:32:24.759]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqWxdXw/1 with map size 1073741824
22:32:24 [22:32:24.763]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:24 [22:32:24.781]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4b716d702ea53a9d epochNumber=2
{"provingJobId":"1:BASE_PARITY:4b716d702ea53a9d"}
22:32:24 [22:32:24.782]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqWxdXw/2 with map size 1073741824
22:32:24 [22:32:24.784]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:25 [22:32:25.854]
INFO:
prover-client:proving-broker Proving Broker started
22:32:25 [22:32:25.861]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:09960e2089544c01 settled=false
{"provingJobId":"1:BASE_PARITY:09960e2089544c01","status":"pending"}
22:32:25 [22:32:25.863]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4b716d702ea53a9d settled=false
{"provingJobId":"1:BASE_PARITY:4b716d702ea53a9d","status":"pending"}
22:32:25 [22:32:25.864]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:09960e2089544c01 type=BASE_PARITY startedAt=1750372344800 older agent has taken job
{"provingJobId":"1:BASE_PARITY:09960e2089544c01"}
22:32:25 [22:32:25.864]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:09960e2089544c01 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:09960e2089544c01"}
22:32:25 [22:32:25.868]
INFO:
prover-client:proving-broker Proving Broker started
22:32:25 [22:32:25.869]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:136b1e22f0750d77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:136b1e22f0750d77"}
22:32:25 [22:32:25.869]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvnjfgv/1 with map size 1073741824
22:32:25 [22:32:25.871]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:25 [22:32:25.890]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:687db2a56418ae4a epochNumber=2
{"provingJobId":"1:BASE_PARITY:687db2a56418ae4a"}
22:32:25 [22:32:25.891]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testvnjfgv/2 with map size 1073741824
22:32:25 [22:32:25.893]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:26 [22:32:26.910]
INFO:
prover-client:proving-broker Proving Broker started
22:32:26 [22:32:26.916]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:136b1e22f0750d77 settled=false
{"provingJobId":"1:BASE_PARITY:136b1e22f0750d77","status":"pending"}
22:32:26 [22:32:26.920]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:687db2a56418ae4a settled=false
{"provingJobId":"1:BASE_PARITY:687db2a56418ae4a","status":"pending"}
22:32:26 [22:32:26.921]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:136b1e22f0750d77 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:136b1e22f0750d77"}
22:32:26 [22:32:26.926]
INFO:
prover-client:proving-broker Proving Broker started
22:32:26 [22:32:26.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd789ef0cecb077e epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd789ef0cecb077e"}
22:32:26 [22:32:26.928]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJwsnYR/1 with map size 1073741824
22:32:26 [22:32:26.934]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:26 [22:32:26.957]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d5ccca4c2dbd07d4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d5ccca4c2dbd07d4"}
22:32:26 [22:32:26.958]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testJwsnYR/2 with map size 1073741824
22:32:26 [22:32:26.960]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:27 [22:32:27.981]
INFO:
prover-client:proving-broker Proving Broker started
22:32:27 [22:32:27.985]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bd789ef0cecb077e settled=false
{"provingJobId":"1:BASE_PARITY:bd789ef0cecb077e","status":"pending"}
22:32:27 [22:32:27.987]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d5ccca4c2dbd07d4 settled=false
{"provingJobId":"1:BASE_PARITY:d5ccca4c2dbd07d4","status":"pending"}
22:32:27 [22:32:27.989]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bd789ef0cecb077e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bd789ef0cecb077e"}
22:32:27 [22:32:27.989]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bd789ef0cecb077e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:bd789ef0cecb077e"}
22:32:28 [22:32:28.002]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.004]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c4b8d7e5967f3a5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6c4b8d7e5967f3a5"}
22:32:28 [22:32:28.005]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIg0JDy/1 with map size 1073741824
22:32:28 [22:32:28.007]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.025]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b6a4c1062e660c71 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b6a4c1062e660c71"}
22:32:28 [22:32:28.027]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testIg0JDy/2 with map size 1073741824
22:32:28 [22:32:28.030]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.048]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6c4b8d7e5967f3a5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6c4b8d7e5967f3a5"}
22:32:28 [22:32:28.054]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b6a4c1062e660c71 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:b6a4c1062e660c71"}
22:32:28 [22:32:28.065]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.067]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ff569eb82a9c6eb epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ff569eb82a9c6eb"}
22:32:28 [22:32:28.067]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEzXCkL/1 with map size 1073741824
22:32:28 [22:32:28.070]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.087]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c9b2ca948665cfa2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c9b2ca948665cfa2"}
22:32:28 [22:32:28.090]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testEzXCkL/2 with map size 1073741824
22:32:28 [22:32:28.092]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.110]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2ff569eb82a9c6eb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2ff569eb82a9c6eb"}
22:32:28 [22:32:28.111]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2ff569eb82a9c6eb type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2ff569eb82a9c6eb"}
22:32:28 [22:32:28.118]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c9b2ca948665cfa2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c9b2ca948665cfa2"}
22:32:28 [22:32:28.129]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.130]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c15d9785434850db err=test error
{"provingJoId":"1:BASE_PARITY:c15d9785434850db"}
22:32:28 [22:32:28.133]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.134]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dab025fb2e7ead66 not found
{"provingJobId":"1:BASE_PARITY:dab025fb2e7ead66"}
22:32:28 [22:32:28.136]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.137]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6849e5f5a70ae980 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6849e5f5a70ae980"}
22:32:28 [22:32:28.139]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9iba0w/1 with map size 1073741824
22:32:28 [22:32:28.141]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.162]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.162]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:20d5c37e96916609 epochNumber=1
{"provingJobId":"1:BASE_PARITY:20d5c37e96916609"}
22:32:28 [22:32:28.163]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5jjtkJ/1 with map size 1073741824
22:32:28 [22:32:28.165]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.290]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:20d5c37e96916609 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:20d5c37e96916609"}
22:32:28 [22:32:28.297]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.298]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f6557364be682797 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f6557364be682797"}
22:32:28 [22:32:28.301]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4Xuxp5/1 with map size 1073741824
22:32:28 [22:32:28.304]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.452]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f6557364be682797 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:f6557364be682797"}
22:32:28 [22:32:28.454]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a24b039a1ac4f75b epochNumber=10
{"provingJobId":"1:BASE_PARITY:a24b039a1ac4f75b"}
22:32:28 [22:32:28.455]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test4Xuxp5/10 with map size 1073741824
22:32:28 [22:32:28.457]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.479]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:28 [22:32:28.479]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
22:32:28 [22:32:28.603]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.604]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b46beae0f6c8946 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b46beae0f6c8946"}
22:32:28 [22:32:28.604]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testysQvcW/1 with map size 1073741824
22:32:28 [22:32:28.607]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.780]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b46beae0f6c8946 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2b46beae0f6c8946"}
22:32:28 [22:32:28.832]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.833]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cead8e0200adbc51 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cead8e0200adbc51"}
22:32:28 [22:32:28.835]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpXkbtb/1 with map size 1073741824
22:32:28 [22:32:28.839]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.857]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:cead8e0200adbc51 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:cead8e0200adbc51"}
22:32:28 [22:32:28.863]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.864]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b034823da438c0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b034823da438c0f"}
22:32:28 [22:32:28.865]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPDdAeJ/1 with map size 1073741824
22:32:28 [22:32:28.866]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.883]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2b034823da438c0f type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:2b034823da438c0f"}
22:32:28 [22:32:28.884]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2b034823da438c0f type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:2b034823da438c0f"}
22:32:28 [22:32:28.892]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.892]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b191fd6f4501cb1c epochNumber=1
{"provingJobId":"1:BASE_PARITY:b191fd6f4501cb1c"}
22:32:28 [22:32:28.893]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcA6m37/1 with map size 1073741824
22:32:28 [22:32:28.895]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.913]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b191fd6f4501cb1c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:b191fd6f4501cb1c"}
22:32:28 [22:32:28.921]
INFO:
prover-client:proving-broker Proving Broker started
22:32:28 [22:32:28.922]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ee08bb66557f079 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7ee08bb66557f079"}
22:32:28 [22:32:28.923]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEaxuAO/1 with map size 1073741824
22:32:28 [22:32:28.924]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.941]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7ee08bb66557f079 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:7ee08bb66557f079"}
22:32:28 [22:32:28.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1713341608fd0101 epochNumber=3
{"provingJobId":"1:BASE_PARITY:1713341608fd0101"}
22:32:28 [22:32:28.942]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testEaxuAO/3 with map size 1073741824
22:32:28 [22:32:28.944]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:28 [22:32:28.972]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:28 [22:32:28.973]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
22:32:28 [22:32:28.984]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7ee08bb66557f079 err=test error
{"provingJoId":"1:BASE_PARITY:7ee08bb66557f079"}
22:32:28 [22:32:28.993]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAmwpt6/1 with map size 1073741824
22:32:28 [22:32:28.995]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.012]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAmwpt6/2 with map size 1073741824
22:32:29 [22:32:29.014]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.035]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.037]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d1fb39341ad0af00 settled=false
{"provingJobId":"1:BASE_PARITY:d1fb39341ad0af00","status":"pending"}
22:32:29 [22:32:29.037]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:27dfe857bcde7c4c settled=false
{"provingJobId":"1:BASE_PARITY:27dfe857bcde7c4c","status":"pending"}
22:32:29 [22:32:29.043]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg0aIAb/1 with map size 1073741824
22:32:29 [22:32:29.045]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.065]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testg0aIAb/2 with map size 1073741824
22:32:29 [22:32:29.068]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.098]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.100]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:12da6e0e3df78255 settled=true
{"provingJobId":"1:BASE_PARITY:12da6e0e3df78255","status":"fulfilled"}
22:32:29 [22:32:29.102]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:31efbf6b9fba569a settled=true
{"provingJobId":"2:BASE_PARITY:31efbf6b9fba569a","status":"fulfilled"}
22:32:29 [22:32:29.108]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrxXWU7/1 with map size 1073741824
22:32:29 [22:32:29.110]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.138]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrxXWU7/2 with map size 1073741824
22:32:29 [22:32:29.140]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.162]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.164]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:754d1a8d4ac04da3 settled=true
{"provingJobId":"1:BASE_PARITY:754d1a8d4ac04da3","status":"fulfilled"}
22:32:29 [22:32:29.165]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a4da51a88eddd8f9 settled=false
{"provingJobId":"1:BASE_PARITY:a4da51a88eddd8f9","status":"pending"}
22:32:29 [22:32:29.171]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.172]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60a99f525ba32fee epochNumber=1
{"provingJobId":"1:BASE_PARITY:60a99f525ba32fee"}
22:32:29 [22:32:29.172]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDYu98u/1 with map size 1073741824
22:32:29 [22:32:29.175]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.194]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.195]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80a55d4ef34b9270 epochNumber=1
{"provingJobId":"1:BASE_PARITY:80a55d4ef34b9270"}
22:32:29 [22:32:29.195]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:80a55d4ef34b9270: Error: db error: Error: db error
22:32:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:80a55d4ef34b9270"}
22:32:29 [22:32:29.201]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.202]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54aa987dadfe2c0a epochNumber=1
{"provingJobId":"1:BASE_PARITY:54aa987dadfe2c0a"}
22:32:29 [22:32:29.202]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3Q35Bu/1 with map size 1073741824
22:32:29 [22:32:29.204]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.220]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:54aa987dadfe2c0a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:54aa987dadfe2c0a"}
22:32:29 [22:32:29.220]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:54aa987dadfe2c0a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:54aa987dadfe2c0a"}
22:32:29 [22:32:29.230]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.231]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:478dadc50591320b epochNumber=1
{"provingJobId":"1:BASE_PARITY:478dadc50591320b"}
22:32:29 [22:32:29.233]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJUYYBI/1 with map size 1073741824
22:32:29 [22:32:29.235]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.250]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:478dadc50591320b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:478dadc50591320b"}
22:32:29 [22:32:29.251]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:478dadc50591320b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:478dadc50591320b"}
22:32:29 [22:32:29.252]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:478dadc50591320b: Error: db error
22:32:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:478dadc50591320b"}
22:32:29 [22:32:29.255]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.256]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f74cf21f51e2dd02 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f74cf21f51e2dd02"}
22:32:29 [22:32:29.257]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test72KQYd/1 with map size 1073741824
22:32:29 [22:32:29.259]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.275]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f74cf21f51e2dd02 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f74cf21f51e2dd02"}
22:32:29 [22:32:29.275]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f74cf21f51e2dd02 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f74cf21f51e2dd02"}
22:32:29 [22:32:29.284]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.285]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:da2ce6fc98b37b0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:da2ce6fc98b37b0f"}
22:32:29 [22:32:29.287]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGE4rh9/1 with map size 1073741824
22:32:29 [22:32:29.289]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.307]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:da2ce6fc98b37b0f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:da2ce6fc98b37b0f"}
22:32:29 [22:32:29.307]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:da2ce6fc98b37b0f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:da2ce6fc98b37b0f"}
22:32:29 [22:32:29.308]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:da2ce6fc98b37b0f jobErr=test error: Error: db error
22:32:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:da2ce6fc98b37b0f"}
22:32:29 [22:32:29.314]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.315]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ea4bbc256f29892a not found
{"provingJobId":"1:BASE_PARITY:ea4bbc256f29892a"}
22:32:29 [22:32:29.319]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.319]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:4db16d018e158e51 err=test error
{"provingJoId":"1:BASE_PARITY:4db16d018e158e51"}
22:32:29 [22:32:29.324]
INFO:
prover-client:proving-broker Proving Broker started
22:32:29 [22:32:29.335]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6749d08bfa2ef3b epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6749d08bfa2ef3b"}
22:32:29 [22:32:29.336]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZK6iSl/1 with map size 1073741824
22:32:29 [22:32:29.338]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.356]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e6749d08bfa2ef3b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e6749d08bfa2ef3b"}
22:32:29 [22:32:29.357]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e6749d08bfa2ef3b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e6749d08bfa2ef3b"}
22:32:29 [22:32:29.376]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:2b9da5b56c7cf9d8 epochNumber=2
{"provingJobId":"2:BASE_PARITY:2b9da5b56c7cf9d8"}
22:32:29 [22:32:29.377]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testZK6iSl/2 with map size 1073741824
22:32:29 [22:32:29.380]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.400]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:2b9da5b56c7cf9d8 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:2b9da5b56c7cf9d8"}
22:32:29 [22:32:29.401]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:2b9da5b56c7cf9d8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:2b9da5b56c7cf9d8"}
22:32:29 [22:32:29.468]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:09401558a4b7f12b epochNumber=3
{"provingJobId":"3:BASE_PARITY:09401558a4b7f12b"}
22:32:29 [22:32:29.470]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testZK6iSl/3 with map size 1073741824
22:32:29 [22:32:29.473]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.501]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:29 [22:32:29.502]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
22:32:29 [22:32:29.557]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:8c1196453a73c287 epochNumber=4
{"provingJobId":"4:BASE_PARITY:8c1196453a73c287"}
22:32:29 [22:32:29.558]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testZK6iSl/4 with map size 1073741824
22:32:29 [22:32:29.560]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.607]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:29 [22:32:29.608]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
22:32:29 [22:32:29.645]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:be3ba8b87989dc8d epochNumber=5
{"provingJobId":"5:BASE_PARITY:be3ba8b87989dc8d"}
22:32:29 [22:32:29.646]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testZK6iSl/5 with map size 1073741824
22:32:29 [22:32:29.648]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
22:32:29 [22:32:29.687]
INFO:
prover-client:proving-broker Cleaned up jobs=1
22:32:29 [22:32:29.688]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
22:32:29
PASS src/proving_broker/proving_broker.test.ts (
13.041 s)
22:32:29 ProvingBroker
22:32:29 Producer API
22:32:29
✓ refuses stale jobs (47 ms)
22:32:29
✓ enqueues jobs (3 ms)
22:32:29
✓ ignores duplicate jobs (2 ms)
22:32:29
✓ reports correct status when enqueuing repeat jobs (3 ms)
22:32:29
✓ throws an error in case of duplicate job IDs (19 ms)
22:32:29
✓ returns not-found status for non-existing jobs (1 ms)
22:32:29
✓ cancels jobs in queue (3 ms)
22:32:29
✓ cancels jobs in-progress (2 ms)
22:32:29
✓ returns job result if successful (2 ms)
22:32:29
✓ returns job error if failed (2 ms)
22:32:29
✓ correctly returns job status for concurrent writes (2 ms)
22:32:29
✓ refuses stale jobs (38 ms)
22:32:29
✓ enqueues jobs (50 ms)
22:32:29
✓ ignores duplicate jobs (34 ms)
22:32:29
✓ reports correct status when enqueuing repeat jobs (34 ms)
22:32:29
✓ throws an error in case of duplicate job IDs (31 ms)
22:32:29
✓ returns not-found status for non-existing jobs (5 ms)
22:32:29
✓ cancels jobs in queue (30 ms)
22:32:29
✓ cancels jobs in-progress (33 ms)
22:32:29
✓ returns job result if successful (46 ms)
22:32:29
✓ returns job error if failed (31 ms)
22:32:29
✓ correctly returns job status for concurrent writes (47 ms)
22:32:29 Consumer API
22:32:29
✓ returns undefined if no jobs are available (8 ms)
22:32:29
✓ returns jobs in priority order (2 ms)
22:32:29
✓ returns undefined if no jobs are available for the given allowList (1 ms)
22:32:29
✓ returns a job if it is in the allowList (2 ms)
22:32:29
✓ returns the most important job if it is in the allowList (2 ms)
22:32:29
✓ returns any job if filter is empty (2 ms)
22:32:29
✓ returns a new job when reporting job success (2 ms)
22:32:29
✓ returns a new job when reporting permanent error (2 ms)
22:32:29
✓ returns a new job when reporting retry-able error (2 ms)
22:32:29
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
22:32:29
✓ returns a new job if job is already in progress elsewhere (1058 ms)
22:32:29
✓ avoids sending the same job to a new agent after a restart (1004 ms)
22:32:29
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
22:32:29
✓ tracks job result if in progress (2 ms)
22:32:29
✓ tracks job result even if job is in queue (1 ms)
22:32:29
✓ ignores reported job error if unknown job (1 ms)
22:32:29
✓ ignores job result if unknown job (1 ms)
22:32:29
✓ returns undefined if no jobs are available (2 ms)
22:32:29
✓ returns jobs in priority order (40 ms)
22:32:29
✓ returns undefined if no jobs are available for the given allowList (23 ms)
22:32:29
✓ returns a job if it is in the allowList (69 ms)
22:32:29
✓ returns the most important job if it is in the allowList (61 ms)
22:32:29
✓ returns any job if filter is empty (52 ms)
22:32:29
✓ returns a new job when reporting job success (43 ms)
22:32:29
✓ returns a new job when reporting permanent error (39 ms)
22:32:29
✓ returns a new job when reporting retry-able error (31 ms)
22:32:29
✓ returns a new job when reporting progress if current one is cancelled (34 ms)
22:32:29
✓ returns a new job if job is already in progress elsewhere (1111 ms)
22:32:29
✓ avoids sending the same job to a new agent after a restart (1058 ms)
22:32:29
✓ avoids sending a completed job to a new agent after a restart (1075 ms)
22:32:29
✓ tracks job result if in progress (62 ms)
22:32:29
✓ tracks job result even if job is in queue (64 ms)
22:32:29
✓ ignores reported job error if unknown job (4 ms)
22:32:29
✓ ignores job result if unknown job (2 ms)
22:32:29 Timeouts
22:32:29
✓ tracks in progress jobs (2 ms)
22:32:29
✓ re-enqueues jobs that time out (107 ms)
22:32:29
✓ cancel stale jobs that time out (235 ms)
22:32:29
✓ keeps the jobs in progress while it is alive (202 ms)
22:32:29
✓ tracks in progress jobs (24 ms)
22:32:29
✓ re-enqueues jobs that time out (136 ms)
22:32:29
✓ cancel stale jobs that time out (304 ms)
22:32:29
✓ keeps the jobs in progress while it is alive (226 ms)
22:32:29 Retries
22:32:29
✓ retries jobs (2 ms)
22:32:29
✓ retries up to a maximum number of times (2 ms)
22:32:29
✓ passing retry=false does not retry (2 ms)
22:32:29
✓ does not retry if job is stale (27 ms)
22:32:29
✓ retries jobs (33 ms)
22:32:29
✓ retries up to a maximum number of times (28 ms)
22:32:29
✓ passing retry=false does not retry (30 ms)
22:32:29
✓ does not retry if job is stale (66 ms)
22:32:29 Database management
22:32:29
✓ re-enqueues proof requests on start (4 ms)
22:32:29
✓ restores proof results on start (2 ms)
22:32:29
✓ only re-enqueues unfinished jobs (2 ms)
22:32:29
✓ saves job when enqueued (2 ms)
22:32:29
✓ does not retain job if database fails to save (3 ms)
22:32:29
✓ saves job result (2 ms)
22:32:29
✓ saves result even if database fails to save (3 ms)
22:32:29
✓ saves job error (2 ms)
22:32:29
✓ saves job error even if database fails to save (3 ms)
22:32:29
✓ does not save job result if job is unknown (2 ms)
22:32:29
✓ does not save job error if job is unknown (2 ms)
22:32:29
✓ cleans up old jobs periodically (258 ms)
22:32:29
✓ re-enqueues proof requests on start (53 ms)
22:32:29
✓ restores proof results on start (64 ms)
22:32:29
✓ only re-enqueues unfinished jobs (63 ms)
22:32:29
✓ saves job when enqueued (23 ms)
22:32:29
✓ does not retain job if database fails to save (5 ms)
22:32:29
✓ saves job result (30 ms)
22:32:29
✓ saves result even if database fails to save (26 ms)
22:32:29
✓ saves job error (28 ms)
22:32:29
✓ saves job error even if database fails to save (27 ms)
22:32:29
✓ does not save job result if job is unknown (5 ms)
22:32:29
✓ does not save job error if job is unknown (4 ms)
22:32:29
✓ cleans up old jobs periodically (399 ms)
22:32:29
22:32:29
Test Suites: 1 passed, 1 total
22:32:29
Tests: 96 passed, 96 total
22:32:29
Snapshots: 0 total
22:32:29
Time: 13.141 s
22:32:29
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
22:32:29
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?