Parent Log:
http://ci.aztec-labs.com/195ccbc9422e4e2c
Command: 8e1bd0706feebe1f yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/227fd751f8d3e07a96e4c1564665f0676f8222af
Env: REF_NAME=gh-readonly-queue/next/pr-15209-c245b3e704a64008c8dea40ee7b26418fd59554d CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 23 11:49:30 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15209_amd64_x4-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
11:49:34 [11:49:34.848]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.855]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:34062e9c22ee5c06 epochNumber=42
{"provingJobId":"1:BASE_PARITY:34062e9c22ee5c06"}
11:49:34 [11:49:34.858]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:f8db741a90db78d0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f8db741a90db78d0"}
11:49:34 [11:49:34.882]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.883]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:651e11a6987707fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:651e11a6987707fd"}
11:49:34 [11:49:34.884]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0cf40f8df730824 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0cf40f8df730824"}
11:49:34 [11:49:34.885]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.886]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c85b1cf07e27ec12 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c85b1cf07e27ec12"}
11:49:34 [11:49:34.887]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c85b1cf07e27ec12 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c85b1cf07e27ec12"}
11:49:34 [11:49:34.888]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.889]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0ec810840a7a07a epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0ec810840a7a07a"}
11:49:34 [11:49:34.890]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d0ec810840a7a07a epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d0ec810840a7a07a"}
11:49:34 [11:49:34.890]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d0ec810840a7a07a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d0ec810840a7a07a"}
11:49:34 [11:49:34.891]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d0ec810840a7a07a epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d0ec810840a7a07a"}
11:49:34 [11:49:34.892]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.893]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81968e4e46a1e816 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81968e4e46a1e816"}
11:49:34 [11:49:34.923]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.925]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a2861168066547f epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a2861168066547f"}
11:49:34 [11:49:34.927]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7a2861168066547f
{"provingJobId":"1:BASE_PARITY:7a2861168066547f"}
11:49:34 [11:49:34.927]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7a2861168066547f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7a2861168066547f"}
11:49:34 [11:49:34.927]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a2861168066547f type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:7a2861168066547f"}
11:49:34 [11:49:34.929]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.930]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7dfdea4d464343e4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7dfdea4d464343e4"}
11:49:34 [11:49:34.930]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7dfdea4d464343e4
{"provingJobId":"1:BASE_PARITY:7dfdea4d464343e4"}
11:49:34 [11:49:34.931]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7dfdea4d464343e4 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:7dfdea4d464343e4"}
11:49:34 [11:49:34.932]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.933]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4943c2ddacd9f05a epochNumber=1
{"provingJobId":"1:BASE_PARITY:4943c2ddacd9f05a"}
11:49:34 [11:49:34.933]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4943c2ddacd9f05a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4943c2ddacd9f05a"}
11:49:34 [11:49:34.933]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4943c2ddacd9f05a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4943c2ddacd9f05a"}
11:49:34 [11:49:34.935]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2c9906b9f2c2cd3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2c9906b9f2c2cd3"}
11:49:34 [11:49:34.936]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a2c9906b9f2c2cd3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a2c9906b9f2c2cd3"}
11:49:34 [11:49:34.936]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a2c9906b9f2c2cd3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a2c9906b9f2c2cd3"}
11:49:34 [11:49:34.937]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:85e79b36fea8c567 epochNumber=0
{"provingJobId":"1:BASE_PARITY:85e79b36fea8c567"}
11:49:34 [11:49:34.938]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:85e79b36fea8c567 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:85e79b36fea8c567"}
11:49:34 [11:49:34.938]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3de2c32b843abe2 epochNumber=0
{"provingJobId":"1:BASE_PARITY:d3de2c32b843abe2"}
11:49:34 [11:49:34.938]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:85e79b36fea8c567 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:85e79b36fea8c567"}
11:49:34 [11:49:34.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5f7409dd6b5df97f epochNumber=0
{"provingJobId":"1:BASE_PARITY:5f7409dd6b5df97f"}
11:49:34 [11:49:34.940]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.941]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.941]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7b7bfe04d3d70d8 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a7b7bfe04d3d70d8"}
11:49:34 [11:49:34.941]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:20eb3610866a3dfc epochNumber=1
{"provingJobId":"1:BASE_PARITY:20eb3610866a3dfc"}
11:49:34 [11:49:34.942]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.943]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:111fb9b9b5f3c77e epochNumber=1
{"provingJobId":"1:BASE_PARITY:111fb9b9b5f3c77e"}
11:49:34 [11:49:34.944]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.952]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ef88f0512dae6535 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ef88f0512dae6535"}
11:49:34 [11:49:34.952]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a9045b1cc48aa814 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a9045b1cc48aa814"}
11:49:34 [11:49:34.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bae26d28705372af epochNumber=2
{"provingJobId":"1:BASE_PARITY:bae26d28705372af"}
11:49:34 [11:49:34.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c6ad2f2b7ded760f epochNumber=1
{"provingJobId":"1:BASE_PARITY:c6ad2f2b7ded760f"}
11:49:34 [11:49:34.955]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.955]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:94ccf4f5119126e9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:94ccf4f5119126e9"}
11:49:34 [11:49:34.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0588da7b363cef32 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0588da7b363cef32"}
11:49:34 [11:49:34.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c9ec919c783bd3b3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c9ec919c783bd3b3"}
11:49:34 [11:49:34.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:152747bb9ec3c9e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:152747bb9ec3c9e3"}
11:49:34 [11:49:34.957]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.958]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a77424757802479 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a77424757802479"}
11:49:34 [11:49:34.958]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62046d1599e9468d epochNumber=1
{"provingJobId":"1:BASE_PARITY:62046d1599e9468d"}
11:49:34 [11:49:34.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71dfab537f5604fb epochNumber=2
{"provingJobId":"1:BASE_PARITY:71dfab537f5604fb"}
11:49:34 [11:49:34.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e69dff6cb36d68d4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e69dff6cb36d68d4"}
11:49:34 [11:49:34.960]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f864e5e0258a4807 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f864e5e0258a4807"}
11:49:34 [11:49:34.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eabf076125bce034 epochNumber=1
{"provingJobId":"1:BASE_PARITY:eabf076125bce034"}
11:49:34 [11:49:34.962]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f864e5e0258a4807 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f864e5e0258a4807"}
11:49:34 [11:49:34.963]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.963]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e5652a389f87f5ed epochNumber=1
{"provingJobId":"1:BASE_PARITY:e5652a389f87f5ed"}
11:49:34 [11:49:34.964]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:380ba8bc558cbd09 epochNumber=1
{"provingJobId":"1:BASE_PARITY:380ba8bc558cbd09"}
11:49:34 [11:49:34.964]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e5652a389f87f5ed type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:e5652a389f87f5ed"}
11:49:34 [11:49:34.965]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:10366fdc459d2586 epochNumber=1
{"provingJobId":"1:BASE_PARITY:10366fdc459d2586"}
11:49:34 [11:49:34.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a748993ea28edbea epochNumber=1
{"provingJobId":"1:BASE_PARITY:a748993ea28edbea"}
11:49:34 [11:49:34.967]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:10366fdc459d2586 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:10366fdc459d2586"}
11:49:34 [11:49:34.968]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.968]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdba38df92671980 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdba38df92671980"}
11:49:34 [11:49:34.969]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:fdba38df92671980
{"provingJobId":"1:BASE_PARITY:fdba38df92671980"}
11:49:34 [11:49:34.969]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fdba38df92671980 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:fdba38df92671980"}
11:49:34 [11:49:34.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f91bbc6aa572e94 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f91bbc6aa572e94"}
11:49:34 [11:49:34.970]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fdba38df92671980 has already been completed
{"provingJobId":"1:BASE_PARITY:fdba38df92671980"}
11:49:34 [11:49:34.971]
INFO:
prover-client:proving-broker Proving Broker started
11:49:34 [11:49:34.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68f7826e39182f04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68f7826e39182f04"}
11:49:34 [11:49:34.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81e2274ae0e1f1ea epochNumber=2
{"provingJobId":"1:BASE_PARITY:81e2274ae0e1f1ea"}
11:49:36 [11:49:36.025]
INFO:
prover-client:proving-broker Proving Broker started
11:49:36 [11:49:36.026]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:68f7826e39182f04 settled=false
{"provingJobId":"1:BASE_PARITY:68f7826e39182f04","status":"pending"}
11:49:36 [11:49:36.027]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:81e2274ae0e1f1ea settled=false
{"provingJobId":"1:BASE_PARITY:81e2274ae0e1f1ea","status":"pending"}
11:49:36 [11:49:36.028]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:68f7826e39182f04 type=BASE_PARITY startedAt=1750679374972 older agent has taken job
{"provingJobId":"1:BASE_PARITY:68f7826e39182f04"}
11:49:36 [11:49:36.028]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:68f7826e39182f04 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:68f7826e39182f04"}
11:49:36 [11:49:36.030]
INFO:
prover-client:proving-broker Proving Broker started
11:49:36 [11:49:36.030]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b99d10cdd8e1c172 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b99d10cdd8e1c172"}
11:49:36 [11:49:36.031]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d764ce529e0be52c epochNumber=2
{"provingJobId":"1:BASE_PARITY:d764ce529e0be52c"}
11:49:37 [11:49:37.032]
INFO:
prover-client:proving-broker Proving Broker started
11:49:37 [11:49:37.033]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b99d10cdd8e1c172 settled=false
{"provingJobId":"1:BASE_PARITY:b99d10cdd8e1c172","status":"pending"}
11:49:37 [11:49:37.033]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d764ce529e0be52c settled=false
{"provingJobId":"1:BASE_PARITY:d764ce529e0be52c","status":"pending"}
11:49:37 [11:49:37.034]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b99d10cdd8e1c172 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:b99d10cdd8e1c172"}
11:49:37 [11:49:37.035]
INFO:
prover-client:proving-broker Proving Broker started
11:49:37 [11:49:37.035]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:134d0706d925df22 epochNumber=1
{"provingJobId":"1:BASE_PARITY:134d0706d925df22"}
11:49:37 [11:49:37.036]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:afb2b94a2725cee9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:afb2b94a2725cee9"}
11:49:38 [11:49:38.037]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.038]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:134d0706d925df22 settled=false
{"provingJobId":"1:BASE_PARITY:134d0706d925df22","status":"pending"}
11:49:38 [11:49:38.038]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:afb2b94a2725cee9 settled=false
{"provingJobId":"1:BASE_PARITY:afb2b94a2725cee9","status":"pending"}
11:49:38 [11:49:38.040]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:134d0706d925df22 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:134d0706d925df22"}
11:49:38 [11:49:38.040]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:134d0706d925df22 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:134d0706d925df22"}
11:49:38 [11:49:38.044]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.044]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ddceb9fc32a155fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:ddceb9fc32a155fa"}
11:49:38 [11:49:38.045]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a005eaed8a0fc42f epochNumber=2
{"provingJobId":"1:BASE_PARITY:a005eaed8a0fc42f"}
11:49:38 [11:49:38.045]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ddceb9fc32a155fa type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ddceb9fc32a155fa"}
11:49:38 [11:49:38.046]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a005eaed8a0fc42f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a005eaed8a0fc42f"}
11:49:38 [11:49:38.047]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.048]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81923d91cbf0e09a epochNumber=1
{"provingJobId":"1:BASE_PARITY:81923d91cbf0e09a"}
11:49:38 [11:49:38.048]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4595d4d848ad57ae epochNumber=2
{"provingJobId":"1:BASE_PARITY:4595d4d848ad57ae"}
11:49:38 [11:49:38.049]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:81923d91cbf0e09a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:81923d91cbf0e09a"}
11:49:38 [11:49:38.049]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:81923d91cbf0e09a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:81923d91cbf0e09a"}
11:49:38 [11:49:38.049]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4595d4d848ad57ae type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4595d4d848ad57ae"}
11:49:38 [11:49:38.051]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.051]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e8112d4df5039708 err=test error
{"provingJoId":"1:BASE_PARITY:e8112d4df5039708"}
11:49:38 [11:49:38.052]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.053]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:698e20184012bba3 not found
{"provingJobId":"1:BASE_PARITY:698e20184012bba3"}
11:49:38 [11:49:38.054]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b6823992c31dc69b epochNumber=1
{"provingJobId":"1:BASE_PARITY:b6823992c31dc69b"}
11:49:38 [11:49:38.056]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd4553563c73081d epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd4553563c73081d"}
11:49:38 [11:49:38.158]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fd4553563c73081d timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:fd4553563c73081d"}
11:49:38 [11:49:38.165]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.166]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8561e7145d5da2fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:8561e7145d5da2fc"}
11:49:38 [11:49:38.268]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8561e7145d5da2fc timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:8561e7145d5da2fc"}
11:49:38 [11:49:38.274]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9805513a40685e4d epochNumber=10
{"provingJobId":"1:BASE_PARITY:9805513a40685e4d"}
11:49:38 [11:49:38.294]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:38 [11:49:38.401]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.403]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5365dd455eda0ed5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5365dd455eda0ed5"}
11:49:38 [11:49:38.555]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5365dd455eda0ed5 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:5365dd455eda0ed5"}
11:49:38 [11:49:38.607]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.608]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc6aa879f416bf0b epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc6aa879f416bf0b"}
11:49:38 [11:49:38.608]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fc6aa879f416bf0b type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:fc6aa879f416bf0b"}
11:49:38 [11:49:38.609]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.610]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a5d5e788b9d0a9f0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a5d5e788b9d0a9f0"}
11:49:38 [11:49:38.610]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a5d5e788b9d0a9f0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a5d5e788b9d0a9f0"}
11:49:38 [11:49:38.611]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a5d5e788b9d0a9f0 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:a5d5e788b9d0a9f0"}
11:49:38 [11:49:38.612]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.613]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f722e9e470a57e1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f722e9e470a57e1"}
11:49:38 [11:49:38.613]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0f722e9e470a57e1 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0f722e9e470a57e1"}
11:49:38 [11:49:38.614]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.615]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aeae59ae5be3f333 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aeae59ae5be3f333"}
11:49:38 [11:49:38.615]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:aeae59ae5be3f333 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:aeae59ae5be3f333"}
11:49:38 [11:49:38.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2f233a259266c7e epochNumber=3
{"provingJobId":"1:BASE_PARITY:a2f233a259266c7e"}
11:49:38 [11:49:38.640]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:38 [11:49:38.641]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:aeae59ae5be3f333 err=test error
{"provingJoId":"1:BASE_PARITY:aeae59ae5be3f333"}
11:49:38 [11:49:38.643]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.644]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1be22ceb55e4601c settled=false
{"provingJobId":"1:BASE_PARITY:1be22ceb55e4601c","status":"pending"}
11:49:38 [11:49:38.644]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a7e06f25dbae62c4 settled=false
{"provingJobId":"1:BASE_PARITY:a7e06f25dbae62c4","status":"pending"}
11:49:38 [11:49:38.646]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.647]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7269936ff0bcec3a settled=true
{"provingJobId":"1:BASE_PARITY:7269936ff0bcec3a","status":"fulfilled"}
11:49:38 [11:49:38.647]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8891bb21d4246c25 settled=true
{"provingJobId":"2:BASE_PARITY:8891bb21d4246c25","status":"fulfilled"}
11:49:38 [11:49:38.648]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.648]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f31f85a2d0e532d5 settled=true
{"provingJobId":"1:BASE_PARITY:f31f85a2d0e532d5","status":"fulfilled"}
11:49:38 [11:49:38.649]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c1cfc8bb25d4f6ee settled=false
{"provingJobId":"1:BASE_PARITY:c1cfc8bb25d4f6ee","status":"pending"}
11:49:38 [11:49:38.650]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.652]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2322f756ec22198 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2322f756ec22198"}
11:49:38 [11:49:38.653]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.654]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:90e1a23d66570348 epochNumber=1
{"provingJobId":"1:BASE_PARITY:90e1a23d66570348"}
11:49:38 [11:49:38.655]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:90e1a23d66570348: Error: db error: Error: db error
11:49:38 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:90e1a23d66570348"}
11:49:38 [11:49:38.657]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.658]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f6a693a7a31ce494 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f6a693a7a31ce494"}
11:49:38 [11:49:38.658]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f6a693a7a31ce494 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f6a693a7a31ce494"}
11:49:38 [11:49:38.659]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f6a693a7a31ce494 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f6a693a7a31ce494"}
11:49:38 [11:49:38.661]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.661]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc968a42291d97c1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc968a42291d97c1"}
11:49:38 [11:49:38.662]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fc968a42291d97c1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fc968a42291d97c1"}
11:49:38 [11:49:38.662]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fc968a42291d97c1 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fc968a42291d97c1"}
11:49:38 [11:49:38.663]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:fc968a42291d97c1: Error: db error
11:49:38 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:fc968a42291d97c1"}
11:49:38 [11:49:38.665]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.665]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:796034bf375de339 epochNumber=1
{"provingJobId":"1:BASE_PARITY:796034bf375de339"}
11:49:38 [11:49:38.666]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:796034bf375de339 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:796034bf375de339"}
11:49:38 [11:49:38.666]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:796034bf375de339 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:796034bf375de339"}
11:49:38 [11:49:38.667]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.668]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c26777ed2bbf3045 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c26777ed2bbf3045"}
11:49:38 [11:49:38.668]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c26777ed2bbf3045 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c26777ed2bbf3045"}
11:49:38 [11:49:38.668]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c26777ed2bbf3045 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c26777ed2bbf3045"}
11:49:38 [11:49:38.668]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:c26777ed2bbf3045 jobErr=test error: Error: db error
11:49:38 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:c26777ed2bbf3045"}
11:49:38 [11:49:38.669]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.670]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f4e62ac0e570b83e not found
{"provingJobId":"1:BASE_PARITY:f4e62ac0e570b83e"}
11:49:38 [11:49:38.675]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.676]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ad9f4704bf470e69 err=test error
{"provingJoId":"1:BASE_PARITY:ad9f4704bf470e69"}
11:49:38 [11:49:38.677]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.688]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d9e094abd6fc9f82 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d9e094abd6fc9f82"}
11:49:38 [11:49:38.689]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d9e094abd6fc9f82 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d9e094abd6fc9f82"}
11:49:38 [11:49:38.689]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d9e094abd6fc9f82 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d9e094abd6fc9f82"}
11:49:38 [11:49:38.700]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:7832796dce6b402c epochNumber=2
{"provingJobId":"2:BASE_PARITY:7832796dce6b402c"}
11:49:38 [11:49:38.701]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:7832796dce6b402c type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:7832796dce6b402c"}
11:49:38 [11:49:38.701]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:7832796dce6b402c type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:7832796dce6b402c"}
11:49:38 [11:49:38.762]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:85477379548d155a epochNumber=3
{"provingJobId":"3:BASE_PARITY:85477379548d155a"}
11:49:38 [11:49:38.779]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:38 [11:49:38.824]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:2ac223efe4a182c2 epochNumber=4
{"provingJobId":"4:BASE_PARITY:2ac223efe4a182c2"}
11:49:38 [11:49:38.831]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:38 [11:49:38.886]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:1638ecec2cd33e2c epochNumber=5
{"provingJobId":"5:BASE_PARITY:1638ecec2cd33e2c"}
11:49:38 [11:49:38.909]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:38 [11:49:38.941]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:009a8becd12b4b2a epochNumber=42
{"provingJobId":"1:BASE_PARITY:009a8becd12b4b2a"}
11:49:38 [11:49:38.943]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testSKNAj8/42 with map size 1073741824
11:49:38 [11:49:38.948]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:38 [11:49:38.976]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:4b4fa4868b172abf epochNumber=1
{"provingJobId":"1:BASE_PARITY:4b4fa4868b172abf"}
11:49:38 [11:49:38.982]
INFO:
prover-client:proving-broker Proving Broker started
11:49:38 [11:49:38.983]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e9067f0a5f344ab3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e9067f0a5f344ab3"}
11:49:38 [11:49:38.984]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7YlYL9/1 with map size 1073741824
11:49:38 [11:49:38.987]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.008]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ad68bd51c8d8cdf epochNumber=1
{"provingJobId":"1:BASE_PARITY:3ad68bd51c8d8cdf"}
11:49:39 [11:49:39.022]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.024]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b4bcd01a1a11d17 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0b4bcd01a1a11d17"}
11:49:39 [11:49:39.025]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtV17KS/1 with map size 1073741824
11:49:39 [11:49:39.028]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.050]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0b4bcd01a1a11d17 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0b4bcd01a1a11d17"}
11:49:39 [11:49:39.055]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.056]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:033ef63b2e240318 epochNumber=1
{"provingJobId":"1:BASE_PARITY:033ef63b2e240318"}
11:49:39 [11:49:39.057]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-teste3i7bz/1 with map size 1073741824
11:49:39 [11:49:39.062]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.082]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:033ef63b2e240318 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:033ef63b2e240318"}
11:49:39 [11:49:39.082]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:033ef63b2e240318 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:033ef63b2e240318"}
11:49:39 [11:49:39.089]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:033ef63b2e240318 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:033ef63b2e240318"}
11:49:39 [11:49:39.108]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.108]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c038445bcd2bcfb epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c038445bcd2bcfb"}
11:49:39 [11:49:39.109]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test00XqkY/1 with map size 1073741824
11:49:39 [11:49:39.112]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.134]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.139]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.140]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d357cc9eeed145fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:d357cc9eeed145fc"}
11:49:39 [11:49:39.141]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWDqWB7/1 with map size 1073741824
11:49:39 [11:49:39.143]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.161]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d357cc9eeed145fc
{"provingJobId":"1:BASE_PARITY:d357cc9eeed145fc"}
11:49:39 [11:49:39.162]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d357cc9eeed145fc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d357cc9eeed145fc"}
11:49:39 [11:49:39.162]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d357cc9eeed145fc type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:d357cc9eeed145fc"}
11:49:39 [11:49:39.173]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:00da23270a7ecdc8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:00da23270a7ecdc8"}
11:49:39 [11:49:39.175]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBAVEVC/1 with map size 1073741824
11:49:39 [11:49:39.178]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.197]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:00da23270a7ecdc8
{"provingJobId":"1:BASE_PARITY:00da23270a7ecdc8"}
11:49:39 [11:49:39.197]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:00da23270a7ecdc8 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:00da23270a7ecdc8"}
11:49:39 [11:49:39.213]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.215]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e080a92e5fdba13 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6e080a92e5fdba13"}
11:49:39 [11:49:39.216]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfqUlVD/1 with map size 1073741824
11:49:39 [11:49:39.221]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.240]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6e080a92e5fdba13 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6e080a92e5fdba13"}
11:49:39 [11:49:39.241]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6e080a92e5fdba13 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6e080a92e5fdba13"}
11:49:39 [11:49:39.253]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c6b56b759d91e3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c6b56b759d91e3e"}
11:49:39 [11:49:39.255]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjBWAEd/1 with map size 1073741824
11:49:39 [11:49:39.257]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.276]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2c6b56b759d91e3e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2c6b56b759d91e3e"}
11:49:39 [11:49:39.277]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2c6b56b759d91e3e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2c6b56b759d91e3e"}
11:49:39 [11:49:39.286]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.287]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9accbbc6e52318a epochNumber=0
{"provingJobId":"1:BASE_PARITY:f9accbbc6e52318a"}
11:49:39 [11:49:39.287]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testyV9rOe/0 with map size 1073741824
11:49:39 [11:49:39.294]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.316]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f9accbbc6e52318a epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f9accbbc6e52318a"}
11:49:39 [11:49:39.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71dece470da8d212 epochNumber=0
{"provingJobId":"1:BASE_PARITY:71dece470da8d212"}
11:49:39 [11:49:39.317]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f9accbbc6e52318a epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f9accbbc6e52318a"}
11:49:39 [11:49:39.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0b445cd01ec2418 epochNumber=0
{"provingJobId":"1:BASE_PARITY:b0b445cd01ec2418"}
11:49:39 [11:49:39.335]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.339]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:24e288aeb705ec92 epochNumber=2
{"provingJobId":"1:BASE_PARITY:24e288aeb705ec92"}
11:49:39 [11:49:39.340]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfI3120/2 with map size 1073741824
11:49:39 [11:49:39.342]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:891bcd9f70dc4149 epochNumber=1
{"provingJobId":"1:BASE_PARITY:891bcd9f70dc4149"}
11:49:39 [11:49:39.363]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfI3120/1 with map size 1073741824
11:49:39 [11:49:39.366]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.392]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.392]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:78e64f72fba547fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:78e64f72fba547fe"}
11:49:39 [11:49:39.393]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgRMJEC/1 with map size 1073741824
11:49:39 [11:49:39.395]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.419]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.420]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e45292558f70765 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6e45292558f70765"}
11:49:39 [11:49:39.421]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwIWafB/1 with map size 1073741824
11:49:39 [11:49:39.422]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:42076a71b13f0f07 epochNumber=1
{"provingJobId":"1:BASE_PARITY:42076a71b13f0f07"}
11:49:39 [11:49:39.448]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c058d8af09179e38 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c058d8af09179e38"}
11:49:39 [11:49:39.450]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwIWafB/2 with map size 1073741824
11:49:39 [11:49:39.452]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.480]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe25da6845a6fb7d epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe25da6845a6fb7d"}
11:49:39 [11:49:39.490]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.491]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5d30a4ed549b82b epochNumber=1
{"provingJobId":"1:BASE_PARITY:c5d30a4ed549b82b"}
11:49:39 [11:49:39.492]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7DyHqo/1 with map size 1073741824
11:49:39 [11:49:39.494]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.521]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c5d352544c03c8b epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c5d352544c03c8b"}
11:49:39 [11:49:39.528]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:107f308d9d3273c7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:107f308d9d3273c7"}
11:49:39 [11:49:39.530]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7DyHqo/2 with map size 1073741824
11:49:39 [11:49:39.532]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.551]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:86e4efa10b977cf4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:86e4efa10b977cf4"}
11:49:39 [11:49:39.562]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.563]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8c9052e219a88d19 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8c9052e219a88d19"}
11:49:39 [11:49:39.564]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeYqaa4/1 with map size 1073741824
11:49:39 [11:49:39.567]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.590]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a271555bb49c37ce epochNumber=1
{"provingJobId":"1:BASE_PARITY:a271555bb49c37ce"}
11:49:39 [11:49:39.599]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25f273a794724b63 epochNumber=2
{"provingJobId":"1:BASE_PARITY:25f273a794724b63"}
11:49:39 [11:49:39.601]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testeYqaa4/2 with map size 1073741824
11:49:39 [11:49:39.606]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.624]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ba98b589b3e7c79c epochNumber=1
{"provingJobId":"1:BASE_PARITY:ba98b589b3e7c79c"}
11:49:39 [11:49:39.635]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.636]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:24863109916c69ea epochNumber=1
{"provingJobId":"1:BASE_PARITY:24863109916c69ea"}
11:49:39 [11:49:39.636]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDwWkHt/1 with map size 1073741824
11:49:39 [11:49:39.638]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.657]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9f1ccdaed77ffaf epochNumber=1
{"provingJobId":"1:BASE_PARITY:f9f1ccdaed77ffaf"}
11:49:39 [11:49:39.663]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:24863109916c69ea type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:24863109916c69ea"}
11:49:39 [11:49:39.672]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.673]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c49c01a1de2237f4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c49c01a1de2237f4"}
11:49:39 [11:49:39.673]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2jRROD/1 with map size 1073741824
11:49:39 [11:49:39.675]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.696]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ab7c36b35101d190 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ab7c36b35101d190"}
11:49:39 [11:49:39.711]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c49c01a1de2237f4 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:c49c01a1de2237f4"}
11:49:39 [11:49:39.722]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:56624261e81111d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:56624261e81111d7"}
11:49:39 [11:49:39.727]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWZSQdp/1 with map size 1073741824
11:49:39 [11:49:39.734]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.753]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b95aac2b3dc3d328 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b95aac2b3dc3d328"}
11:49:39 [11:49:39.760]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:56624261e81111d7 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:56624261e81111d7"}
11:49:39 [11:49:39.764]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.764]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c783cdae00f21e1e epochNumber=1
{"provingJobId":"1:BASE_PARITY:c783cdae00f21e1e"}
11:49:39 [11:49:39.765]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5x19s1/1 with map size 1073741824
11:49:39 [11:49:39.769]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.787]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c783cdae00f21e1e
{"provingJobId":"1:BASE_PARITY:c783cdae00f21e1e"}
11:49:39 [11:49:39.788]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c783cdae00f21e1e type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c783cdae00f21e1e"}
11:49:39 [11:49:39.794]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ad834e519ffbdc6f epochNumber=1
{"provingJobId":"1:BASE_PARITY:ad834e519ffbdc6f"}
11:49:39 [11:49:39.805]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c783cdae00f21e1e has already been completed
{"provingJobId":"1:BASE_PARITY:c783cdae00f21e1e"}
11:49:39 [11:49:39.813]
INFO:
prover-client:proving-broker Proving Broker started
11:49:39 [11:49:39.814]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cf26e57d009e4306 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cf26e57d009e4306"}
11:49:39 [11:49:39.816]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM9fWXQ/1 with map size 1073741824
11:49:39 [11:49:39.818]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:39 [11:49:39.839]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dfccb4c91b501d0b epochNumber=2
{"provingJobId":"1:BASE_PARITY:dfccb4c91b501d0b"}
11:49:39 [11:49:39.840]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testM9fWXQ/2 with map size 1073741824
11:49:39 [11:49:39.846]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:40 [11:49:40.919]
INFO:
prover-client:proving-broker Proving Broker started
11:49:40 [11:49:40.925]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cf26e57d009e4306 settled=false
{"provingJobId":"1:BASE_PARITY:cf26e57d009e4306","status":"pending"}
11:49:40 [11:49:40.927]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dfccb4c91b501d0b settled=false
{"provingJobId":"1:BASE_PARITY:dfccb4c91b501d0b","status":"pending"}
11:49:40 [11:49:40.928]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cf26e57d009e4306 type=BASE_PARITY startedAt=1750679379864 older agent has taken job
{"provingJobId":"1:BASE_PARITY:cf26e57d009e4306"}
11:49:40 [11:49:40.928]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cf26e57d009e4306 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:cf26e57d009e4306"}
11:49:40 [11:49:40.932]
INFO:
prover-client:proving-broker Proving Broker started
11:49:40 [11:49:40.933]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:626eb1656eac5282 epochNumber=1
{"provingJobId":"1:BASE_PARITY:626eb1656eac5282"}
11:49:40 [11:49:40.934]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxAkhQf/1 with map size 1073741824
11:49:40 [11:49:40.936]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:40 [11:49:40.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b4ff8b31f7b0a7f epochNumber=2
{"provingJobId":"1:BASE_PARITY:2b4ff8b31f7b0a7f"}
11:49:40 [11:49:40.956]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testxAkhQf/2 with map size 1073741824
11:49:40 [11:49:40.958]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:41 [11:49:41.980]
INFO:
prover-client:proving-broker Proving Broker started
11:49:41 [11:49:41.982]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:626eb1656eac5282 settled=false
{"provingJobId":"1:BASE_PARITY:626eb1656eac5282","status":"pending"}
11:49:41 [11:49:41.983]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2b4ff8b31f7b0a7f settled=false
{"provingJobId":"1:BASE_PARITY:2b4ff8b31f7b0a7f","status":"pending"}
11:49:41 [11:49:41.984]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:626eb1656eac5282 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:626eb1656eac5282"}
11:49:41 [11:49:41.987]
INFO:
prover-client:proving-broker Proving Broker started
11:49:41 [11:49:41.988]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d9c98cb49f1b6bb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d9c98cb49f1b6bb1"}
11:49:41 [11:49:41.988]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnh3e0n/1 with map size 1073741824
11:49:41 [11:49:41.989]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:42 [11:49:42.012]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:769e5f146e19c377 epochNumber=2
{"provingJobId":"1:BASE_PARITY:769e5f146e19c377"}
11:49:42 [11:49:42.013]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testnh3e0n/2 with map size 1073741824
11:49:42 [11:49:42.017]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.037]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.040]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d9c98cb49f1b6bb1 settled=false
{"provingJobId":"1:BASE_PARITY:d9c98cb49f1b6bb1","status":"pending"}
11:49:43 [11:49:43.041]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:769e5f146e19c377 settled=false
{"provingJobId":"1:BASE_PARITY:769e5f146e19c377","status":"pending"}
11:49:43 [11:49:43.042]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d9c98cb49f1b6bb1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d9c98cb49f1b6bb1"}
11:49:43 [11:49:43.042]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d9c98cb49f1b6bb1 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d9c98cb49f1b6bb1"}
11:49:43 [11:49:43.053]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.054]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8d9e5b400372a21f epochNumber=1
{"provingJobId":"1:BASE_PARITY:8d9e5b400372a21f"}
11:49:43 [11:49:43.055]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAdQbrS/1 with map size 1073741824
11:49:43 [11:49:43.057]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.073]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f38147a217d797b7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:f38147a217d797b7"}
11:49:43 [11:49:43.076]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testAdQbrS/2 with map size 1073741824
11:49:43 [11:49:43.078]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.096]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8d9e5b400372a21f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8d9e5b400372a21f"}
11:49:43 [11:49:43.105]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f38147a217d797b7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f38147a217d797b7"}
11:49:43 [11:49:43.120]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.121]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cd31cf3cee4126c4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cd31cf3cee4126c4"}
11:49:43 [11:49:43.122]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnU1m5D/1 with map size 1073741824
11:49:43 [11:49:43.123]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.140]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:133e3abab11e6cf8 epochNumber=2
{"provingJobId":"1:BASE_PARITY:133e3abab11e6cf8"}
11:49:43 [11:49:43.142]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testnU1m5D/2 with map size 1073741824
11:49:43 [11:49:43.143]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.159]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cd31cf3cee4126c4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cd31cf3cee4126c4"}
11:49:43 [11:49:43.160]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:cd31cf3cee4126c4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:cd31cf3cee4126c4"}
11:49:43 [11:49:43.165]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:133e3abab11e6cf8 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:133e3abab11e6cf8"}
11:49:43 [11:49:43.174]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.175]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:74db40f14d82d07e err=test error
{"provingJoId":"1:BASE_PARITY:74db40f14d82d07e"}
11:49:43 [11:49:43.180]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.181]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e7556c39bdff8c4d not found
{"provingJobId":"1:BASE_PARITY:e7556c39bdff8c4d"}
11:49:43 [11:49:43.183]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.184]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e13dec1cd0a01f8a epochNumber=1
{"provingJobId":"1:BASE_PARITY:e13dec1cd0a01f8a"}
11:49:43 [11:49:43.184]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test69If3D/1 with map size 1073741824
11:49:43 [11:49:43.190]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.219]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.220]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ddada3e3c0be5e1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ddada3e3c0be5e1"}
11:49:43 [11:49:43.221]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5kFVQ5/1 with map size 1073741824
11:49:43 [11:49:43.224]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.346]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2ddada3e3c0be5e1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2ddada3e3c0be5e1"}
11:49:43 [11:49:43.352]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.353]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f507fc4a4d6cfd61 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f507fc4a4d6cfd61"}
11:49:43 [11:49:43.353]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDm2oYq/1 with map size 1073741824
11:49:43 [11:49:43.356]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.482]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f507fc4a4d6cfd61 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:f507fc4a4d6cfd61"}
11:49:43 [11:49:43.484]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0db61f310c0fdc03 epochNumber=10
{"provingJobId":"1:BASE_PARITY:0db61f310c0fdc03"}
11:49:43 [11:49:43.485]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testDm2oYq/10 with map size 1073741824
11:49:43 [11:49:43.487]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.508]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:43 [11:49:43.509]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
11:49:43 [11:49:43.636]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.637]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf42a269a1fe3ea5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bf42a269a1fe3ea5"}
11:49:43 [11:49:43.637]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI4wdF2/1 with map size 1073741824
11:49:43 [11:49:43.639]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.814]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bf42a269a1fe3ea5 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:bf42a269a1fe3ea5"}
11:49:43 [11:49:43.864]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.865]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:92aac10f1b14c076 epochNumber=1
{"provingJobId":"1:BASE_PARITY:92aac10f1b14c076"}
11:49:43 [11:49:43.866]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTWoXBs/1 with map size 1073741824
11:49:43 [11:49:43.868]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.885]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:92aac10f1b14c076 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:92aac10f1b14c076"}
11:49:43 [11:49:43.889]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.890]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:af1e9b66971cacae epochNumber=1
{"provingJobId":"1:BASE_PARITY:af1e9b66971cacae"}
11:49:43 [11:49:43.891]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgrch7m/1 with map size 1073741824
11:49:43 [11:49:43.893]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.922]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:af1e9b66971cacae type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:af1e9b66971cacae"}
11:49:43 [11:49:43.923]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:af1e9b66971cacae type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:af1e9b66971cacae"}
11:49:43 [11:49:43.932]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.933]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1cc5fab078f733ec epochNumber=1
{"provingJobId":"1:BASE_PARITY:1cc5fab078f733ec"}
11:49:43 [11:49:43.935]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVIHNUe/1 with map size 1073741824
11:49:43 [11:49:43.937]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.956]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1cc5fab078f733ec type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1cc5fab078f733ec"}
11:49:43 [11:49:43.965]
INFO:
prover-client:proving-broker Proving Broker started
11:49:43 [11:49:43.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bfb575e1d9273421 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bfb575e1d9273421"}
11:49:43 [11:49:43.967]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjLnNE4/1 with map size 1073741824
11:49:43 [11:49:43.969]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:43 [11:49:43.985]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bfb575e1d9273421 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:bfb575e1d9273421"}
11:49:43 [11:49:43.987]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de6dbbf84c223c35 epochNumber=3
{"provingJobId":"1:BASE_PARITY:de6dbbf84c223c35"}
11:49:43 [11:49:43.988]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testjLnNE4/3 with map size 1073741824
11:49:43 [11:49:43.990]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.017]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:44 [11:49:44.017]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
11:49:44 [11:49:44.036]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:bfb575e1d9273421 err=test error
{"provingJoId":"1:BASE_PARITY:bfb575e1d9273421"}
11:49:44 [11:49:44.041]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUufQkg/1 with map size 1073741824
11:49:44 [11:49:44.044]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.062]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUufQkg/2 with map size 1073741824
11:49:44 [11:49:44.064]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.087]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.089]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:76b022f7b1363333 settled=false
{"provingJobId":"1:BASE_PARITY:76b022f7b1363333","status":"pending"}
11:49:44 [11:49:44.090]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0406cabff1afca4a settled=false
{"provingJobId":"1:BASE_PARITY:0406cabff1afca4a","status":"pending"}
11:49:44 [11:49:44.096]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgCIduG/1 with map size 1073741824
11:49:44 [11:49:44.098]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.124]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testgCIduG/2 with map size 1073741824
11:49:44 [11:49:44.127]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.154]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.156]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1f0e084257d4aa4c settled=true
{"provingJobId":"1:BASE_PARITY:1f0e084257d4aa4c","status":"fulfilled"}
11:49:44 [11:49:44.159]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:6deecd6324e164d7 settled=true
{"provingJobId":"2:BASE_PARITY:6deecd6324e164d7","status":"fulfilled"}
11:49:44 [11:49:44.166]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTtnepu/1 with map size 1073741824
11:49:44 [11:49:44.169]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.191]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTtnepu/2 with map size 1073741824
11:49:44 [11:49:44.195]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.215]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.218]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ce911bde00440b1c settled=true
{"provingJobId":"1:BASE_PARITY:ce911bde00440b1c","status":"fulfilled"}
11:49:44 [11:49:44.220]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a7a825a1cf54c21d settled=false
{"provingJobId":"1:BASE_PARITY:a7a825a1cf54c21d","status":"pending"}
11:49:44 [11:49:44.225]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.226]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:655bdbcfb14e110b epochNumber=1
{"provingJobId":"1:BASE_PARITY:655bdbcfb14e110b"}
11:49:44 [11:49:44.227]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0AwHZJ/1 with map size 1073741824
11:49:44 [11:49:44.228]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.250]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.251]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55df209f1d887ad4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55df209f1d887ad4"}
11:49:44 [11:49:44.252]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:55df209f1d887ad4: Error: db error: Error: db error
11:49:44 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:55df209f1d887ad4"}
11:49:44 [11:49:44.259]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:76ca43db959d80bd epochNumber=1
{"provingJobId":"1:BASE_PARITY:76ca43db959d80bd"}
11:49:44 [11:49:44.262]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoaH0sH/1 with map size 1073741824
11:49:44 [11:49:44.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.292]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:76ca43db959d80bd type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:76ca43db959d80bd"}
11:49:44 [11:49:44.292]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:76ca43db959d80bd type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:76ca43db959d80bd"}
11:49:44 [11:49:44.308]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.309]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2a15358b07ec671 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2a15358b07ec671"}
11:49:44 [11:49:44.310]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqzN6h7/1 with map size 1073741824
11:49:44 [11:49:44.319]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.339]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f2a15358b07ec671 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f2a15358b07ec671"}
11:49:44 [11:49:44.340]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f2a15358b07ec671 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f2a15358b07ec671"}
11:49:44 [11:49:44.341]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:f2a15358b07ec671: Error: db error
11:49:44 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:f2a15358b07ec671"}
11:49:44 [11:49:44.346]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.346]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f64992c207e765a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f64992c207e765a0"}
11:49:44 [11:49:44.347]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiAQAQv/1 with map size 1073741824
11:49:44 [11:49:44.353]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.371]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f64992c207e765a0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f64992c207e765a0"}
11:49:44 [11:49:44.372]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f64992c207e765a0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f64992c207e765a0"}
11:49:44 [11:49:44.385]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.386]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49312f4f309a3f77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:49312f4f309a3f77"}
11:49:44 [11:49:44.387]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6ZfV4d/1 with map size 1073741824
11:49:44 [11:49:44.391]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.414]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:49312f4f309a3f77 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:49312f4f309a3f77"}
11:49:44 [11:49:44.415]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:49312f4f309a3f77 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:49312f4f309a3f77"}
11:49:44 [11:49:44.417]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:49312f4f309a3f77 jobErr=test error: Error: db error
11:49:44 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:49312f4f309a3f77"}
11:49:44 [11:49:44.425]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.426]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e39820f1f8905c1e not found
{"provingJobId":"1:BASE_PARITY:e39820f1f8905c1e"}
11:49:44 [11:49:44.430]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.431]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:02da9aeba3600a17 err=test error
{"provingJoId":"1:BASE_PARITY:02da9aeba3600a17"}
11:49:44 [11:49:44.435]
INFO:
prover-client:proving-broker Proving Broker started
11:49:44 [11:49:44.446]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:447255ea025ed1c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:447255ea025ed1c5"}
11:49:44 [11:49:44.447]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyq2w9D/1 with map size 1073741824
11:49:44 [11:49:44.450]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.471]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:447255ea025ed1c5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:447255ea025ed1c5"}
11:49:44 [11:49:44.472]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:447255ea025ed1c5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:447255ea025ed1c5"}
11:49:44 [11:49:44.490]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:5bd63577a36bef98 epochNumber=2
{"provingJobId":"2:BASE_PARITY:5bd63577a36bef98"}
11:49:44 [11:49:44.490]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyq2w9D/2 with map size 1073741824
11:49:44 [11:49:44.494]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.517]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:5bd63577a36bef98 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:5bd63577a36bef98"}
11:49:44 [11:49:44.518]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:5bd63577a36bef98 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:5bd63577a36bef98"}
11:49:44 [11:49:44.587]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:d3db06ff4da20830 epochNumber=3
{"provingJobId":"3:BASE_PARITY:d3db06ff4da20830"}
11:49:44 [11:49:44.588]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testyq2w9D/3 with map size 1073741824
11:49:44 [11:49:44.590]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.638]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:44 [11:49:44.639]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
11:49:44 [11:49:44.676]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:ba6007d1d5274e96 epochNumber=4
{"provingJobId":"4:BASE_PARITY:ba6007d1d5274e96"}
11:49:44 [11:49:44.677]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testyq2w9D/4 with map size 1073741824
11:49:44 [11:49:44.680]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.718]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:44 [11:49:44.719]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
11:49:44 [11:49:44.765]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:872945e51bd04c8a epochNumber=5
{"provingJobId":"5:BASE_PARITY:872945e51bd04c8a"}
11:49:44 [11:49:44.766]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testyq2w9D/5 with map size 1073741824
11:49:44 [11:49:44.768]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
11:49:44 [11:49:44.799]
INFO:
prover-client:proving-broker Cleaned up jobs=1
11:49:44 [11:49:44.800]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
11:49:44
PASS src/proving_broker/proving_broker.test.ts (
13.248 s)
11:49:44 ProvingBroker
11:49:44 Producer API
11:49:44
✓ refuses stale jobs (35 ms)
11:49:44
✓ enqueues jobs (4 ms)
11:49:44
✓ ignores duplicate jobs (3 ms)
11:49:44
✓ reports correct status when enqueuing repeat jobs (4 ms)
11:49:44
✓ throws an error in case of duplicate job IDs (30 ms)
11:49:44
✓ returns not-found status for non-existing jobs (2 ms)
11:49:44
✓ cancels jobs in queue (3 ms)
11:49:44
✓ cancels jobs in-progress (2 ms)
11:49:44
✓ returns job result if successful (3 ms)
11:49:44
✓ returns job error if failed (2 ms)
11:49:44
✓ correctly returns job status for concurrent writes (2 ms)
11:49:44
✓ refuses stale jobs (42 ms)
11:49:44
✓ enqueues jobs (37 ms)
11:49:44
✓ ignores duplicate jobs (34 ms)
11:49:44
✓ reports correct status when enqueuing repeat jobs (51 ms)
11:49:44
✓ throws an error in case of duplicate job IDs (27 ms)
11:49:44
✓ returns not-found status for non-existing jobs (3 ms)
11:49:44
✓ cancels jobs in queue (35 ms)
11:49:44
✓ cancels jobs in-progress (37 ms)
11:49:44
✓ returns job result if successful (43 ms)
11:49:44
✓ returns job error if failed (32 ms)
11:49:44
✓ correctly returns job status for concurrent writes (48 ms)
11:49:44 Consumer API
11:49:44
✓ returns undefined if no jobs are available (1 ms)
11:49:44
✓ returns jobs in priority order (2 ms)
11:49:44
✓ returns undefined if no jobs are available for the given allowList (1 ms)
11:49:44
✓ returns a job if it is in the allowList (11 ms)
11:49:44
✓ returns the most important job if it is in the allowList (3 ms)
11:49:44
✓ returns any job if filter is empty (3 ms)
11:49:44
✓ returns a new job when reporting job success (2 ms)
11:49:44
✓ returns a new job when reporting permanent error (2 ms)
11:49:44
✓ returns a new job when reporting retry-able error (3 ms)
11:49:44
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
11:49:44
✓ returns a new job if job is already in progress elsewhere (1058 ms)
11:49:44
✓ avoids sending the same job to a new agent after a restart (1005 ms)
11:49:44
✓ avoids sending a completed job to a new agent after a restart (1008 ms)
11:49:44
✓ tracks job result if in progress (4 ms)
11:49:44
✓ tracks job result even if job is in queue (3 ms)
11:49:44
✓ ignores reported job error if unknown job (2 ms)
11:49:44
✓ ignores job result if unknown job (1 ms)
11:49:44
✓ returns undefined if no jobs are available (2 ms)
11:49:44
✓ returns jobs in priority order (49 ms)
11:49:44
✓ returns undefined if no jobs are available for the given allowList (31 ms)
11:49:44
✓ returns a job if it is in the allowList (71 ms)
11:49:44
✓ returns the most important job if it is in the allowList (71 ms)
11:49:44
✓ returns any job if filter is empty (72 ms)
11:49:44
✓ returns a new job when reporting job success (37 ms)
11:49:44
✓ returns a new job when reporting permanent error (48 ms)
11:49:44
✓ returns a new job when reporting retry-able error (42 ms)
11:49:44
✓ returns a new job when reporting progress if current one is cancelled (48 ms)
11:49:44
✓ returns a new job if job is already in progress elsewhere (1120 ms)
11:49:44
✓ avoids sending the same job to a new agent after a restart (1055 ms)
11:49:44
✓ avoids sending a completed job to a new agent after a restart (1065 ms)
11:49:44
✓ tracks job result if in progress (64 ms)
11:49:44
✓ tracks job result even if job is in queue (57 ms)
11:49:44
✓ ignores reported job error if unknown job (5 ms)
11:49:44
✓ ignores job result if unknown job (5 ms)
11:49:44 Timeouts
11:49:44
✓ tracks in progress jobs (2 ms)
11:49:44
✓ re-enqueues jobs that time out (108 ms)
11:49:44
✓ cancel stale jobs that time out (235 ms)
11:49:44
✓ keeps the jobs in progress while it is alive (205 ms)
11:49:44
✓ tracks in progress jobs (34 ms)
11:49:44
✓ re-enqueues jobs that time out (134 ms)
11:49:44
✓ cancel stale jobs that time out (284 ms)
11:49:44
✓ keeps the jobs in progress while it is alive (228 ms)
11:49:44 Retries
11:49:44
✓ retries jobs (3 ms)
11:49:44
✓ retries up to a maximum number of times (2 ms)
11:49:44
✓ passing retry=false does not retry (2 ms)
11:49:44
✓ does not retry if job is stale (28 ms)
11:49:44
✓ retries jobs (24 ms)
11:49:44
✓ retries up to a maximum number of times (42 ms)
11:49:44
✓ passing retry=false does not retry (33 ms)
11:49:44
✓ does not retry if job is stale (75 ms)
11:49:44 Database management
11:49:44
✓ re-enqueues proof requests on start (3 ms)
11:49:44
✓ restores proof results on start (1 ms)
11:49:44
✓ only re-enqueues unfinished jobs (1 ms)
11:49:44
✓ saves job when enqueued (4 ms)
11:49:44
✓ does not retain job if database fails to save (3 ms)
11:49:44
✓ saves job result (4 ms)
11:49:44
✓ saves result even if database fails to save (4 ms)
11:49:44
✓ saves job error (3 ms)
11:49:44
✓ saves job error even if database fails to save (2 ms)
11:49:44
✓ does not save job result if job is unknown (1 ms)
11:49:44
✓ does not save job error if job is unknown (6 ms)
11:49:44
✓ cleans up old jobs periodically (261 ms)
11:49:44
✓ re-enqueues proof requests on start (54 ms)
11:49:44
✓ restores proof results on start (68 ms)
11:49:44
✓ only re-enqueues unfinished jobs (61 ms)
11:49:44
✓ saves job when enqueued (23 ms)
11:49:44
✓ does not retain job if database fails to save (6 ms)
11:49:44
✓ saves job result (46 ms)
11:49:44
✓ saves result even if database fails to save (42 ms)
11:49:44
✓ saves job error (37 ms)
11:49:44
✓ saves job error even if database fails to save (40 ms)
11:49:44
✓ does not save job result if job is unknown (6 ms)
11:49:44
✓ does not save job error if job is unknown (4 ms)
11:49:44
✓ cleans up old jobs periodically (411 ms)
11:49:44
11:49:44
Test Suites: 1 passed, 1 total
11:49:44
Tests: 96 passed, 96 total
11:49:44
Snapshots: 0 total
11:49:44
Time: 13.354 s
11:49:44
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
11:49:44
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?