Parent Log:
http://ci.aztec-labs.com/d9cbe39da05505e1
Command: f682b45a820409ef yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/c0e766a133071c44b2c0d8fac5df352c6e059df1
Env: REF_NAME=gh-readonly-queue/next/pr-15015-cf8be0f9e81e248048560619de041e90d9d6990a CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Fri Jun 13 07:55:53 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15015_amd64_x1-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
07:55:57 [07:55:57.832]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.839]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3b53fb5c1978cb47 epochNumber=42
{"provingJobId":"1:BASE_PARITY:3b53fb5c1978cb47"}
07:55:57 [07:55:57.843]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:06be38fb030f17b2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:06be38fb030f17b2"}
07:55:57 [07:55:57.883]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.884]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ee727bbfdec7bd64 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ee727bbfdec7bd64"}
07:55:57 [07:55:57.886]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7e2a6602cb05fad8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e2a6602cb05fad8"}
07:55:57 [07:55:57.889]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.890]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d1e96f5064105a74 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d1e96f5064105a74"}
07:55:57 [07:55:57.892]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d1e96f5064105a74 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d1e96f5064105a74"}
07:55:57 [07:55:57.894]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.895]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:995f5e855c79b687 epochNumber=1
{"provingJobId":"1:BASE_PARITY:995f5e855c79b687"}
07:55:57 [07:55:57.896]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:995f5e855c79b687 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:995f5e855c79b687"}
07:55:57 [07:55:57.897]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:995f5e855c79b687 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:995f5e855c79b687"}
07:55:57 [07:55:57.897]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:995f5e855c79b687 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:995f5e855c79b687"}
07:55:57 [07:55:57.898]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.899]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0ee8e77f4d7e3d66 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0ee8e77f4d7e3d66"}
07:55:57 [07:55:57.928]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.930]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.930]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6bf296100f4d9e21 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6bf296100f4d9e21"}
07:55:57 [07:55:57.931]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:6bf296100f4d9e21
{"provingJobId":"1:BASE_PARITY:6bf296100f4d9e21"}
07:55:57 [07:55:57.932]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6bf296100f4d9e21 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6bf296100f4d9e21"}
07:55:57 [07:55:57.932]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6bf296100f4d9e21 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:6bf296100f4d9e21"}
07:55:57 [07:55:57.933]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.934]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e5d1e5d145be40b7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e5d1e5d145be40b7"}
07:55:57 [07:55:57.935]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e5d1e5d145be40b7
{"provingJobId":"1:BASE_PARITY:e5d1e5d145be40b7"}
07:55:57 [07:55:57.935]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e5d1e5d145be40b7 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:e5d1e5d145be40b7"}
07:55:57 [07:55:57.936]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d73529acc9f0dd1f epochNumber=1
{"provingJobId":"1:BASE_PARITY:d73529acc9f0dd1f"}
07:55:57 [07:55:57.937]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d73529acc9f0dd1f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d73529acc9f0dd1f"}
07:55:57 [07:55:57.938]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d73529acc9f0dd1f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d73529acc9f0dd1f"}
07:55:57 [07:55:57.945]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.946]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9a0bf266748877dc epochNumber=1
{"provingJobId":"1:BASE_PARITY:9a0bf266748877dc"}
07:55:57 [07:55:57.947]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9a0bf266748877dc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9a0bf266748877dc"}
07:55:57 [07:55:57.947]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9a0bf266748877dc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9a0bf266748877dc"}
07:55:57 [07:55:57.948]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.949]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d14de04dafdd91a2 epochNumber=0
{"provingJobId":"1:BASE_PARITY:d14de04dafdd91a2"}
07:55:57 [07:55:57.949]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d14de04dafdd91a2 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d14de04dafdd91a2"}
07:55:57 [07:55:57.949]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b4dbecc3c5a20618 epochNumber=0
{"provingJobId":"1:BASE_PARITY:b4dbecc3c5a20618"}
07:55:57 [07:55:57.949]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d14de04dafdd91a2 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d14de04dafdd91a2"}
07:55:57 [07:55:57.950]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8f6157dec225977b epochNumber=0
{"provingJobId":"1:BASE_PARITY:8f6157dec225977b"}
07:55:57 [07:55:57.951]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.952]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.952]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:97c5586f0073362c epochNumber=2
{"provingJobId":"1:BASE_PARITY:97c5586f0073362c"}
07:55:57 [07:55:57.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:425d50d350361fff epochNumber=1
{"provingJobId":"1:BASE_PARITY:425d50d350361fff"}
07:55:57 [07:55:57.954]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7e2fbe42d10032f epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7e2fbe42d10032f"}
07:55:57 [07:55:57.955]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.955]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc85030ef5433a62 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc85030ef5433a62"}
07:55:57 [07:55:57.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:482528127df98bc4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:482528127df98bc4"}
07:55:57 [07:55:57.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de6961c3baa626e7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:de6961c3baa626e7"}
07:55:57 [07:55:57.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6181f48459ed8474 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6181f48459ed8474"}
07:55:57 [07:55:57.958]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.958]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c87aa9a0e72142f3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c87aa9a0e72142f3"}
07:55:57 [07:55:57.958]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:456079c00bbfbd80 epochNumber=1
{"provingJobId":"1:BASE_PARITY:456079c00bbfbd80"}
07:55:57 [07:55:57.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4e9b615d039f5c3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:e4e9b615d039f5c3"}
07:55:57 [07:55:57.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:08319c11c9be5140 epochNumber=1
{"provingJobId":"1:BASE_PARITY:08319c11c9be5140"}
07:55:57 [07:55:57.960]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6a8b381b483aadb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6a8b381b483aadb1"}
07:55:57 [07:55:57.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1eda3b7397d18faa epochNumber=1
{"provingJobId":"1:BASE_PARITY:1eda3b7397d18faa"}
07:55:57 [07:55:57.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:636bacecd711c541 epochNumber=2
{"provingJobId":"1:BASE_PARITY:636bacecd711c541"}
07:55:57 [07:55:57.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e09618fb1d9efd29 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e09618fb1d9efd29"}
07:55:57 [07:55:57.963]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.963]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:74f163360be01acf epochNumber=1
{"provingJobId":"1:BASE_PARITY:74f163360be01acf"}
07:55:57 [07:55:57.964]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc754d6033ece5da epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc754d6033ece5da"}
07:55:57 [07:55:57.964]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:74f163360be01acf type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:74f163360be01acf"}
07:55:57 [07:55:57.966]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2cc0f6d976c010f epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2cc0f6d976c010f"}
07:55:57 [07:55:57.967]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1b075fcd7563637e epochNumber=1
{"provingJobId":"1:BASE_PARITY:1b075fcd7563637e"}
07:55:57 [07:55:57.967]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a2cc0f6d976c010f type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:a2cc0f6d976c010f"}
07:55:57 [07:55:57.968]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0f8025300c1b75d epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0f8025300c1b75d"}
07:55:57 [07:55:57.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4a2d52c77b526d20 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4a2d52c77b526d20"}
07:55:57 [07:55:57.969]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e0f8025300c1b75d type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:e0f8025300c1b75d"}
07:55:57 [07:55:57.975]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.976]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9cb16f99e45b1f1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f9cb16f99e45b1f1"}
07:55:57 [07:55:57.977]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:f9cb16f99e45b1f1
{"provingJobId":"1:BASE_PARITY:f9cb16f99e45b1f1"}
07:55:57 [07:55:57.978]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f9cb16f99e45b1f1 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:f9cb16f99e45b1f1"}
07:55:57 [07:55:57.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c61014867089bb3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c61014867089bb3"}
07:55:57 [07:55:57.979]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f9cb16f99e45b1f1 has already been completed
{"provingJobId":"1:BASE_PARITY:f9cb16f99e45b1f1"}
07:55:57 [07:55:57.981]
INFO:
prover-client:proving-broker Proving Broker started
07:55:57 [07:55:57.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6d2920af1c2484e2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6d2920af1c2484e2"}
07:55:57 [07:55:57.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d84bfab9900275ea epochNumber=2
{"provingJobId":"1:BASE_PARITY:d84bfab9900275ea"}
07:55:59 [07:55:59.035]
INFO:
prover-client:proving-broker Proving Broker started
07:55:59 [07:55:59.036]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6d2920af1c2484e2 settled=false
{"provingJobId":"1:BASE_PARITY:6d2920af1c2484e2","status":"pending"}
07:55:59 [07:55:59.036]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d84bfab9900275ea settled=false
{"provingJobId":"1:BASE_PARITY:d84bfab9900275ea","status":"pending"}
07:55:59 [07:55:59.036]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6d2920af1c2484e2 type=BASE_PARITY startedAt=1749801357982 older agent has taken job
{"provingJobId":"1:BASE_PARITY:6d2920af1c2484e2"}
07:55:59 [07:55:59.037]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6d2920af1c2484e2 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:6d2920af1c2484e2"}
07:55:59 [07:55:59.037]
INFO:
prover-client:proving-broker Proving Broker started
07:55:59 [07:55:59.038]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:daedbbdbfcf91f7b epochNumber=1
{"provingJobId":"1:BASE_PARITY:daedbbdbfcf91f7b"}
07:55:59 [07:55:59.038]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c0183e77521d42e2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c0183e77521d42e2"}
07:56:00 [07:56:00.040]
INFO:
prover-client:proving-broker Proving Broker started
07:56:00 [07:56:00.041]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:daedbbdbfcf91f7b settled=false
{"provingJobId":"1:BASE_PARITY:daedbbdbfcf91f7b","status":"pending"}
07:56:00 [07:56:00.041]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c0183e77521d42e2 settled=false
{"provingJobId":"1:BASE_PARITY:c0183e77521d42e2","status":"pending"}
07:56:00 [07:56:00.042]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:daedbbdbfcf91f7b type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:daedbbdbfcf91f7b"}
07:56:00 [07:56:00.043]
INFO:
prover-client:proving-broker Proving Broker started
07:56:00 [07:56:00.044]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0fd37cd61962107a epochNumber=1
{"provingJobId":"1:BASE_PARITY:0fd37cd61962107a"}
07:56:00 [07:56:00.044]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6ec7f09889ec3df0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:6ec7f09889ec3df0"}
07:56:01 [07:56:01.045]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.046]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0fd37cd61962107a settled=false
{"provingJobId":"1:BASE_PARITY:0fd37cd61962107a","status":"pending"}
07:56:01 [07:56:01.046]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6ec7f09889ec3df0 settled=false
{"provingJobId":"1:BASE_PARITY:6ec7f09889ec3df0","status":"pending"}
07:56:01 [07:56:01.047]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0fd37cd61962107a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0fd37cd61962107a"}
07:56:01 [07:56:01.047]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0fd37cd61962107a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0fd37cd61962107a"}
07:56:01 [07:56:01.048]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.049]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f50fddabad5d5502 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f50fddabad5d5502"}
07:56:01 [07:56:01.049]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b601b8863a1cf72 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9b601b8863a1cf72"}
07:56:01 [07:56:01.050]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f50fddabad5d5502 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f50fddabad5d5502"}
07:56:01 [07:56:01.050]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9b601b8863a1cf72 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9b601b8863a1cf72"}
07:56:01 [07:56:01.052]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.052]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eb82f60c4ba0b8de epochNumber=1
{"provingJobId":"1:BASE_PARITY:eb82f60c4ba0b8de"}
07:56:01 [07:56:01.052]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:005cdb947c764dc7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:005cdb947c764dc7"}
07:56:01 [07:56:01.053]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eb82f60c4ba0b8de type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:eb82f60c4ba0b8de"}
07:56:01 [07:56:01.053]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:eb82f60c4ba0b8de type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:eb82f60c4ba0b8de"}
07:56:01 [07:56:01.053]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:005cdb947c764dc7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:005cdb947c764dc7"}
07:56:01 [07:56:01.054]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.055]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a38c147b333cf287 err=test error
{"provingJoId":"1:BASE_PARITY:a38c147b333cf287"}
07:56:01 [07:56:01.055]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.056]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:82b0566b25c7d708 not found
{"provingJobId":"1:BASE_PARITY:82b0566b25c7d708"}
07:56:01 [07:56:01.057]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e5751e6b2602f77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e5751e6b2602f77"}
07:56:01 [07:56:01.059]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.059]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce282866d71ab5d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce282866d71ab5d9"}
07:56:01 [07:56:01.160]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ce282866d71ab5d9 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:ce282866d71ab5d9"}
07:56:01 [07:56:01.167]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.167]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:586f4ad23cc7a105 epochNumber=1
{"provingJobId":"1:BASE_PARITY:586f4ad23cc7a105"}
07:56:01 [07:56:01.268]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:586f4ad23cc7a105 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:586f4ad23cc7a105"}
07:56:01 [07:56:01.275]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f4793aa70fdfd8d0 epochNumber=10
{"provingJobId":"1:BASE_PARITY:f4793aa70fdfd8d0"}
07:56:01 [07:56:01.298]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:01 [07:56:01.404]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.405]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ace529d4a1799360 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ace529d4a1799360"}
07:56:01 [07:56:01.557]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ace529d4a1799360 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:ace529d4a1799360"}
07:56:01 [07:56:01.608]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.609]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:95b1ef7ddbcad522 epochNumber=1
{"provingJobId":"1:BASE_PARITY:95b1ef7ddbcad522"}
07:56:01 [07:56:01.609]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:95b1ef7ddbcad522 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:95b1ef7ddbcad522"}
07:56:01 [07:56:01.610]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.611]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc4f25080ef3a5a5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bc4f25080ef3a5a5"}
07:56:01 [07:56:01.612]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bc4f25080ef3a5a5 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:bc4f25080ef3a5a5"}
07:56:01 [07:56:01.614]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bc4f25080ef3a5a5 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:bc4f25080ef3a5a5"}
07:56:01 [07:56:01.616]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.617]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:421e4fb560103a6d epochNumber=1
{"provingJobId":"1:BASE_PARITY:421e4fb560103a6d"}
07:56:01 [07:56:01.617]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:421e4fb560103a6d type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:421e4fb560103a6d"}
07:56:01 [07:56:01.618]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.619]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3baecfe640f56d80 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3baecfe640f56d80"}
07:56:01 [07:56:01.620]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3baecfe640f56d80 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3baecfe640f56d80"}
07:56:01 [07:56:01.621]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7f2737bd84ac58d epochNumber=3
{"provingJobId":"1:BASE_PARITY:b7f2737bd84ac58d"}
07:56:01 [07:56:01.644]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:01 [07:56:01.646]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:3baecfe640f56d80 err=test error
{"provingJoId":"1:BASE_PARITY:3baecfe640f56d80"}
07:56:01 [07:56:01.648]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.648]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:30f42b821a24d4c8 settled=false
{"provingJobId":"1:BASE_PARITY:30f42b821a24d4c8","status":"pending"}
07:56:01 [07:56:01.649]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ce5804e15daf475f settled=false
{"provingJobId":"1:BASE_PARITY:ce5804e15daf475f","status":"pending"}
07:56:01 [07:56:01.651]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.651]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3c807fce20702b67 settled=true
{"provingJobId":"1:BASE_PARITY:3c807fce20702b67","status":"fulfilled"}
07:56:01 [07:56:01.652]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:9a957fa7081252c4 settled=true
{"provingJobId":"2:BASE_PARITY:9a957fa7081252c4","status":"fulfilled"}
07:56:01 [07:56:01.655]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.655]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c90bdc89bdcd8f55 settled=true
{"provingJobId":"1:BASE_PARITY:c90bdc89bdcd8f55","status":"fulfilled"}
07:56:01 [07:56:01.655]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fbbbe6860ee4a925 settled=false
{"provingJobId":"1:BASE_PARITY:fbbbe6860ee4a925","status":"pending"}
07:56:01 [07:56:01.656]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.657]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38d992d8a5e9e9d1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38d992d8a5e9e9d1"}
07:56:01 [07:56:01.659]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55769040beba8392 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55769040beba8392"}
07:56:01 [07:56:01.660]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:55769040beba8392: Error: db error: Error: db error
07:56:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:55769040beba8392"}
07:56:01 [07:56:01.661]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.662]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:47e82625654ce58a epochNumber=1
{"provingJobId":"1:BASE_PARITY:47e82625654ce58a"}
07:56:01 [07:56:01.662]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:47e82625654ce58a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:47e82625654ce58a"}
07:56:01 [07:56:01.662]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:47e82625654ce58a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:47e82625654ce58a"}
07:56:01 [07:56:01.663]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.663]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60aaa8992a5ab1db epochNumber=1
{"provingJobId":"1:BASE_PARITY:60aaa8992a5ab1db"}
07:56:01 [07:56:01.663]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:60aaa8992a5ab1db type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:60aaa8992a5ab1db"}
07:56:01 [07:56:01.664]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:60aaa8992a5ab1db type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:60aaa8992a5ab1db"}
07:56:01 [07:56:01.664]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:60aaa8992a5ab1db: Error: db error
07:56:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:60aaa8992a5ab1db"}
07:56:01 [07:56:01.665]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.665]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:545655021ff8208c epochNumber=1
{"provingJobId":"1:BASE_PARITY:545655021ff8208c"}
07:56:01 [07:56:01.665]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:545655021ff8208c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:545655021ff8208c"}
07:56:01 [07:56:01.665]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:545655021ff8208c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:545655021ff8208c"}
07:56:01 [07:56:01.666]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.667]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:67f98fee88a27c47 epochNumber=1
{"provingJobId":"1:BASE_PARITY:67f98fee88a27c47"}
07:56:01 [07:56:01.667]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:67f98fee88a27c47 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:67f98fee88a27c47"}
07:56:01 [07:56:01.667]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:67f98fee88a27c47 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:67f98fee88a27c47"}
07:56:01 [07:56:01.667]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:67f98fee88a27c47 jobErr=test error: Error: db error
07:56:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:67f98fee88a27c47"}
07:56:01 [07:56:01.668]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.668]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:59e53354c5e5a0d1 not found
{"provingJobId":"1:BASE_PARITY:59e53354c5e5a0d1"}
07:56:01 [07:56:01.670]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.670]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:2c396c4903fdbd12 err=test error
{"provingJoId":"1:BASE_PARITY:2c396c4903fdbd12"}
07:56:01 [07:56:01.671]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.683]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:11f4abc9d4334657 epochNumber=1
{"provingJobId":"1:BASE_PARITY:11f4abc9d4334657"}
07:56:01 [07:56:01.684]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:11f4abc9d4334657 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:11f4abc9d4334657"}
07:56:01 [07:56:01.684]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:11f4abc9d4334657 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:11f4abc9d4334657"}
07:56:01 [07:56:01.694]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:0a878519456a83e2 epochNumber=2
{"provingJobId":"2:BASE_PARITY:0a878519456a83e2"}
07:56:01 [07:56:01.696]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:0a878519456a83e2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:0a878519456a83e2"}
07:56:01 [07:56:01.697]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:0a878519456a83e2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:0a878519456a83e2"}
07:56:01 [07:56:01.759]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:00b2abfd3d13bb2c epochNumber=3
{"provingJobId":"3:BASE_PARITY:00b2abfd3d13bb2c"}
07:56:01 [07:56:01.777]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:01 [07:56:01.820]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:020f611e515b42b0 epochNumber=4
{"provingJobId":"4:BASE_PARITY:020f611e515b42b0"}
07:56:01 [07:56:01.828]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:01 [07:56:01.882]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:7a733203749c3a7b epochNumber=5
{"provingJobId":"5:BASE_PARITY:7a733203749c3a7b"}
07:56:01 [07:56:01.904]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:01 [07:56:01.938]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1be832a7b04f5721 epochNumber=42
{"provingJobId":"1:BASE_PARITY:1be832a7b04f5721"}
07:56:01 [07:56:01.942]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testVirtMj/42 with map size 1073741824
07:56:01 [07:56:01.947]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:01 [07:56:01.978]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:43f65965b4b7e431 epochNumber=1
{"provingJobId":"1:BASE_PARITY:43f65965b4b7e431"}
07:56:01 [07:56:01.983]
INFO:
prover-client:proving-broker Proving Broker started
07:56:01 [07:56:01.983]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc1cf7db4407d617 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc1cf7db4407d617"}
07:56:01 [07:56:01.985]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRPZIIE/1 with map size 1073741824
07:56:01 [07:56:01.988]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.009]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12eeeea7196d071d epochNumber=1
{"provingJobId":"1:BASE_PARITY:12eeeea7196d071d"}
07:56:02 [07:56:02.018]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.020]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:72c5bd027c1bbc8c epochNumber=1
{"provingJobId":"1:BASE_PARITY:72c5bd027c1bbc8c"}
07:56:02 [07:56:02.022]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testB3DJkN/1 with map size 1073741824
07:56:02 [07:56:02.025]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.044]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:72c5bd027c1bbc8c epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:72c5bd027c1bbc8c"}
07:56:02 [07:56:02.055]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:935f76643e7fd3c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:935f76643e7fd3c2"}
07:56:02 [07:56:02.056]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqeBn0m/1 with map size 1073741824
07:56:02 [07:56:02.057]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.078]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:935f76643e7fd3c2 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:935f76643e7fd3c2"}
07:56:02 [07:56:02.079]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:935f76643e7fd3c2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:935f76643e7fd3c2"}
07:56:02 [07:56:02.087]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:935f76643e7fd3c2 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:935f76643e7fd3c2"}
07:56:02 [07:56:02.093]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.095]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:15a1a890c87ab809 epochNumber=1
{"provingJobId":"1:BASE_PARITY:15a1a890c87ab809"}
07:56:02 [07:56:02.098]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testf6jmq1/1 with map size 1073741824
07:56:02 [07:56:02.101]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.125]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.129]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.130]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb3270a7daa00588 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cb3270a7daa00588"}
07:56:02 [07:56:02.132]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGRH2Bo/1 with map size 1073741824
07:56:02 [07:56:02.135]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.161]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cb3270a7daa00588
{"provingJobId":"1:BASE_PARITY:cb3270a7daa00588"}
07:56:02 [07:56:02.162]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cb3270a7daa00588 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cb3270a7daa00588"}
07:56:02 [07:56:02.162]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cb3270a7daa00588 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:cb3270a7daa00588"}
07:56:02 [07:56:02.175]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.176]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c979666e5e280d25 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c979666e5e280d25"}
07:56:02 [07:56:02.178]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxzAEZx/1 with map size 1073741824
07:56:02 [07:56:02.182]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.203]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c979666e5e280d25
{"provingJobId":"1:BASE_PARITY:c979666e5e280d25"}
07:56:02 [07:56:02.204]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c979666e5e280d25 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c979666e5e280d25"}
07:56:02 [07:56:02.215]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.216]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebed6c98c24e91ba epochNumber=1
{"provingJobId":"1:BASE_PARITY:ebed6c98c24e91ba"}
07:56:02 [07:56:02.217]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testB00STI/1 with map size 1073741824
07:56:02 [07:56:02.221]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.240]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ebed6c98c24e91ba type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ebed6c98c24e91ba"}
07:56:02 [07:56:02.240]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ebed6c98c24e91ba type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ebed6c98c24e91ba"}
07:56:02 [07:56:02.249]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.250]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6f135ffe02d61f94 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6f135ffe02d61f94"}
07:56:02 [07:56:02.250]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLr0uxi/1 with map size 1073741824
07:56:02 [07:56:02.253]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.273]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6f135ffe02d61f94 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6f135ffe02d61f94"}
07:56:02 [07:56:02.274]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6f135ffe02d61f94 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6f135ffe02d61f94"}
07:56:02 [07:56:02.286]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.286]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9bccd7194b64c825 epochNumber=0
{"provingJobId":"1:BASE_PARITY:9bccd7194b64c825"}
07:56:02 [07:56:02.287]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testHAZHgj/0 with map size 1073741824
07:56:02 [07:56:02.289]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.311]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9bccd7194b64c825 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9bccd7194b64c825"}
07:56:02 [07:56:02.313]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9be85e6d954cfd4f epochNumber=0
{"provingJobId":"1:BASE_PARITY:9be85e6d954cfd4f"}
07:56:02 [07:56:02.314]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9bccd7194b64c825 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9bccd7194b64c825"}
07:56:02 [07:56:02.315]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9168fcf475eb117 epochNumber=0
{"provingJobId":"1:BASE_PARITY:f9168fcf475eb117"}
07:56:02 [07:56:02.336]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.339]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55f8da5bcd5d5639 epochNumber=2
{"provingJobId":"1:BASE_PARITY:55f8da5bcd5d5639"}
07:56:02 [07:56:02.341]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUiMciI/2 with map size 1073741824
07:56:02 [07:56:02.343]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.372]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2bb914994e99383 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2bb914994e99383"}
07:56:02 [07:56:02.375]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUiMciI/1 with map size 1073741824
07:56:02 [07:56:02.377]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.406]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.407]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce0ea2d91cbaf825 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce0ea2d91cbaf825"}
07:56:02 [07:56:02.409]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHNEvTm/1 with map size 1073741824
07:56:02 [07:56:02.412]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.433]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.434]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5d364ff558ce4233 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5d364ff558ce4233"}
07:56:02 [07:56:02.435]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testa0nhIL/1 with map size 1073741824
07:56:02 [07:56:02.437]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.457]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a3953720a5a5544 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a3953720a5a5544"}
07:56:02 [07:56:02.465]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dfc2089bcf909db1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:dfc2089bcf909db1"}
07:56:02 [07:56:02.466]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testa0nhIL/2 with map size 1073741824
07:56:02 [07:56:02.467]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.486]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dd6eba72b9432d0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:dd6eba72b9432d0f"}
07:56:02 [07:56:02.495]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.496]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b60bd976311dcdc6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b60bd976311dcdc6"}
07:56:02 [07:56:02.497]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYe1pTj/1 with map size 1073741824
07:56:02 [07:56:02.498]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8fc6d98ec5572d04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8fc6d98ec5572d04"}
07:56:02 [07:56:02.522]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:76e4dcefceb13de9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:76e4dcefceb13de9"}
07:56:02 [07:56:02.523]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testYe1pTj/2 with map size 1073741824
07:56:02 [07:56:02.527]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.546]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2f62d56dbe9133d epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2f62d56dbe9133d"}
07:56:02 [07:56:02.563]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.565]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:820a34da381e33d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:820a34da381e33d6"}
07:56:02 [07:56:02.565]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZJ5bRa/1 with map size 1073741824
07:56:02 [07:56:02.569]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.595]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:59dec76cf2a57f07 epochNumber=1
{"provingJobId":"1:BASE_PARITY:59dec76cf2a57f07"}
07:56:02 [07:56:02.603]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2adda9c2b568713f epochNumber=2
{"provingJobId":"1:BASE_PARITY:2adda9c2b568713f"}
07:56:02 [07:56:02.604]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testZJ5bRa/2 with map size 1073741824
07:56:02 [07:56:02.610]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.634]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f143da6525afdac7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f143da6525afdac7"}
07:56:02 [07:56:02.650]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.651]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a627e72739176c7f epochNumber=1
{"provingJobId":"1:BASE_PARITY:a627e72739176c7f"}
07:56:02 [07:56:02.652]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRODjPT/1 with map size 1073741824
07:56:02 [07:56:02.656]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.680]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6865e337a5debf76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6865e337a5debf76"}
07:56:02 [07:56:02.692]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a627e72739176c7f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a627e72739176c7f"}
07:56:02 [07:56:02.706]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.707]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:56aa3875e6ae7b36 epochNumber=1
{"provingJobId":"1:BASE_PARITY:56aa3875e6ae7b36"}
07:56:02 [07:56:02.710]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbXOgjf/1 with map size 1073741824
07:56:02 [07:56:02.713]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.734]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1dba73dc47986d9b epochNumber=1
{"provingJobId":"1:BASE_PARITY:1dba73dc47986d9b"}
07:56:02 [07:56:02.743]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:56aa3875e6ae7b36 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:56aa3875e6ae7b36"}
07:56:02 [07:56:02.756]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.758]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c721f3aef34a839 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1c721f3aef34a839"}
07:56:02 [07:56:02.758]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQ1uwSy/1 with map size 1073741824
07:56:02 [07:56:02.762]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.791]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8a9750c157684a18 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8a9750c157684a18"}
07:56:02 [07:56:02.798]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:1c721f3aef34a839 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:1c721f3aef34a839"}
07:56:02 [07:56:02.804]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.805]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:99a2842d66f0aea6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:99a2842d66f0aea6"}
07:56:02 [07:56:02.806]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtma6wc/1 with map size 1073741824
07:56:02 [07:56:02.811]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.836]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:99a2842d66f0aea6
{"provingJobId":"1:BASE_PARITY:99a2842d66f0aea6"}
07:56:02 [07:56:02.837]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:99a2842d66f0aea6 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:99a2842d66f0aea6"}
07:56:02 [07:56:02.845]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cd8aa91c379dae41 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cd8aa91c379dae41"}
07:56:02 [07:56:02.853]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:99a2842d66f0aea6 has already been completed
{"provingJobId":"1:BASE_PARITY:99a2842d66f0aea6"}
07:56:02 [07:56:02.863]
INFO:
prover-client:proving-broker Proving Broker started
07:56:02 [07:56:02.864]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:18e0ac365583d037 epochNumber=1
{"provingJobId":"1:BASE_PARITY:18e0ac365583d037"}
07:56:02 [07:56:02.864]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBHshYV/1 with map size 1073741824
07:56:02 [07:56:02.869]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:02 [07:56:02.897]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4101782f01759cb epochNumber=2
{"provingJobId":"1:BASE_PARITY:a4101782f01759cb"}
07:56:02 [07:56:02.900]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBHshYV/2 with map size 1073741824
07:56:02 [07:56:02.905]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:03 [07:56:03.983]
INFO:
prover-client:proving-broker Proving Broker started
07:56:03 [07:56:03.991]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:18e0ac365583d037 settled=false
{"provingJobId":"1:BASE_PARITY:18e0ac365583d037","status":"pending"}
07:56:03 [07:56:03.993]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a4101782f01759cb settled=false
{"provingJobId":"1:BASE_PARITY:a4101782f01759cb","status":"pending"}
07:56:03 [07:56:03.994]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:18e0ac365583d037 type=BASE_PARITY startedAt=1749801362929 older agent has taken job
{"provingJobId":"1:BASE_PARITY:18e0ac365583d037"}
07:56:03 [07:56:03.995]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:18e0ac365583d037 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:18e0ac365583d037"}
07:56:04 [07:56:04.002]
INFO:
prover-client:proving-broker Proving Broker started
07:56:04 [07:56:04.003]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c2a6114828f35fc2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c2a6114828f35fc2"}
07:56:04 [07:56:04.004]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtO4rk0/1 with map size 1073741824
07:56:04 [07:56:04.007]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:04 [07:56:04.029]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:991185ec5b9d288a epochNumber=2
{"provingJobId":"1:BASE_PARITY:991185ec5b9d288a"}
07:56:04 [07:56:04.030]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtO4rk0/2 with map size 1073741824
07:56:04 [07:56:04.031]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:05 [07:56:05.050]
INFO:
prover-client:proving-broker Proving Broker started
07:56:05 [07:56:05.052]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c2a6114828f35fc2 settled=false
{"provingJobId":"1:BASE_PARITY:c2a6114828f35fc2","status":"pending"}
07:56:05 [07:56:05.054]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:991185ec5b9d288a settled=false
{"provingJobId":"1:BASE_PARITY:991185ec5b9d288a","status":"pending"}
07:56:05 [07:56:05.055]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c2a6114828f35fc2 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:c2a6114828f35fc2"}
07:56:05 [07:56:05.059]
INFO:
prover-client:proving-broker Proving Broker started
07:56:05 [07:56:05.060]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:413cdbb51f756b5c epochNumber=1
{"provingJobId":"1:BASE_PARITY:413cdbb51f756b5c"}
07:56:05 [07:56:05.064]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEFU23H/1 with map size 1073741824
07:56:05 [07:56:05.067]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:05 [07:56:05.087]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a89fd032d04a3075 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a89fd032d04a3075"}
07:56:05 [07:56:05.088]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testEFU23H/2 with map size 1073741824
07:56:05 [07:56:05.091]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.111]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.113]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:413cdbb51f756b5c settled=false
{"provingJobId":"1:BASE_PARITY:413cdbb51f756b5c","status":"pending"}
07:56:06 [07:56:06.115]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a89fd032d04a3075 settled=false
{"provingJobId":"1:BASE_PARITY:a89fd032d04a3075","status":"pending"}
07:56:06 [07:56:06.116]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:413cdbb51f756b5c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:413cdbb51f756b5c"}
07:56:06 [07:56:06.116]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:413cdbb51f756b5c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:413cdbb51f756b5c"}
07:56:06 [07:56:06.128]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.129]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c896cc154bc1b9d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c896cc154bc1b9d6"}
07:56:06 [07:56:06.130]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testURka9o/1 with map size 1073741824
07:56:06 [07:56:06.132]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.149]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60d20c8f18f84acf epochNumber=2
{"provingJobId":"1:BASE_PARITY:60d20c8f18f84acf"}
07:56:06 [07:56:06.150]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testURka9o/2 with map size 1073741824
07:56:06 [07:56:06.152]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.171]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c896cc154bc1b9d6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c896cc154bc1b9d6"}
07:56:06 [07:56:06.177]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:60d20c8f18f84acf type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:60d20c8f18f84acf"}
07:56:06 [07:56:06.190]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.191]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f9e9bfa4b2e0ebde epochNumber=1
{"provingJobId":"1:BASE_PARITY:f9e9bfa4b2e0ebde"}
07:56:06 [07:56:06.191]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testY5VzUZ/1 with map size 1073741824
07:56:06 [07:56:06.193]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.212]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5afc26d55a190b89 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5afc26d55a190b89"}
07:56:06 [07:56:06.214]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testY5VzUZ/2 with map size 1073741824
07:56:06 [07:56:06.217]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.233]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f9e9bfa4b2e0ebde type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f9e9bfa4b2e0ebde"}
07:56:06 [07:56:06.234]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f9e9bfa4b2e0ebde type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f9e9bfa4b2e0ebde"}
07:56:06 [07:56:06.241]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5afc26d55a190b89 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:5afc26d55a190b89"}
07:56:06 [07:56:06.254]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.256]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8e323b385de6731b err=test error
{"provingJoId":"1:BASE_PARITY:8e323b385de6731b"}
07:56:06 [07:56:06.259]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.259]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ea54e396e0054360 not found
{"provingJobId":"1:BASE_PARITY:ea54e396e0054360"}
07:56:06 [07:56:06.262]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.262]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3055bf8bb01ff7d4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3055bf8bb01ff7d4"}
07:56:06 [07:56:06.263]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2oKYeZ/1 with map size 1073741824
07:56:06 [07:56:06.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.298]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.299]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0ae5669f9ee6a812 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0ae5669f9ee6a812"}
07:56:06 [07:56:06.305]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7JCFf5/1 with map size 1073741824
07:56:06 [07:56:06.309]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.451]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0ae5669f9ee6a812 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:0ae5669f9ee6a812"}
07:56:06 [07:56:06.460]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.461]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1abf115c2be20cfe epochNumber=1
{"provingJobId":"1:BASE_PARITY:1abf115c2be20cfe"}
07:56:06 [07:56:06.462]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testflR5Wm/1 with map size 1073741824
07:56:06 [07:56:06.466]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.614]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1abf115c2be20cfe timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:1abf115c2be20cfe"}
07:56:06 [07:56:06.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:179d4f65fe6f9c32 epochNumber=10
{"provingJobId":"1:BASE_PARITY:179d4f65fe6f9c32"}
07:56:06 [07:56:06.617]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testflR5Wm/10 with map size 1073741824
07:56:06 [07:56:06.620]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.640]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:06 [07:56:06.641]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
07:56:06 [07:56:06.767]
INFO:
prover-client:proving-broker Proving Broker started
07:56:06 [07:56:06.768]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b50baa9182968215 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b50baa9182968215"}
07:56:06 [07:56:06.768]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBCmMZb/1 with map size 1073741824
07:56:06 [07:56:06.772]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:06 [07:56:06.947]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b50baa9182968215 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b50baa9182968215"}
07:56:07 [07:56:07.001]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.002]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:42793e68219f9f97 epochNumber=1
{"provingJobId":"1:BASE_PARITY:42793e68219f9f97"}
07:56:07 [07:56:07.003]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDcXToA/1 with map size 1073741824
07:56:07 [07:56:07.007]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.026]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:42793e68219f9f97 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:42793e68219f9f97"}
07:56:07 [07:56:07.032]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.033]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cadb1a6b0bddf017 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cadb1a6b0bddf017"}
07:56:07 [07:56:07.034]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1lVIX6/1 with map size 1073741824
07:56:07 [07:56:07.037]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.059]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:cadb1a6b0bddf017 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:cadb1a6b0bddf017"}
07:56:07 [07:56:07.060]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cadb1a6b0bddf017 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:cadb1a6b0bddf017"}
07:56:07 [07:56:07.074]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.075]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3f42b312e4922574 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3f42b312e4922574"}
07:56:07 [07:56:07.076]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8QYXYZ/1 with map size 1073741824
07:56:07 [07:56:07.080]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.106]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3f42b312e4922574 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3f42b312e4922574"}
07:56:07 [07:56:07.116]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.117]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7642d2697c7e8132 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7642d2697c7e8132"}
07:56:07 [07:56:07.117]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDWfOM8/1 with map size 1073741824
07:56:07 [07:56:07.121]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.140]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:7642d2697c7e8132 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:7642d2697c7e8132"}
07:56:07 [07:56:07.141]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3830bef413ddb490 epochNumber=3
{"provingJobId":"1:BASE_PARITY:3830bef413ddb490"}
07:56:07 [07:56:07.143]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testDWfOM8/3 with map size 1073741824
07:56:07 [07:56:07.148]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.194]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:07 [07:56:07.195]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
07:56:07 [07:56:07.195]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7642d2697c7e8132 err=test error
{"provingJoId":"1:BASE_PARITY:7642d2697c7e8132"}
07:56:07 [07:56:07.202]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVE6L5h/1 with map size 1073741824
07:56:07 [07:56:07.207]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.231]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVE6L5h/2 with map size 1073741824
07:56:07 [07:56:07.234]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.254]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.256]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8f1ba0819b0b385f settled=false
{"provingJobId":"1:BASE_PARITY:8f1ba0819b0b385f","status":"pending"}
07:56:07 [07:56:07.258]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7ec00bf5b97e1c7d settled=false
{"provingJobId":"1:BASE_PARITY:7ec00bf5b97e1c7d","status":"pending"}
07:56:07 [07:56:07.268]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZrLhZ3/1 with map size 1073741824
07:56:07 [07:56:07.275]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.301]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testZrLhZ3/2 with map size 1073741824
07:56:07 [07:56:07.306]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.341]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.345]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7b6e0fe5258c7501 settled=true
{"provingJobId":"1:BASE_PARITY:7b6e0fe5258c7501","status":"fulfilled"}
07:56:07 [07:56:07.347]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:81c47a2b2da4bf5c settled=true
{"provingJobId":"2:BASE_PARITY:81c47a2b2da4bf5c","status":"fulfilled"}
07:56:07 [07:56:07.353]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testizwI1w/1 with map size 1073741824
07:56:07 [07:56:07.359]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.398]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testizwI1w/2 with map size 1073741824
07:56:07 [07:56:07.408]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.431]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.436]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6b0f472b52e489e7 settled=true
{"provingJobId":"1:BASE_PARITY:6b0f472b52e489e7","status":"fulfilled"}
07:56:07 [07:56:07.442]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e6c18878758e9f51 settled=false
{"provingJobId":"1:BASE_PARITY:e6c18878758e9f51","status":"pending"}
07:56:07 [07:56:07.453]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.455]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b6e4ce54473301af epochNumber=1
{"provingJobId":"1:BASE_PARITY:b6e4ce54473301af"}
07:56:07 [07:56:07.456]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgR2Dfz/1 with map size 1073741824
07:56:07 [07:56:07.460]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.495]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.496]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:745bd5c496cf3ffd epochNumber=1
{"provingJobId":"1:BASE_PARITY:745bd5c496cf3ffd"}
07:56:07 [07:56:07.497]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:745bd5c496cf3ffd: Error: db error: Error: db error
07:56:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:745bd5c496cf3ffd"}
07:56:07 [07:56:07.507]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.508]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5f5e8fc11abea8ac epochNumber=1
{"provingJobId":"1:BASE_PARITY:5f5e8fc11abea8ac"}
07:56:07 [07:56:07.509]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGbcJvo/1 with map size 1073741824
07:56:07 [07:56:07.512]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.529]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5f5e8fc11abea8ac type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5f5e8fc11abea8ac"}
07:56:07 [07:56:07.530]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5f5e8fc11abea8ac type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5f5e8fc11abea8ac"}
07:56:07 [07:56:07.541]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.542]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ec256f243fdc58a epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ec256f243fdc58a"}
07:56:07 [07:56:07.543]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl4ZgQw/1 with map size 1073741824
07:56:07 [07:56:07.544]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.565]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5ec256f243fdc58a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5ec256f243fdc58a"}
07:56:07 [07:56:07.565]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5ec256f243fdc58a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5ec256f243fdc58a"}
07:56:07 [07:56:07.566]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:5ec256f243fdc58a: Error: db error
07:56:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:5ec256f243fdc58a"}
07:56:07 [07:56:07.578]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.579]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c1d6d7793e84e42 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4c1d6d7793e84e42"}
07:56:07 [07:56:07.580]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6DJrIo/1 with map size 1073741824
07:56:07 [07:56:07.586]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.608]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4c1d6d7793e84e42 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4c1d6d7793e84e42"}
07:56:07 [07:56:07.608]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4c1d6d7793e84e42 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4c1d6d7793e84e42"}
07:56:07 [07:56:07.623]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.624]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3cb2faf7a60526d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3cb2faf7a60526d7"}
07:56:07 [07:56:07.625]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDxlZDY/1 with map size 1073741824
07:56:07 [07:56:07.630]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.653]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3cb2faf7a60526d7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3cb2faf7a60526d7"}
07:56:07 [07:56:07.654]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3cb2faf7a60526d7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3cb2faf7a60526d7"}
07:56:07 [07:56:07.655]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:3cb2faf7a60526d7 jobErr=test error: Error: db error
07:56:07 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:3cb2faf7a60526d7"}
07:56:07 [07:56:07.660]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.661]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a2f25ff85a00a836 not found
{"provingJobId":"1:BASE_PARITY:a2f25ff85a00a836"}
07:56:07 [07:56:07.666]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.667]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d90f55b7363d3b63 err=test error
{"provingJoId":"1:BASE_PARITY:d90f55b7363d3b63"}
07:56:07 [07:56:07.684]
INFO:
prover-client:proving-broker Proving Broker started
07:56:07 [07:56:07.695]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:909c1c7a1cd2baf6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:909c1c7a1cd2baf6"}
07:56:07 [07:56:07.697]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testC6rxZE/1 with map size 1073741824
07:56:07 [07:56:07.699]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.722]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:909c1c7a1cd2baf6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:909c1c7a1cd2baf6"}
07:56:07 [07:56:07.723]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:909c1c7a1cd2baf6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:909c1c7a1cd2baf6"}
07:56:07 [07:56:07.743]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:de4e1278217c8868 epochNumber=2
{"provingJobId":"2:BASE_PARITY:de4e1278217c8868"}
07:56:07 [07:56:07.745]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testC6rxZE/2 with map size 1073741824
07:56:07 [07:56:07.751]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.789]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:de4e1278217c8868 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:de4e1278217c8868"}
07:56:07 [07:56:07.790]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:de4e1278217c8868 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:de4e1278217c8868"}
07:56:07 [07:56:07.857]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:80178bfe175315ce epochNumber=3
{"provingJobId":"3:BASE_PARITY:80178bfe175315ce"}
07:56:07 [07:56:07.859]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testC6rxZE/3 with map size 1073741824
07:56:07 [07:56:07.862]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.889]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:07 [07:56:07.890]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
07:56:07 [07:56:07.949]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:efd42c96fa59b3fe epochNumber=4
{"provingJobId":"4:BASE_PARITY:efd42c96fa59b3fe"}
07:56:07 [07:56:07.950]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testC6rxZE/4 with map size 1073741824
07:56:07 [07:56:07.954]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:07 [07:56:07.974]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:07 [07:56:07.975]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
07:56:08 [07:56:08.036]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:0303aea5f31f5131 epochNumber=5
{"provingJobId":"5:BASE_PARITY:0303aea5f31f5131"}
07:56:08 [07:56:08.037]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testC6rxZE/5 with map size 1073741824
07:56:08 [07:56:08.040]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
07:56:08 [07:56:08.079]
INFO:
prover-client:proving-broker Cleaned up jobs=1
07:56:08 [07:56:08.080]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
07:56:08
PASS src/proving_broker/proving_broker.test.ts (
13.352 s)
07:56:08 ProvingBroker
07:56:08 Producer API
07:56:08
✓ refuses stale jobs (53 ms)
07:56:08
✓ enqueues jobs (6 ms)
07:56:08
✓ ignores duplicate jobs (5 ms)
07:56:08
✓ reports correct status when enqueuing repeat jobs (5 ms)
07:56:08
✓ throws an error in case of duplicate job IDs (29 ms)
07:56:08
✓ returns not-found status for non-existing jobs (2 ms)
07:56:08
✓ cancels jobs in queue (4 ms)
07:56:08
✓ cancels jobs in-progress (2 ms)
07:56:08
✓ returns job result if successful (2 ms)
07:56:08
✓ returns job error if failed (3 ms)
07:56:08
✓ correctly returns job status for concurrent writes (2 ms)
07:56:08
✓ refuses stale jobs (45 ms)
07:56:08
✓ enqueues jobs (35 ms)
07:56:08
✓ ignores duplicate jobs (37 ms)
07:56:08
✓ reports correct status when enqueuing repeat jobs (37 ms)
07:56:08
✓ throws an error in case of duplicate job IDs (32 ms)
07:56:08
✓ returns not-found status for non-existing jobs (3 ms)
07:56:08
✓ cancels jobs in queue (46 ms)
07:56:08
✓ cancels jobs in-progress (39 ms)
07:56:08
✓ returns job result if successful (35 ms)
07:56:08
✓ returns job error if failed (34 ms)
07:56:08
✓ correctly returns job status for concurrent writes (52 ms)
07:56:08 Consumer API
07:56:08
✓ returns undefined if no jobs are available (1 ms)
07:56:08
✓ returns jobs in priority order (1 ms)
07:56:08
✓ returns undefined if no jobs are available for the given allowList (1 ms)
07:56:08
✓ returns a job if it is in the allowList (2 ms)
07:56:08
✓ returns the most important job if it is in the allowList (3 ms)
07:56:08
✓ returns any job if filter is empty (2 ms)
07:56:08
✓ returns a new job when reporting job success (3 ms)
07:56:08
✓ returns a new job when reporting permanent error (3 ms)
07:56:08
✓ returns a new job when reporting retry-able error (6 ms)
07:56:08
✓ returns a new job when reporting progress if current one is cancelled (5 ms)
07:56:08
✓ returns a new job if job is already in progress elsewhere (1057 ms)
07:56:08
✓ avoids sending the same job to a new agent after a restart (1005 ms)
07:56:08
✓ avoids sending a completed job to a new agent after a restart (1005 ms)
07:56:08
✓ tracks job result if in progress (3 ms)
07:56:08
✓ tracks job result even if job is in queue (3 ms)
07:56:08
✓ ignores reported job error if unknown job (1 ms)
07:56:08
✓ ignores job result if unknown job (1 ms)
07:56:08
✓ returns undefined if no jobs are available (3 ms)
07:56:08
✓ returns jobs in priority order (68 ms)
07:56:08
✓ returns undefined if no jobs are available for the given allowList (27 ms)
07:56:08
✓ returns a job if it is in the allowList (61 ms)
07:56:08
✓ returns the most important job if it is in the allowList (62 ms)
07:56:08
✓ returns any job if filter is empty (90 ms)
07:56:08
✓ returns a new job when reporting job success (56 ms)
07:56:08
✓ returns a new job when reporting permanent error (49 ms)
07:56:08
✓ returns a new job when reporting retry-able error (48 ms)
07:56:08
✓ returns a new job when reporting progress if current one is cancelled (58 ms)
07:56:08
✓ returns a new job if job is already in progress elsewhere (1139 ms)
07:56:08
✓ avoids sending the same job to a new agent after a restart (1057 ms)
07:56:08
✓ avoids sending a completed job to a new agent after a restart (1067 ms)
07:56:08
✓ tracks job result if in progress (62 ms)
07:56:08
✓ tracks job result even if job is in queue (62 ms)
07:56:08
✓ ignores reported job error if unknown job (6 ms)
07:56:08
✓ ignores job result if unknown job (3 ms)
07:56:08 Timeouts
07:56:08
✓ tracks in progress jobs (1 ms)
07:56:08
✓ re-enqueues jobs that time out (108 ms)
07:56:08
✓ cancel stale jobs that time out (237 ms)
07:56:08
✓ keeps the jobs in progress while it is alive (203 ms)
07:56:08
✓ tracks in progress jobs (36 ms)
07:56:08
✓ re-enqueues jobs that time out (159 ms)
07:56:08
✓ cancel stale jobs that time out (309 ms)
07:56:08
✓ keeps the jobs in progress while it is alive (234 ms)
07:56:08 Retries
07:56:08
✓ retries jobs (3 ms)
07:56:08
✓ retries up to a maximum number of times (5 ms)
07:56:08
✓ passing retry=false does not retry (3 ms)
07:56:08
✓ does not retry if job is stale (29 ms)
07:56:08
✓ retries jobs (29 ms)
07:56:08
✓ retries up to a maximum number of times (39 ms)
07:56:08
✓ passing retry=false does not retry (44 ms)
07:56:08
✓ does not retry if job is stale (83 ms)
07:56:08 Database management
07:56:08
✓ re-enqueues proof requests on start (3 ms)
07:56:08
✓ restores proof results on start (4 ms)
07:56:08
✓ only re-enqueues unfinished jobs (2 ms)
07:56:08
✓ saves job when enqueued (3 ms)
07:56:08
✓ does not retain job if database fails to save (2 ms)
07:56:08
✓ saves job result (1 ms)
07:56:08
✓ saves result even if database fails to save (1 ms)
07:56:08
✓ saves job error (2 ms)
07:56:08
✓ saves job error even if database fails to save (2 ms)
07:56:08
✓ does not save job result if job is unknown (1 ms)
07:56:08
✓ does not save job error if job is unknown (2 ms)
07:56:08
✓ cleans up old jobs periodically (264 ms)
07:56:08
✓ re-enqueues proof requests on start (65 ms)
07:56:08
✓ restores proof results on start (87 ms)
07:56:08
✓ only re-enqueues unfinished jobs (99 ms)
07:56:08
✓ saves job when enqueued (41 ms)
07:56:08
✓ does not retain job if database fails to save (8 ms)
07:56:08
✓ saves job result (38 ms)
07:56:08
✓ saves result even if database fails to save (30 ms)
07:56:08
✓ saves job error (51 ms)
07:56:08
✓ saves job error even if database fails to save (37 ms)
07:56:08
✓ does not save job result if job is unknown (4 ms)
07:56:08
✓ does not save job error if job is unknown (6 ms)
07:56:08
✓ cleans up old jobs periodically (443 ms)
07:56:08
07:56:08
Test Suites: 1 passed, 1 total
07:56:08
Tests: 96 passed, 96 total
07:56:08
Snapshots: 0 total
07:56:08
Time: 13.439 s
07:56:08
Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i
.
07:56:08
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?