Parent Log:
http://ci.aztec-labs.com/16dd5eb533779d2c
Command: 0b621446113df0ea yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/4129b6ec31c9c754859dfd38c25a0f14947e8c67
Env: REF_NAME=gh-readonly-queue/next/pr-15216-a16ffcb44336b117d56ec8733e0b8bbfc787edf7 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 23 14:56:08 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15216_amd64_x3-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
14:56:12 [14:56:12.846]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.853]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce34c125557a1d2d epochNumber=42
{"provingJobId":"1:BASE_PARITY:ce34c125557a1d2d"}
14:56:12 [14:56:12.857]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:9c6c52273df8ae4b epochNumber=1
{"provingJobId":"1:BASE_PARITY:9c6c52273df8ae4b"}
14:56:12 [14:56:12.903]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.904]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f24991e6dbda81ec epochNumber=1
{"provingJobId":"1:BASE_PARITY:f24991e6dbda81ec"}
14:56:12 [14:56:12.905]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8ff7745a9e2e5e26 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8ff7745a9e2e5e26"}
14:56:12 [14:56:12.907]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.908]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6fccd8006e54b2f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6fccd8006e54b2f8"}
14:56:12 [14:56:12.909]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6fccd8006e54b2f8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6fccd8006e54b2f8"}
14:56:12 [14:56:12.911]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.912]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f09ab1da2f0f9327 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f09ab1da2f0f9327"}
14:56:12 [14:56:12.913]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f09ab1da2f0f9327 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f09ab1da2f0f9327"}
14:56:12 [14:56:12.914]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f09ab1da2f0f9327 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f09ab1da2f0f9327"}
14:56:12 [14:56:12.915]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f09ab1da2f0f9327 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f09ab1da2f0f9327"}
14:56:12 [14:56:12.916]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.917]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f3d93fce17d911e epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f3d93fce17d911e"}
14:56:12 [14:56:12.942]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.943]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.943]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec3e4e4c3f39c4c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ec3e4e4c3f39c4c5"}
14:56:12 [14:56:12.944]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ec3e4e4c3f39c4c5
{"provingJobId":"1:BASE_PARITY:ec3e4e4c3f39c4c5"}
14:56:12 [14:56:12.944]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ec3e4e4c3f39c4c5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ec3e4e4c3f39c4c5"}
14:56:12 [14:56:12.944]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ec3e4e4c3f39c4c5 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:ec3e4e4c3f39c4c5"}
14:56:12 [14:56:12.945]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.946]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ea029a858353b0c epochNumber=1
{"provingJobId":"1:BASE_PARITY:3ea029a858353b0c"}
14:56:12 [14:56:12.946]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3ea029a858353b0c
{"provingJobId":"1:BASE_PARITY:3ea029a858353b0c"}
14:56:12 [14:56:12.947]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3ea029a858353b0c type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3ea029a858353b0c"}
14:56:12 [14:56:12.948]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.949]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b02fb60275693d78 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b02fb60275693d78"}
14:56:12 [14:56:12.949]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b02fb60275693d78 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b02fb60275693d78"}
14:56:12 [14:56:12.949]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b02fb60275693d78 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b02fb60275693d78"}
14:56:12 [14:56:12.950]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.951]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a900d918aafd878 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a900d918aafd878"}
14:56:12 [14:56:12.951]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0a900d918aafd878 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0a900d918aafd878"}
14:56:12 [14:56:12.951]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0a900d918aafd878 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0a900d918aafd878"}
14:56:12 [14:56:12.952]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.952]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:016591a388fa972d epochNumber=0
{"provingJobId":"1:BASE_PARITY:016591a388fa972d"}
14:56:12 [14:56:12.952]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:016591a388fa972d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:016591a388fa972d"}
14:56:12 [14:56:12.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b8fc91d94cb1a7e epochNumber=0
{"provingJobId":"1:BASE_PARITY:2b8fc91d94cb1a7e"}
14:56:12 [14:56:12.953]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:016591a388fa972d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:016591a388fa972d"}
14:56:12 [14:56:12.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:906735b01a0563ef epochNumber=0
{"provingJobId":"1:BASE_PARITY:906735b01a0563ef"}
14:56:12 [14:56:12.954]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.955]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c7a3917646350d43 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c7a3917646350d43"}
14:56:12 [14:56:12.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c0136ac7cc3665e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c0136ac7cc3665e3"}
14:56:12 [14:56:12.957]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.965]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e7a799d4e2793e38 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e7a799d4e2793e38"}
14:56:12 [14:56:12.967]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.967]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9045e370b42cd48c epochNumber=1
{"provingJobId":"1:BASE_PARITY:9045e370b42cd48c"}
14:56:12 [14:56:12.967]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e141aed96cfcce0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:e141aed96cfcce0f"}
14:56:12 [14:56:12.968]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:473c0240510f5c23 epochNumber=2
{"provingJobId":"1:BASE_PARITY:473c0240510f5c23"}
14:56:12 [14:56:12.968]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81c2078d70cc8d63 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81c2078d70cc8d63"}
14:56:12 [14:56:12.969]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d000b282be96ebf6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d000b282be96ebf6"}
14:56:12 [14:56:12.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:501244fdcd853481 epochNumber=1
{"provingJobId":"1:BASE_PARITY:501244fdcd853481"}
14:56:12 [14:56:12.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1417d7a1c87c9319 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1417d7a1c87c9319"}
14:56:12 [14:56:12.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b943173b38327e8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9b943173b38327e8"}
14:56:12 [14:56:12.971]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.971]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cf16246c014279f3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cf16246c014279f3"}
14:56:12 [14:56:12.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4dff03c00f355529 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4dff03c00f355529"}
14:56:12 [14:56:12.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:02b8f9878ff39e3e epochNumber=2
{"provingJobId":"1:BASE_PARITY:02b8f9878ff39e3e"}
14:56:12 [14:56:12.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe0675cad033bccc epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe0675cad033bccc"}
14:56:12 [14:56:12.973]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.973]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1e0a0afc6b12f481 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1e0a0afc6b12f481"}
14:56:12 [14:56:12.974]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a887cc9f70cbb9ad epochNumber=1
{"provingJobId":"1:BASE_PARITY:a887cc9f70cbb9ad"}
14:56:12 [14:56:12.974]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1e0a0afc6b12f481 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1e0a0afc6b12f481"}
14:56:12 [14:56:12.975]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.975]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ddbff51d3ea6438 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ddbff51d3ea6438"}
14:56:12 [14:56:12.976]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:99ad97f71e3e728e epochNumber=1
{"provingJobId":"1:BASE_PARITY:99ad97f71e3e728e"}
14:56:12 [14:56:12.976]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5ddbff51d3ea6438 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:5ddbff51d3ea6438"}
14:56:12 [14:56:12.977]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.977]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aa4ea251cc2e8992 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aa4ea251cc2e8992"}
14:56:12 [14:56:12.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3b62b3d7fb9d7e26 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3b62b3d7fb9d7e26"}
14:56:12 [14:56:12.978]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:aa4ea251cc2e8992 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:aa4ea251cc2e8992"}
14:56:12 [14:56:12.979]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.980]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ab3cbc1b23dc0351 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ab3cbc1b23dc0351"}
14:56:12 [14:56:12.980]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ab3cbc1b23dc0351
{"provingJobId":"1:BASE_PARITY:ab3cbc1b23dc0351"}
14:56:12 [14:56:12.980]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ab3cbc1b23dc0351 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:ab3cbc1b23dc0351"}
14:56:12 [14:56:12.985]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:16889a3de8f97fcc epochNumber=1
{"provingJobId":"1:BASE_PARITY:16889a3de8f97fcc"}
14:56:12 [14:56:12.985]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ab3cbc1b23dc0351 has already been completed
{"provingJobId":"1:BASE_PARITY:ab3cbc1b23dc0351"}
14:56:12 [14:56:12.986]
INFO:
prover-client:proving-broker Proving Broker started
14:56:12 [14:56:12.987]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c356f09c82b01932 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c356f09c82b01932"}
14:56:12 [14:56:12.987]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e2a879e798c9b1b epochNumber=2
{"provingJobId":"1:BASE_PARITY:9e2a879e798c9b1b"}
14:56:14 [14:56:14.040]
INFO:
prover-client:proving-broker Proving Broker started
14:56:14 [14:56:14.042]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c356f09c82b01932 settled=false
{"provingJobId":"1:BASE_PARITY:c356f09c82b01932","status":"pending"}
14:56:14 [14:56:14.043]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9e2a879e798c9b1b settled=false
{"provingJobId":"1:BASE_PARITY:9e2a879e798c9b1b","status":"pending"}
14:56:14 [14:56:14.043]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c356f09c82b01932 type=BASE_PARITY startedAt=1750690572987 older agent has taken job
{"provingJobId":"1:BASE_PARITY:c356f09c82b01932"}
14:56:14 [14:56:14.044]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c356f09c82b01932 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:c356f09c82b01932"}
14:56:14 [14:56:14.045]
INFO:
prover-client:proving-broker Proving Broker started
14:56:14 [14:56:14.046]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c9950d8bbde0c4d epochNumber=1
{"provingJobId":"1:BASE_PARITY:7c9950d8bbde0c4d"}
14:56:14 [14:56:14.046]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:937fb11893b15471 epochNumber=2
{"provingJobId":"1:BASE_PARITY:937fb11893b15471"}
14:56:15 [14:56:15.046]
INFO:
prover-client:proving-broker Proving Broker started
14:56:15 [14:56:15.047]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7c9950d8bbde0c4d settled=false
{"provingJobId":"1:BASE_PARITY:7c9950d8bbde0c4d","status":"pending"}
14:56:15 [14:56:15.047]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:937fb11893b15471 settled=false
{"provingJobId":"1:BASE_PARITY:937fb11893b15471","status":"pending"}
14:56:15 [14:56:15.048]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7c9950d8bbde0c4d type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:7c9950d8bbde0c4d"}
14:56:15 [14:56:15.049]
INFO:
prover-client:proving-broker Proving Broker started
14:56:15 [14:56:15.050]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:86cbba756f300825 epochNumber=1
{"provingJobId":"1:BASE_PARITY:86cbba756f300825"}
14:56:15 [14:56:15.050]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:901dbe9dd1a06834 epochNumber=2
{"provingJobId":"1:BASE_PARITY:901dbe9dd1a06834"}
14:56:16 [14:56:16.051]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.051]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:86cbba756f300825 settled=false
{"provingJobId":"1:BASE_PARITY:86cbba756f300825","status":"pending"}
14:56:16 [14:56:16.052]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:901dbe9dd1a06834 settled=false
{"provingJobId":"1:BASE_PARITY:901dbe9dd1a06834","status":"pending"}
14:56:16 [14:56:16.052]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:86cbba756f300825 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:86cbba756f300825"}
14:56:16 [14:56:16.052]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:86cbba756f300825 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:86cbba756f300825"}
14:56:16 [14:56:16.054]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.054]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ed6569616b0ef6b2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ed6569616b0ef6b2"}
14:56:16 [14:56:16.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0e84ccc9f73bdf7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d0e84ccc9f73bdf7"}
14:56:16 [14:56:16.055]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ed6569616b0ef6b2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ed6569616b0ef6b2"}
14:56:16 [14:56:16.055]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d0e84ccc9f73bdf7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d0e84ccc9f73bdf7"}
14:56:16 [14:56:16.057]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4925b61ceb2222b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4925b61ceb2222b0"}
14:56:16 [14:56:16.057]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7db1f4b53acc8af3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7db1f4b53acc8af3"}
14:56:16 [14:56:16.058]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4925b61ceb2222b0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4925b61ceb2222b0"}
14:56:16 [14:56:16.058]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4925b61ceb2222b0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4925b61ceb2222b0"}
14:56:16 [14:56:16.058]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7db1f4b53acc8af3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7db1f4b53acc8af3"}
14:56:16 [14:56:16.059]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.059]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:6480dd6398c31534 err=test error
{"provingJoId":"1:BASE_PARITY:6480dd6398c31534"}
14:56:16 [14:56:16.060]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.061]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ec56d8e4dae8a96a not found
{"provingJobId":"1:BASE_PARITY:ec56d8e4dae8a96a"}
14:56:16 [14:56:16.062]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.063]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9fa8fafd07989a6b epochNumber=1
{"provingJobId":"1:BASE_PARITY:9fa8fafd07989a6b"}
14:56:16 [14:56:16.064]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.064]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09a78293fbca3089 epochNumber=1
{"provingJobId":"1:BASE_PARITY:09a78293fbca3089"}
14:56:16 [14:56:16.166]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:09a78293fbca3089 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:09a78293fbca3089"}
14:56:16 [14:56:16.173]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:618fa233b5eab9fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:618fa233b5eab9fd"}
14:56:16 [14:56:16.275]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:618fa233b5eab9fd timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:618fa233b5eab9fd"}
14:56:16 [14:56:16.276]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b92d407e11909c70 epochNumber=10
{"provingJobId":"1:BASE_PARITY:b92d407e11909c70"}
14:56:16 [14:56:16.300]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:16 [14:56:16.404]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.405]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d2a122ec500069c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d2a122ec500069c2"}
14:56:16 [14:56:16.558]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d2a122ec500069c2 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:d2a122ec500069c2"}
14:56:16 [14:56:16.608]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.609]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:44d99991e5d4d3cb epochNumber=1
{"provingJobId":"1:BASE_PARITY:44d99991e5d4d3cb"}
14:56:16 [14:56:16.609]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:44d99991e5d4d3cb type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:44d99991e5d4d3cb"}
14:56:16 [14:56:16.611]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.611]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a65d35f856890ff epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a65d35f856890ff"}
14:56:16 [14:56:16.612]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3a65d35f856890ff type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3a65d35f856890ff"}
14:56:16 [14:56:16.612]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3a65d35f856890ff type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:3a65d35f856890ff"}
14:56:16 [14:56:16.613]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.614]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:79e3bdafe8555c15 epochNumber=1
{"provingJobId":"1:BASE_PARITY:79e3bdafe8555c15"}
14:56:16 [14:56:16.614]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:79e3bdafe8555c15 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:79e3bdafe8555c15"}
14:56:16 [14:56:16.615]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ddfe7d66f0461cda epochNumber=1
{"provingJobId":"1:BASE_PARITY:ddfe7d66f0461cda"}
14:56:16 [14:56:16.616]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ddfe7d66f0461cda type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:ddfe7d66f0461cda"}
14:56:16 [14:56:16.617]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b8bead21c39eace epochNumber=3
{"provingJobId":"1:BASE_PARITY:0b8bead21c39eace"}
14:56:16 [14:56:16.640]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:16 [14:56:16.642]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ddfe7d66f0461cda err=test error
{"provingJoId":"1:BASE_PARITY:ddfe7d66f0461cda"}
14:56:16 [14:56:16.644]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.644]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2165cf69255c29a6 settled=false
{"provingJobId":"1:BASE_PARITY:2165cf69255c29a6","status":"pending"}
14:56:16 [14:56:16.644]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:514a6dcaeb846ffc settled=false
{"provingJobId":"1:BASE_PARITY:514a6dcaeb846ffc","status":"pending"}
14:56:16 [14:56:16.646]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.647]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5c31927709cd9b3f settled=true
{"provingJobId":"1:BASE_PARITY:5c31927709cd9b3f","status":"fulfilled"}
14:56:16 [14:56:16.647]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:25036315869849d0 settled=true
{"provingJobId":"2:BASE_PARITY:25036315869849d0","status":"fulfilled"}
14:56:16 [14:56:16.648]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.649]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:727c7aa15d25ec02 settled=true
{"provingJobId":"1:BASE_PARITY:727c7aa15d25ec02","status":"fulfilled"}
14:56:16 [14:56:16.649]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7851345b294a4929 settled=false
{"provingJobId":"1:BASE_PARITY:7851345b294a4929","status":"pending"}
14:56:16 [14:56:16.650]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.651]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ed655fe8afe701e0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ed655fe8afe701e0"}
14:56:16 [14:56:16.652]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.653]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:085107e205df66d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:085107e205df66d9"}
14:56:16 [14:56:16.653]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:085107e205df66d9: Error: db error: Error: db error
14:56:16 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:085107e205df66d9"}
14:56:16 [14:56:16.655]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.655]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c33f80a3e103c002 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c33f80a3e103c002"}
14:56:16 [14:56:16.655]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c33f80a3e103c002 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c33f80a3e103c002"}
14:56:16 [14:56:16.655]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c33f80a3e103c002 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c33f80a3e103c002"}
14:56:16 [14:56:16.656]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.657]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b9c8b2f82dc8d067 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b9c8b2f82dc8d067"}
14:56:16 [14:56:16.657]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b9c8b2f82dc8d067 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b9c8b2f82dc8d067"}
14:56:16 [14:56:16.657]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b9c8b2f82dc8d067 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b9c8b2f82dc8d067"}
14:56:16 [14:56:16.657]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:b9c8b2f82dc8d067: Error: db error
14:56:16 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:b9c8b2f82dc8d067"}
14:56:16 [14:56:16.658]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.659]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d432f93fb58141df epochNumber=1
{"provingJobId":"1:BASE_PARITY:d432f93fb58141df"}
14:56:16 [14:56:16.659]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d432f93fb58141df type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d432f93fb58141df"}
14:56:16 [14:56:16.659]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d432f93fb58141df type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d432f93fb58141df"}
14:56:16 [14:56:16.660]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4aca115d6d194d8e epochNumber=1
{"provingJobId":"1:BASE_PARITY:4aca115d6d194d8e"}
14:56:16 [14:56:16.660]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4aca115d6d194d8e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4aca115d6d194d8e"}
14:56:16 [14:56:16.661]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4aca115d6d194d8e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4aca115d6d194d8e"}
14:56:16 [14:56:16.661]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:4aca115d6d194d8e jobErr=test error: Error: db error
14:56:16 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:4aca115d6d194d8e"}
14:56:16 [14:56:16.666]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.667]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:efa42edf0ef804c7 not found
{"provingJobId":"1:BASE_PARITY:efa42edf0ef804c7"}
14:56:16 [14:56:16.668]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.669]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c7a9f8638aad5e83 err=test error
{"provingJoId":"1:BASE_PARITY:c7a9f8638aad5e83"}
14:56:16 [14:56:16.670]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.684]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e5343c8f30c3adba epochNumber=1
{"provingJobId":"1:BASE_PARITY:e5343c8f30c3adba"}
14:56:16 [14:56:16.685]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e5343c8f30c3adba type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e5343c8f30c3adba"}
14:56:16 [14:56:16.685]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e5343c8f30c3adba type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e5343c8f30c3adba"}
14:56:16 [14:56:16.696]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:0ce77a1f85d72639 epochNumber=2
{"provingJobId":"2:BASE_PARITY:0ce77a1f85d72639"}
14:56:16 [14:56:16.696]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:0ce77a1f85d72639 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:0ce77a1f85d72639"}
14:56:16 [14:56:16.697]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:0ce77a1f85d72639 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:0ce77a1f85d72639"}
14:56:16 [14:56:16.758]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:969c73a9ce56630b epochNumber=3
{"provingJobId":"3:BASE_PARITY:969c73a9ce56630b"}
14:56:16 [14:56:16.772]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:16 [14:56:16.819]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:2f411e3c39e5f0c1 epochNumber=4
{"provingJobId":"4:BASE_PARITY:2f411e3c39e5f0c1"}
14:56:16 [14:56:16.823]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:16 [14:56:16.881]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:26f5f8920494fa32 epochNumber=5
{"provingJobId":"5:BASE_PARITY:26f5f8920494fa32"}
14:56:16 [14:56:16.899]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:16 [14:56:16.938]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f6a62ebb9cb4022 epochNumber=42
{"provingJobId":"1:BASE_PARITY:7f6a62ebb9cb4022"}
14:56:16 [14:56:16.940]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testMurQHx/42 with map size 1073741824
14:56:16 [14:56:16.948]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:16 [14:56:16.971]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:32b73e456f092239 epochNumber=1
{"provingJobId":"1:BASE_PARITY:32b73e456f092239"}
14:56:16 [14:56:16.978]
INFO:
prover-client:proving-broker Proving Broker started
14:56:16 [14:56:16.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:83af8c77695243b5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:83af8c77695243b5"}
14:56:16 [14:56:16.981]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testepqT6T/1 with map size 1073741824
14:56:16 [14:56:16.986]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.010]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c24d2d1ab4939982 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c24d2d1ab4939982"}
14:56:17 [14:56:17.022]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ceba577dde547b9e epochNumber=1
{"provingJobId":"1:BASE_PARITY:ceba577dde547b9e"}
14:56:17 [14:56:17.024]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9NyfhK/1 with map size 1073741824
14:56:17 [14:56:17.027]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.047]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ceba577dde547b9e epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ceba577dde547b9e"}
14:56:17 [14:56:17.051]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.051]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:decb4fd9dff63353 epochNumber=1
{"provingJobId":"1:BASE_PARITY:decb4fd9dff63353"}
14:56:17 [14:56:17.052]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYYXUau/1 with map size 1073741824
14:56:17 [14:56:17.054]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.074]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:decb4fd9dff63353 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:decb4fd9dff63353"}
14:56:17 [14:56:17.075]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:decb4fd9dff63353 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:decb4fd9dff63353"}
14:56:17 [14:56:17.084]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:decb4fd9dff63353 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:decb4fd9dff63353"}
14:56:17 [14:56:17.091]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.092]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:390f101333baef95 epochNumber=1
{"provingJobId":"1:BASE_PARITY:390f101333baef95"}
14:56:17 [14:56:17.093]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkquxL7/1 with map size 1073741824
14:56:17 [14:56:17.096]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.117]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.120]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.121]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:11017ce914293f14 epochNumber=1
{"provingJobId":"1:BASE_PARITY:11017ce914293f14"}
14:56:17 [14:56:17.123]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNq1pkh/1 with map size 1073741824
14:56:17 [14:56:17.125]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.145]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:11017ce914293f14
{"provingJobId":"1:BASE_PARITY:11017ce914293f14"}
14:56:17 [14:56:17.146]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:11017ce914293f14 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:11017ce914293f14"}
14:56:17 [14:56:17.146]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:11017ce914293f14 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:11017ce914293f14"}
14:56:17 [14:56:17.155]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.155]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:444b96d5473c3241 epochNumber=1
{"provingJobId":"1:BASE_PARITY:444b96d5473c3241"}
14:56:17 [14:56:17.156]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdMAf4x/1 with map size 1073741824
14:56:17 [14:56:17.158]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.175]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:444b96d5473c3241
{"provingJobId":"1:BASE_PARITY:444b96d5473c3241"}
14:56:17 [14:56:17.176]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:444b96d5473c3241 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:444b96d5473c3241"}
14:56:17 [14:56:17.193]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.195]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d699980f85d5e274 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d699980f85d5e274"}
14:56:17 [14:56:17.196]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testt2zxwQ/1 with map size 1073741824
14:56:17 [14:56:17.203]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.222]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d699980f85d5e274 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d699980f85d5e274"}
14:56:17 [14:56:17.223]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d699980f85d5e274 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d699980f85d5e274"}
14:56:17 [14:56:17.233]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.234]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5e3cbf3f700a4d5b epochNumber=1
{"provingJobId":"1:BASE_PARITY:5e3cbf3f700a4d5b"}
14:56:17 [14:56:17.236]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testG893M7/1 with map size 1073741824
14:56:17 [14:56:17.239]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.256]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5e3cbf3f700a4d5b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5e3cbf3f700a4d5b"}
14:56:17 [14:56:17.257]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5e3cbf3f700a4d5b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:5e3cbf3f700a4d5b"}
14:56:17 [14:56:17.265]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.265]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce58df6c885c673c epochNumber=0
{"provingJobId":"1:BASE_PARITY:ce58df6c885c673c"}
14:56:17 [14:56:17.266]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testN5LZ8v/0 with map size 1073741824
14:56:17 [14:56:17.268]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.294]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ce58df6c885c673c epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ce58df6c885c673c"}
14:56:17 [14:56:17.295]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ebbed310269f576 epochNumber=0
{"provingJobId":"1:BASE_PARITY:5ebbed310269f576"}
14:56:17 [14:56:17.296]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ce58df6c885c673c epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ce58df6c885c673c"}
14:56:17 [14:56:17.296]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5dcc9d2de6698b6e epochNumber=0
{"provingJobId":"1:BASE_PARITY:5dcc9d2de6698b6e"}
14:56:17 [14:56:17.313]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.316]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ec2e6ca5230887f epochNumber=2
{"provingJobId":"1:BASE_PARITY:7ec2e6ca5230887f"}
14:56:17 [14:56:17.317]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testC10veJ/2 with map size 1073741824
14:56:17 [14:56:17.319]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.336]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b5297ac4f9e571e epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b5297ac4f9e571e"}
14:56:17 [14:56:17.337]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testC10veJ/1 with map size 1073741824
14:56:17 [14:56:17.339]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.361]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.362]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9fc861711765081d epochNumber=1
{"provingJobId":"1:BASE_PARITY:9fc861711765081d"}
14:56:17 [14:56:17.362]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2dqzZ5/1 with map size 1073741824
14:56:17 [14:56:17.365]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.395]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.396]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b1e13e2c7ae71d2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0b1e13e2c7ae71d2"}
14:56:17 [14:56:17.398]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testU99SKM/1 with map size 1073741824
14:56:17 [14:56:17.403]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.425]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c512b521ac03bddb epochNumber=1
{"provingJobId":"1:BASE_PARITY:c512b521ac03bddb"}
14:56:17 [14:56:17.433]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5122fabbda90690a epochNumber=2
{"provingJobId":"1:BASE_PARITY:5122fabbda90690a"}
14:56:17 [14:56:17.434]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testU99SKM/2 with map size 1073741824
14:56:17 [14:56:17.439]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.461]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:42e339a8b21c7093 epochNumber=1
{"provingJobId":"1:BASE_PARITY:42e339a8b21c7093"}
14:56:17 [14:56:17.473]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.475]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6f5f332f2304872d epochNumber=1
{"provingJobId":"1:BASE_PARITY:6f5f332f2304872d"}
14:56:17 [14:56:17.477]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkT2cWo/1 with map size 1073741824
14:56:17 [14:56:17.488]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.507]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a7e208ac28cf003 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a7e208ac28cf003"}
14:56:17 [14:56:17.514]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f6f9bc0f867b1b7c epochNumber=2
{"provingJobId":"1:BASE_PARITY:f6f9bc0f867b1b7c"}
14:56:17 [14:56:17.515]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkT2cWo/2 with map size 1073741824
14:56:17 [14:56:17.517]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.536]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d945134e0cde1dc1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d945134e0cde1dc1"}
14:56:17 [14:56:17.548]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.550]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd51047870d30ddd epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd51047870d30ddd"}
14:56:17 [14:56:17.551]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8cl2AM/1 with map size 1073741824
14:56:17 [14:56:17.553]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.572]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4a093d0f03c49032 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4a093d0f03c49032"}
14:56:17 [14:56:17.581]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ccc5fbdbebcf2c1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9ccc5fbdbebcf2c1"}
14:56:17 [14:56:17.582]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8cl2AM/2 with map size 1073741824
14:56:17 [14:56:17.588]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.612]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:be32e7ec1ab04efc epochNumber=1
{"provingJobId":"1:BASE_PARITY:be32e7ec1ab04efc"}
14:56:17 [14:56:17.624]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.625]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:162c15254fd386ec epochNumber=1
{"provingJobId":"1:BASE_PARITY:162c15254fd386ec"}
14:56:17 [14:56:17.625]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwK2lhM/1 with map size 1073741824
14:56:17 [14:56:17.627]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e68fa7d45670ba70 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e68fa7d45670ba70"}
14:56:17 [14:56:17.659]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:162c15254fd386ec type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:162c15254fd386ec"}
14:56:17 [14:56:17.671]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.672]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25403c7a8af2bcb7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:25403c7a8af2bcb7"}
14:56:17 [14:56:17.672]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0v7i6k/1 with map size 1073741824
14:56:17 [14:56:17.675]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.699]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0829d97f9665f67c epochNumber=1
{"provingJobId":"1:BASE_PARITY:0829d97f9665f67c"}
14:56:17 [14:56:17.706]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:25403c7a8af2bcb7 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:25403c7a8af2bcb7"}
14:56:17 [14:56:17.723]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.724]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d17c2837735a5eb epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d17c2837735a5eb"}
14:56:17 [14:56:17.725]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWstOBz/1 with map size 1073741824
14:56:17 [14:56:17.729]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.748]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3458eed92269732 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b3458eed92269732"}
14:56:17 [14:56:17.753]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2d17c2837735a5eb type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:2d17c2837735a5eb"}
14:56:17 [14:56:17.757]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.758]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:210bc000d8b18dd8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:210bc000d8b18dd8"}
14:56:17 [14:56:17.759]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl9lH3w/1 with map size 1073741824
14:56:17 [14:56:17.761]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.777]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:210bc000d8b18dd8
{"provingJobId":"1:BASE_PARITY:210bc000d8b18dd8"}
14:56:17 [14:56:17.777]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:210bc000d8b18dd8 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:210bc000d8b18dd8"}
14:56:17 [14:56:17.784]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e1cd8406c8c84627 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e1cd8406c8c84627"}
14:56:17 [14:56:17.791]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:210bc000d8b18dd8 has already been completed
{"provingJobId":"1:BASE_PARITY:210bc000d8b18dd8"}
14:56:17 [14:56:17.795]
INFO:
prover-client:proving-broker Proving Broker started
14:56:17 [14:56:17.796]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bead5dd3f55322a7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bead5dd3f55322a7"}
14:56:17 [14:56:17.797]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testL7vudt/1 with map size 1073741824
14:56:17 [14:56:17.799]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:17 [14:56:17.816]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb2224429a54c8f6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:bb2224429a54c8f6"}
14:56:17 [14:56:17.819]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testL7vudt/2 with map size 1073741824
14:56:17 [14:56:17.821]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:18 [14:56:18.889]
INFO:
prover-client:proving-broker Proving Broker started
14:56:18 [14:56:18.900]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bead5dd3f55322a7 settled=false
{"provingJobId":"1:BASE_PARITY:bead5dd3f55322a7","status":"pending"}
14:56:18 [14:56:18.902]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bb2224429a54c8f6 settled=false
{"provingJobId":"1:BASE_PARITY:bb2224429a54c8f6","status":"pending"}
14:56:18 [14:56:18.903]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bead5dd3f55322a7 type=BASE_PARITY startedAt=1750690577839 older agent has taken job
{"provingJobId":"1:BASE_PARITY:bead5dd3f55322a7"}
14:56:18 [14:56:18.903]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bead5dd3f55322a7 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:bead5dd3f55322a7"}
14:56:18 [14:56:18.907]
INFO:
prover-client:proving-broker Proving Broker started
14:56:18 [14:56:18.908]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce6c2b08bb5bafe2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce6c2b08bb5bafe2"}
14:56:18 [14:56:18.909]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI7XVZC/1 with map size 1073741824
14:56:18 [14:56:18.910]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:18 [14:56:18.931]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c03a2fd2c1e05aed epochNumber=2
{"provingJobId":"1:BASE_PARITY:c03a2fd2c1e05aed"}
14:56:18 [14:56:18.932]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testI7XVZC/2 with map size 1073741824
14:56:18 [14:56:18.934]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:19 [14:56:19.952]
INFO:
prover-client:proving-broker Proving Broker started
14:56:19 [14:56:19.954]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ce6c2b08bb5bafe2 settled=false
{"provingJobId":"1:BASE_PARITY:ce6c2b08bb5bafe2","status":"pending"}
14:56:19 [14:56:19.957]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c03a2fd2c1e05aed settled=false
{"provingJobId":"1:BASE_PARITY:c03a2fd2c1e05aed","status":"pending"}
14:56:19 [14:56:19.958]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ce6c2b08bb5bafe2 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:ce6c2b08bb5bafe2"}
14:56:19 [14:56:19.963]
INFO:
prover-client:proving-broker Proving Broker started
14:56:19 [14:56:19.964]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4d4036f07b11ddc epochNumber=1
{"provingJobId":"1:BASE_PARITY:e4d4036f07b11ddc"}
14:56:19 [14:56:19.965]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testI8dy4C/1 with map size 1073741824
14:56:19 [14:56:19.966]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:19 [14:56:19.996]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d6f32571ec65729d epochNumber=2
{"provingJobId":"1:BASE_PARITY:d6f32571ec65729d"}
14:56:19 [14:56:19.997]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testI8dy4C/2 with map size 1073741824
14:56:19 [14:56:19.999]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.022]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.025]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e4d4036f07b11ddc settled=false
{"provingJobId":"1:BASE_PARITY:e4d4036f07b11ddc","status":"pending"}
14:56:21 [14:56:21.027]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d6f32571ec65729d settled=false
{"provingJobId":"1:BASE_PARITY:d6f32571ec65729d","status":"pending"}
14:56:21 [14:56:21.029]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e4d4036f07b11ddc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e4d4036f07b11ddc"}
14:56:21 [14:56:21.029]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e4d4036f07b11ddc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e4d4036f07b11ddc"}
14:56:21 [14:56:21.039]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.041]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:39f05d3531928f65 epochNumber=1
{"provingJobId":"1:BASE_PARITY:39f05d3531928f65"}
14:56:21 [14:56:21.042]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOfUsDP/1 with map size 1073741824
14:56:21 [14:56:21.044]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.061]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a4077a4f58a65bb epochNumber=2
{"provingJobId":"1:BASE_PARITY:2a4077a4f58a65bb"}
14:56:21 [14:56:21.061]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOfUsDP/2 with map size 1073741824
14:56:21 [14:56:21.063]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.079]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:39f05d3531928f65 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:39f05d3531928f65"}
14:56:21 [14:56:21.086]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2a4077a4f58a65bb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2a4077a4f58a65bb"}
14:56:21 [14:56:21.099]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.100]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e09b136d6b384cff epochNumber=1
{"provingJobId":"1:BASE_PARITY:e09b136d6b384cff"}
14:56:21 [14:56:21.101]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testT4M8N5/1 with map size 1073741824
14:56:21 [14:56:21.103]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.123]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8761f7495d2d13ab epochNumber=2
{"provingJobId":"1:BASE_PARITY:8761f7495d2d13ab"}
14:56:21 [14:56:21.125]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testT4M8N5/2 with map size 1073741824
14:56:21 [14:56:21.127]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.142]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e09b136d6b384cff type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e09b136d6b384cff"}
14:56:21 [14:56:21.143]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e09b136d6b384cff type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e09b136d6b384cff"}
14:56:21 [14:56:21.149]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8761f7495d2d13ab type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8761f7495d2d13ab"}
14:56:21 [14:56:21.159]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.161]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ece98e2d7b7d6214 err=test error
{"provingJoId":"1:BASE_PARITY:ece98e2d7b7d6214"}
14:56:21 [14:56:21.167]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.168]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:24f2854a34ccc92c not found
{"provingJobId":"1:BASE_PARITY:24f2854a34ccc92c"}
14:56:21 [14:56:21.172]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.172]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:329b0936336ee6e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:329b0936336ee6e6"}
14:56:21 [14:56:21.173]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQq06RZ/1 with map size 1073741824
14:56:21 [14:56:21.174]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.206]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.208]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51e8bb927618cab6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:51e8bb927618cab6"}
14:56:21 [14:56:21.209]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testg98CzE/1 with map size 1073741824
14:56:21 [14:56:21.211]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.335]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:51e8bb927618cab6 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:51e8bb927618cab6"}
14:56:21 [14:56:21.340]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.341]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d5ce971eca973b1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d5ce971eca973b1"}
14:56:21 [14:56:21.341]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1QdQLd/1 with map size 1073741824
14:56:21 [14:56:21.344]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.465]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7d5ce971eca973b1 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:7d5ce971eca973b1"}
14:56:21 [14:56:21.471]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a559404d41efb9bf epochNumber=10
{"provingJobId":"1:BASE_PARITY:a559404d41efb9bf"}
14:56:21 [14:56:21.472]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-test1QdQLd/10 with map size 1073741824
14:56:21 [14:56:21.474]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.517]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:21 [14:56:21.518]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:56:21 [14:56:21.622]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.623]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7fe7d8173bcc3ca epochNumber=1
{"provingJobId":"1:BASE_PARITY:d7fe7d8173bcc3ca"}
14:56:21 [14:56:21.625]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeiZVEG/1 with map size 1073741824
14:56:21 [14:56:21.627]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.800]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d7fe7d8173bcc3ca timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:d7fe7d8173bcc3ca"}
14:56:21 [14:56:21.851]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.852]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68381d4d17830baa epochNumber=1
{"provingJobId":"1:BASE_PARITY:68381d4d17830baa"}
14:56:21 [14:56:21.853]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM4oT8P/1 with map size 1073741824
14:56:21 [14:56:21.855]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.874]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:68381d4d17830baa type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:68381d4d17830baa"}
14:56:21 [14:56:21.880]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.881]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:295e52a48febafa2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:295e52a48febafa2"}
14:56:21 [14:56:21.882]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcNZlqI/1 with map size 1073741824
14:56:21 [14:56:21.884]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.903]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:295e52a48febafa2 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:295e52a48febafa2"}
14:56:21 [14:56:21.905]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:295e52a48febafa2 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:295e52a48febafa2"}
14:56:21 [14:56:21.914]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.915]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f8ba5be54342dc3f epochNumber=1
{"provingJobId":"1:BASE_PARITY:f8ba5be54342dc3f"}
14:56:21 [14:56:21.918]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLO7jwK/1 with map size 1073741824
14:56:21 [14:56:21.921]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.944]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f8ba5be54342dc3f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:f8ba5be54342dc3f"}
14:56:21 [14:56:21.953]
INFO:
prover-client:proving-broker Proving Broker started
14:56:21 [14:56:21.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b65ae68a5d03bc75 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b65ae68a5d03bc75"}
14:56:21 [14:56:21.954]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQcvnTj/1 with map size 1073741824
14:56:21 [14:56:21.956]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:21 [14:56:21.975]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b65ae68a5d03bc75 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:b65ae68a5d03bc75"}
14:56:21 [14:56:21.976]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ba0d945f584488b epochNumber=3
{"provingJobId":"1:BASE_PARITY:3ba0d945f584488b"}
14:56:21 [14:56:21.978]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testQcvnTj/3 with map size 1073741824
14:56:21 [14:56:21.984]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.032]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:22 [14:56:22.033]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:56:22 [14:56:22.034]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b65ae68a5d03bc75 err=test error
{"provingJoId":"1:BASE_PARITY:b65ae68a5d03bc75"}
14:56:22 [14:56:22.042]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdAI1kY/1 with map size 1073741824
14:56:22 [14:56:22.043]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.062]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testdAI1kY/2 with map size 1073741824
14:56:22 [14:56:22.064]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.083]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.096]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4e9d44499c8bec41 settled=false
{"provingJobId":"1:BASE_PARITY:4e9d44499c8bec41","status":"pending"}
14:56:22 [14:56:22.100]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9efa7844ecf65604 settled=false
{"provingJobId":"1:BASE_PARITY:9efa7844ecf65604","status":"pending"}
14:56:22 [14:56:22.106]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7oNnU6/1 with map size 1073741824
14:56:22 [14:56:22.108]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.125]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7oNnU6/2 with map size 1073741824
14:56:22 [14:56:22.127]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.156]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.159]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e71e91a701413c48 settled=true
{"provingJobId":"1:BASE_PARITY:e71e91a701413c48","status":"fulfilled"}
14:56:22 [14:56:22.160]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:5a8142611d66213c settled=true
{"provingJobId":"2:BASE_PARITY:5a8142611d66213c","status":"fulfilled"}
14:56:22 [14:56:22.169]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBNB5fY/1 with map size 1073741824
14:56:22 [14:56:22.174]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.209]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBNB5fY/2 with map size 1073741824
14:56:22 [14:56:22.212]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.229]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.232]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:69bb4fa82cbed081 settled=true
{"provingJobId":"1:BASE_PARITY:69bb4fa82cbed081","status":"fulfilled"}
14:56:22 [14:56:22.234]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:38414b74688e682a settled=false
{"provingJobId":"1:BASE_PARITY:38414b74688e682a","status":"pending"}
14:56:22 [14:56:22.239]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.240]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8c96d49dd3fc37d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8c96d49dd3fc37d6"}
14:56:22 [14:56:22.241]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOPu6RE/1 with map size 1073741824
14:56:22 [14:56:22.243]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.266]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.267]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30b14a68f8d8cef0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:30b14a68f8d8cef0"}
14:56:22 [14:56:22.269]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:30b14a68f8d8cef0: Error: db error: Error: db error
14:56:22 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:30b14a68f8d8cef0"}
14:56:22 [14:56:22.273]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.273]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a582142cfb2b4582 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a582142cfb2b4582"}
14:56:22 [14:56:22.276]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtgwLWq/1 with map size 1073741824
14:56:22 [14:56:22.284]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.310]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a582142cfb2b4582 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a582142cfb2b4582"}
14:56:22 [14:56:22.311]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a582142cfb2b4582 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a582142cfb2b4582"}
14:56:22 [14:56:22.322]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.323]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c7962b03df9252fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:c7962b03df9252fa"}
14:56:22 [14:56:22.325]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBHBMfU/1 with map size 1073741824
14:56:22 [14:56:22.328]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.351]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c7962b03df9252fa type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c7962b03df9252fa"}
14:56:22 [14:56:22.352]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c7962b03df9252fa type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c7962b03df9252fa"}
14:56:22 [14:56:22.353]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:c7962b03df9252fa: Error: db error
14:56:22 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:c7962b03df9252fa"}
14:56:22 [14:56:22.359]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f95e484c949c39b epochNumber=1
{"provingJobId":"1:BASE_PARITY:2f95e484c949c39b"}
14:56:22 [14:56:22.361]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPsk9Mp/1 with map size 1073741824
14:56:22 [14:56:22.364]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.388]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2f95e484c949c39b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2f95e484c949c39b"}
14:56:22 [14:56:22.389]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2f95e484c949c39b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2f95e484c949c39b"}
14:56:22 [14:56:22.404]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.404]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dfcefb6e8b2b876b epochNumber=1
{"provingJobId":"1:BASE_PARITY:dfcefb6e8b2b876b"}
14:56:22 [14:56:22.405]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEUijfF/1 with map size 1073741824
14:56:22 [14:56:22.408]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.431]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dfcefb6e8b2b876b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:dfcefb6e8b2b876b"}
14:56:22 [14:56:22.432]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:dfcefb6e8b2b876b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:dfcefb6e8b2b876b"}
14:56:22 [14:56:22.432]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:dfcefb6e8b2b876b jobErr=test error: Error: db error
14:56:22 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:dfcefb6e8b2b876b"}
14:56:22 [14:56:22.436]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.437]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1fe94fbaad45bb40 not found
{"provingJobId":"1:BASE_PARITY:1fe94fbaad45bb40"}
14:56:22 [14:56:22.444]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.445]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:d05ae58c3728903c err=test error
{"provingJoId":"1:BASE_PARITY:d05ae58c3728903c"}
14:56:22 [14:56:22.450]
INFO:
prover-client:proving-broker Proving Broker started
14:56:22 [14:56:22.461]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:128f206882623a5e epochNumber=1
{"provingJobId":"1:BASE_PARITY:128f206882623a5e"}
14:56:22 [14:56:22.462]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9Fh2ya/1 with map size 1073741824
14:56:22 [14:56:22.465]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.508]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:128f206882623a5e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:128f206882623a5e"}
14:56:22 [14:56:22.509]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:128f206882623a5e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:128f206882623a5e"}
14:56:22 [14:56:22.535]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:18681d2a73dc8c48 epochNumber=2
{"provingJobId":"2:BASE_PARITY:18681d2a73dc8c48"}
14:56:22 [14:56:22.537]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test9Fh2ya/2 with map size 1073741824
14:56:22 [14:56:22.539]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.563]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:18681d2a73dc8c48 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:18681d2a73dc8c48"}
14:56:22 [14:56:22.564]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:18681d2a73dc8c48 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:18681d2a73dc8c48"}
14:56:22 [14:56:22.633]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:cc992b17364519b3 epochNumber=3
{"provingJobId":"3:BASE_PARITY:cc992b17364519b3"}
14:56:22 [14:56:22.636]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test9Fh2ya/3 with map size 1073741824
14:56:22 [14:56:22.640]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.684]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:22 [14:56:22.685]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:56:22 [14:56:22.726]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:c198208416f26a3e epochNumber=4
{"provingJobId":"4:BASE_PARITY:c198208416f26a3e"}
14:56:22 [14:56:22.727]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test9Fh2ya/4 with map size 1073741824
14:56:22 [14:56:22.730]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.768]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:22 [14:56:22.769]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
14:56:22 [14:56:22.818]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:6affb317a200d347 epochNumber=5
{"provingJobId":"5:BASE_PARITY:6affb317a200d347"}
14:56:22 [14:56:22.820]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test9Fh2ya/5 with map size 1073741824
14:56:22 [14:56:22.823]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:56:22 [14:56:22.847]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:56:22 [14:56:22.847]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
14:56:22
PASS src/proving_broker/proving_broker.test.ts (
13.496 s)
14:56:22 ProvingBroker
14:56:22 Producer API
14:56:22
✓ refuses stale jobs (58 ms)
14:56:22
✓ enqueues jobs (4 ms)
14:56:22
✓ ignores duplicate jobs (4 ms)
14:56:22
✓ reports correct status when enqueuing repeat jobs (4 ms)
14:56:22
✓ throws an error in case of duplicate job IDs (25 ms)
14:56:22
✓ returns not-found status for non-existing jobs (1 ms)
14:56:22
✓ cancels jobs in queue (2 ms)
14:56:22
✓ cancels jobs in-progress (2 ms)
14:56:22
✓ returns job result if successful (3 ms)
14:56:22
✓ returns job error if failed (1 ms)
14:56:22
✓ correctly returns job status for concurrent writes (2 ms)
14:56:22
✓ refuses stale jobs (40 ms)
14:56:22
✓ enqueues jobs (44 ms)
14:56:22
✓ ignores duplicate jobs (28 ms)
14:56:22
✓ reports correct status when enqueuing repeat jobs (36 ms)
14:56:22
✓ throws an error in case of duplicate job IDs (30 ms)
14:56:22
✓ returns not-found status for non-existing jobs (3 ms)
14:56:22
✓ cancels jobs in queue (34 ms)
14:56:22
✓ cancels jobs in-progress (33 ms)
14:56:22
✓ returns job result if successful (43 ms)
14:56:22
✓ returns job error if failed (33 ms)
14:56:22
✓ correctly returns job status for concurrent writes (49 ms)
14:56:22 Consumer API
14:56:22
✓ returns undefined if no jobs are available (1 ms)
14:56:22
✓ returns jobs in priority order (2 ms)
14:56:22
✓ returns undefined if no jobs are available for the given allowList (9 ms)
14:56:22
✓ returns a job if it is in the allowList (2 ms)
14:56:22
✓ returns the most important job if it is in the allowList (2 ms)
14:56:22
✓ returns any job if filter is empty (2 ms)
14:56:22
✓ returns a new job when reporting job success (2 ms)
14:56:22
✓ returns a new job when reporting permanent error (2 ms)
14:56:22
✓ returns a new job when reporting retry-able error (2 ms)
14:56:22
✓ returns a new job when reporting progress if current one is cancelled (7 ms)
14:56:22
✓ returns a new job if job is already in progress elsewhere (1058 ms)
14:56:22
✓ avoids sending the same job to a new agent after a restart (1004 ms)
14:56:22
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
14:56:22
✓ tracks job result if in progress (2 ms)
14:56:22
✓ tracks job result even if job is in queue (2 ms)
14:56:22
✓ ignores reported job error if unknown job (1 ms)
14:56:22
✓ ignores job result if unknown job (1 ms)
14:56:22
✓ returns undefined if no jobs are available (3 ms)
14:56:22
✓ returns jobs in priority order (45 ms)
14:56:22
✓ returns undefined if no jobs are available for the given allowList (33 ms)
14:56:22
✓ returns a job if it is in the allowList (76 ms)
14:56:22
✓ returns the most important job if it is in the allowList (76 ms)
14:56:22
✓ returns any job if filter is empty (76 ms)
14:56:22
✓ returns a new job when reporting job success (46 ms)
14:56:22
✓ returns a new job when reporting permanent error (45 ms)
14:56:22
✓ returns a new job when reporting retry-able error (40 ms)
14:56:22
✓ returns a new job when reporting progress if current one is cancelled (37 ms)
14:56:22
✓ returns a new job if job is already in progress elsewhere (1112 ms)
14:56:22
✓ avoids sending the same job to a new agent after a restart (1055 ms)
14:56:22
✓ avoids sending a completed job to a new agent after a restart (1075 ms)
14:56:22
✓ tracks job result if in progress (57 ms)
14:56:22
✓ tracks job result even if job is in queue (62 ms)
14:56:22
✓ ignores reported job error if unknown job (5 ms)
14:56:22
✓ ignores job result if unknown job (7 ms)
14:56:22 Timeouts
14:56:22
✓ tracks in progress jobs (1 ms)
14:56:22
✓ re-enqueues jobs that time out (108 ms)
14:56:22
✓ cancel stale jobs that time out (230 ms)
14:56:22
✓ keeps the jobs in progress while it is alive (203 ms)
14:56:22
✓ tracks in progress jobs (29 ms)
14:56:22
✓ re-enqueues jobs that time out (139 ms)
14:56:22
✓ cancel stale jobs that time out (282 ms)
14:56:22
✓ keeps the jobs in progress while it is alive (228 ms)
14:56:22 Retries
14:56:22
✓ retries jobs (3 ms)
14:56:22
✓ retries up to a maximum number of times (3 ms)
14:56:22
✓ passing retry=false does not retry (2 ms)
14:56:22
✓ does not retry if job is stale (28 ms)
14:56:22
✓ retries jobs (27 ms)
14:56:22
✓ retries up to a maximum number of times (36 ms)
14:56:22
✓ passing retry=false does not retry (39 ms)
14:56:22
✓ does not retry if job is stale (86 ms)
14:56:22 Database management
14:56:22
✓ re-enqueues proof requests on start (2 ms)
14:56:22
✓ restores proof results on start (2 ms)
14:56:22
✓ only re-enqueues unfinished jobs (1 ms)
14:56:22
✓ saves job when enqueued (2 ms)
14:56:22
✓ does not retain job if database fails to save (2 ms)
14:56:22
✓ saves job result (2 ms)
14:56:22
✓ saves result even if database fails to save (2 ms)
14:56:22
✓ saves job error (1 ms)
14:56:22
✓ saves job error even if database fails to save (5 ms)
14:56:22
✓ does not save job result if job is unknown (2 ms)
14:56:22
✓ does not save job error if job is unknown (1 ms)
14:56:22
✓ cleans up old jobs periodically (266 ms)
14:56:22
✓ re-enqueues proof requests on start (65 ms)
14:56:22
✓ restores proof results on start (61 ms)
14:56:22
✓ only re-enqueues unfinished jobs (73 ms)
14:56:22
✓ saves job when enqueued (25 ms)
14:56:22
✓ does not retain job if database fails to save (8 ms)
14:56:22
✓ saves job result (48 ms)
14:56:22
✓ saves result even if database fails to save (35 ms)
14:56:22
✓ saves job error (44 ms)
14:56:22
✓ saves job error even if database fails to save (33 ms)
14:56:22
✓ does not save job result if job is unknown (3 ms)
14:56:22
✓ does not save job error if job is unknown (9 ms)
14:56:22
✓ cleans up old jobs periodically (457 ms)
14:56:22
14:56:22
Test Suites: 1 passed, 1 total
14:56:22
Tests: 96 passed, 96 total
14:56:22
Snapshots: 0 total
14:56:22
Time: 13.591 s
14:56:22
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
14:56:22
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?