Parent Log:
http://ci.aztec-labs.com/475a573e01692f7b
Command: 5484f2347094f66b yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/5ca43853c40a2b8bfbc2148d5fdd46c6f82a74eb
Env: REF_NAME=gh-readonly-queue/next/pr-14178-3cbe7945c84145b8646cf5cf8fb730b915c0dc93 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 16 13:31:42 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-14178_amd64_x1-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
13:31:46 [13:31:46.807]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.815]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:795b2acdd4563a99 epochNumber=42
{"provingJobId":"1:BASE_PARITY:795b2acdd4563a99"}
13:31:46 [13:31:46.819]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:c3e390718ab94c34 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c3e390718ab94c34"}
13:31:46 [13:31:46.857]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.858]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2e8e8e7bb22b553f epochNumber=1
{"provingJobId":"1:BASE_PARITY:2e8e8e7bb22b553f"}
13:31:46 [13:31:46.860]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:187f7cd4a6e1b4dc epochNumber=1
{"provingJobId":"1:BASE_PARITY:187f7cd4a6e1b4dc"}
13:31:46 [13:31:46.862]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.862]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54cc9429bef296c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:54cc9429bef296c3"}
13:31:46 [13:31:46.863]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:54cc9429bef296c3 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:54cc9429bef296c3"}
13:31:46 [13:31:46.866]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.867]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3686c6648201826f epochNumber=1
{"provingJobId":"1:BASE_PARITY:3686c6648201826f"}
13:31:46 [13:31:46.868]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3686c6648201826f epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3686c6648201826f"}
13:31:46 [13:31:46.868]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3686c6648201826f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3686c6648201826f"}
13:31:46 [13:31:46.869]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3686c6648201826f epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:3686c6648201826f"}
13:31:46 [13:31:46.870]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.871]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f9db01b89ab62de epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f9db01b89ab62de"}
13:31:46 [13:31:46.905]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.906]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.907]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f5db764b0e69897 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f5db764b0e69897"}
13:31:46 [13:31:46.907]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:4f5db764b0e69897
{"provingJobId":"1:BASE_PARITY:4f5db764b0e69897"}
13:31:46 [13:31:46.908]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4f5db764b0e69897 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4f5db764b0e69897"}
13:31:46 [13:31:46.908]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4f5db764b0e69897 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:4f5db764b0e69897"}
13:31:46 [13:31:46.909]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.909]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3079757c83c313c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3079757c83c313c0"}
13:31:46 [13:31:46.910]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3079757c83c313c0
{"provingJobId":"1:BASE_PARITY:3079757c83c313c0"}
13:31:46 [13:31:46.910]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3079757c83c313c0 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3079757c83c313c0"}
13:31:46 [13:31:46.911]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.912]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f8104ae2a5a02317 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f8104ae2a5a02317"}
13:31:46 [13:31:46.912]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f8104ae2a5a02317 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f8104ae2a5a02317"}
13:31:46 [13:31:46.913]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f8104ae2a5a02317 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f8104ae2a5a02317"}
13:31:46 [13:31:46.914]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.914]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e641e19ca347746 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e641e19ca347746"}
13:31:46 [13:31:46.914]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3e641e19ca347746 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3e641e19ca347746"}
13:31:46 [13:31:46.915]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3e641e19ca347746 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3e641e19ca347746"}
13:31:46 [13:31:46.915]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.916]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a047d2b4fda3fa0 epochNumber=0
{"provingJobId":"1:BASE_PARITY:7a047d2b4fda3fa0"}
13:31:46 [13:31:46.916]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7a047d2b4fda3fa0 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:7a047d2b4fda3fa0"}
13:31:46 [13:31:46.916]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e06d85f82c90162e epochNumber=0
{"provingJobId":"1:BASE_PARITY:e06d85f82c90162e"}
13:31:46 [13:31:46.916]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7a047d2b4fda3fa0 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:7a047d2b4fda3fa0"}
13:31:46 [13:31:46.917]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9a1d412ea3cdafd1 epochNumber=0
{"provingJobId":"1:BASE_PARITY:9a1d412ea3cdafd1"}
13:31:46 [13:31:46.918]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.919]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.919]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ca2bfd0166d9407a epochNumber=2
{"provingJobId":"1:BASE_PARITY:ca2bfd0166d9407a"}
13:31:46 [13:31:46.920]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d90c4c91fb95afa epochNumber=1
{"provingJobId":"1:BASE_PARITY:1d90c4c91fb95afa"}
13:31:46 [13:31:46.921]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.921]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a14d061cb31edb0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a14d061cb31edb0"}
13:31:46 [13:31:46.922]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8c2bdc7d283b8391 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8c2bdc7d283b8391"}
13:31:46 [13:31:46.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c6dfcee01f7e2aa epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c6dfcee01f7e2aa"}
13:31:46 [13:31:46.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c00a4cd81c856a9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7c00a4cd81c856a9"}
13:31:46 [13:31:46.924]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ba79abb4736cdea7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ba79abb4736cdea7"}
13:31:46 [13:31:46.925]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.925]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:749d2f4c502740ee epochNumber=1
{"provingJobId":"1:BASE_PARITY:749d2f4c502740ee"}
13:31:46 [13:31:46.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b0c8065214efc89 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b0c8065214efc89"}
13:31:46 [13:31:46.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ecd26f9e4fcf015 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3ecd26f9e4fcf015"}
13:31:46 [13:31:46.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:511196b58ab729e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:511196b58ab729e6"}
13:31:46 [13:31:46.927]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.928]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6cba52b376db7567 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6cba52b376db7567"}
13:31:46 [13:31:46.928]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:468a5067746fb404 epochNumber=1
{"provingJobId":"1:BASE_PARITY:468a5067746fb404"}
13:31:46 [13:31:46.928]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91331b54e625f664 epochNumber=2
{"provingJobId":"1:BASE_PARITY:91331b54e625f664"}
13:31:46 [13:31:46.929]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5ecc0ce308dcfdfc epochNumber=1
{"provingJobId":"1:BASE_PARITY:5ecc0ce308dcfdfc"}
13:31:46 [13:31:46.930]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.930]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de106c7995977f68 epochNumber=1
{"provingJobId":"1:BASE_PARITY:de106c7995977f68"}
13:31:46 [13:31:46.931]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:97407fbe55d525be epochNumber=1
{"provingJobId":"1:BASE_PARITY:97407fbe55d525be"}
13:31:46 [13:31:46.931]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:de106c7995977f68 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:de106c7995977f68"}
13:31:46 [13:31:46.932]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.932]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4bbddcc7711d9306 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4bbddcc7711d9306"}
13:31:46 [13:31:46.933]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25421bb3d0dea54c epochNumber=1
{"provingJobId":"1:BASE_PARITY:25421bb3d0dea54c"}
13:31:46 [13:31:46.933]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4bbddcc7711d9306 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:4bbddcc7711d9306"}
13:31:46 [13:31:46.934]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:245a200896d9eeec epochNumber=1
{"provingJobId":"1:BASE_PARITY:245a200896d9eeec"}
13:31:46 [13:31:46.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b5b697421343759 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9b5b697421343759"}
13:31:46 [13:31:46.935]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:245a200896d9eeec type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:245a200896d9eeec"}
13:31:46 [13:31:46.936]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a8eb637d90c73f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a8eb637d90c73f9"}
13:31:46 [13:31:46.937]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3a8eb637d90c73f9
{"provingJobId":"1:BASE_PARITY:3a8eb637d90c73f9"}
13:31:46 [13:31:46.937]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3a8eb637d90c73f9 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3a8eb637d90c73f9"}
13:31:46 [13:31:46.938]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:58f93dc450e9089f epochNumber=1
{"provingJobId":"1:BASE_PARITY:58f93dc450e9089f"}
13:31:46 [13:31:46.938]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3a8eb637d90c73f9 has already been completed
{"provingJobId":"1:BASE_PARITY:3a8eb637d90c73f9"}
13:31:46 [13:31:46.939]
INFO:
prover-client:proving-broker Proving Broker started
13:31:46 [13:31:46.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1b4ffc9aae0e7b11 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1b4ffc9aae0e7b11"}
13:31:46 [13:31:46.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3cf071b804f3747 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d3cf071b804f3747"}
13:31:47 [13:31:47.992]
INFO:
prover-client:proving-broker Proving Broker started
13:31:47 [13:31:47.993]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1b4ffc9aae0e7b11 settled=false
{"provingJobId":"1:BASE_PARITY:1b4ffc9aae0e7b11","status":"pending"}
13:31:47 [13:31:47.993]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d3cf071b804f3747 settled=false
{"provingJobId":"1:BASE_PARITY:d3cf071b804f3747","status":"pending"}
13:31:47 [13:31:47.994]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1b4ffc9aae0e7b11 type=BASE_PARITY startedAt=1750080706940 older agent has taken job
{"provingJobId":"1:BASE_PARITY:1b4ffc9aae0e7b11"}
13:31:47 [13:31:47.995]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1b4ffc9aae0e7b11 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:1b4ffc9aae0e7b11"}
13:31:48 [13:31:48.001]
INFO:
prover-client:proving-broker Proving Broker started
13:31:48 [13:31:48.001]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ddd13a0018968024 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ddd13a0018968024"}
13:31:48 [13:31:48.003]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1072c117649b7656 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1072c117649b7656"}
13:31:49 [13:31:49.003]
INFO:
prover-client:proving-broker Proving Broker started
13:31:49 [13:31:49.004]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ddd13a0018968024 settled=false
{"provingJobId":"1:BASE_PARITY:ddd13a0018968024","status":"pending"}
13:31:49 [13:31:49.004]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1072c117649b7656 settled=false
{"provingJobId":"1:BASE_PARITY:1072c117649b7656","status":"pending"}
13:31:49 [13:31:49.005]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ddd13a0018968024 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:ddd13a0018968024"}
13:31:49 [13:31:49.009]
INFO:
prover-client:proving-broker Proving Broker started
13:31:49 [13:31:49.010]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd294fcdf60a9ac7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd294fcdf60a9ac7"}
13:31:49 [13:31:49.013]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a64c4f19297549cd epochNumber=2
{"provingJobId":"1:BASE_PARITY:a64c4f19297549cd"}
13:31:50 [13:31:50.014]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.016]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bd294fcdf60a9ac7 settled=false
{"provingJobId":"1:BASE_PARITY:bd294fcdf60a9ac7","status":"pending"}
13:31:50 [13:31:50.016]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a64c4f19297549cd settled=false
{"provingJobId":"1:BASE_PARITY:a64c4f19297549cd","status":"pending"}
13:31:50 [13:31:50.017]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bd294fcdf60a9ac7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bd294fcdf60a9ac7"}
13:31:50 [13:31:50.017]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bd294fcdf60a9ac7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:bd294fcdf60a9ac7"}
13:31:50 [13:31:50.019]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.020]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f36f83600ada6104 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f36f83600ada6104"}
13:31:50 [13:31:50.020]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ab79d48d12526a4a epochNumber=2
{"provingJobId":"1:BASE_PARITY:ab79d48d12526a4a"}
13:31:50 [13:31:50.021]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f36f83600ada6104 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f36f83600ada6104"}
13:31:50 [13:31:50.021]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ab79d48d12526a4a type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ab79d48d12526a4a"}
13:31:50 [13:31:50.023]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5cdec433e87355e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5cdec433e87355e3"}
13:31:50 [13:31:50.024]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c8e848844785cb5 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3c8e848844785cb5"}
13:31:50 [13:31:50.024]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5cdec433e87355e3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5cdec433e87355e3"}
13:31:50 [13:31:50.024]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5cdec433e87355e3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5cdec433e87355e3"}
13:31:50 [13:31:50.025]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3c8e848844785cb5 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3c8e848844785cb5"}
13:31:50 [13:31:50.026]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.026]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:fd5793373f5d8704 err=test error
{"provingJoId":"1:BASE_PARITY:fd5793373f5d8704"}
13:31:50 [13:31:50.027]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.028]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b2708641670024b2 not found
{"provingJobId":"1:BASE_PARITY:b2708641670024b2"}
13:31:50 [13:31:50.029]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.030]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5bda1e8fb30b7352 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5bda1e8fb30b7352"}
13:31:50 [13:31:50.031]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.032]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:50dfea8f80630c30 epochNumber=1
{"provingJobId":"1:BASE_PARITY:50dfea8f80630c30"}
13:31:50 [13:31:50.134]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:50dfea8f80630c30 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:50dfea8f80630c30"}
13:31:50 [13:31:50.141]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.141]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d72b0f9799af4a9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d72b0f9799af4a9"}
13:31:50 [13:31:50.243]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0d72b0f9799af4a9 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:0d72b0f9799af4a9"}
13:31:50 [13:31:50.250]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f138ed20318695c9 epochNumber=10
{"provingJobId":"1:BASE_PARITY:f138ed20318695c9"}
13:31:50 [13:31:50.270]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:50 [13:31:50.376]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.377]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9f0d7aadef689a41 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9f0d7aadef689a41"}
13:31:50 [13:31:50.531]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9f0d7aadef689a41 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:9f0d7aadef689a41"}
13:31:50 [13:31:50.580]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.581]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80da4baa5dc422ce epochNumber=1
{"provingJobId":"1:BASE_PARITY:80da4baa5dc422ce"}
13:31:50 [13:31:50.582]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:80da4baa5dc422ce type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:80da4baa5dc422ce"}
13:31:50 [13:31:50.583]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.584]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d31b5229b0f45c8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d31b5229b0f45c8"}
13:31:50 [13:31:50.585]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0d31b5229b0f45c8 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:0d31b5229b0f45c8"}
13:31:50 [13:31:50.585]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0d31b5229b0f45c8 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:0d31b5229b0f45c8"}
13:31:50 [13:31:50.587]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.588]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:485005e5033925bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:485005e5033925bc"}
13:31:50 [13:31:50.593]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:485005e5033925bc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:485005e5033925bc"}
13:31:50 [13:31:50.596]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.597]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:13c410f1687378ca epochNumber=1
{"provingJobId":"1:BASE_PARITY:13c410f1687378ca"}
13:31:50 [13:31:50.598]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:13c410f1687378ca type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:13c410f1687378ca"}
13:31:50 [13:31:50.600]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c71efe486b9efe2 epochNumber=3
{"provingJobId":"1:BASE_PARITY:1c71efe486b9efe2"}
13:31:50 [13:31:50.622]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:50 [13:31:50.625]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:13c410f1687378ca err=test error
{"provingJoId":"1:BASE_PARITY:13c410f1687378ca"}
13:31:50 [13:31:50.627]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.627]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:04e348394a5b2251 settled=false
{"provingJobId":"1:BASE_PARITY:04e348394a5b2251","status":"pending"}
13:31:50 [13:31:50.627]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fe344fc6404c2f00 settled=false
{"provingJobId":"1:BASE_PARITY:fe344fc6404c2f00","status":"pending"}
13:31:50 [13:31:50.630]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.630]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8fb38a9e4a127ccd settled=true
{"provingJobId":"1:BASE_PARITY:8fb38a9e4a127ccd","status":"fulfilled"}
13:31:50 [13:31:50.630]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:5bc162e41b6e66d2 settled=true
{"provingJobId":"2:BASE_PARITY:5bc162e41b6e66d2","status":"fulfilled"}
13:31:50 [13:31:50.632]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.632]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9d94eb5cea51c04e settled=true
{"provingJobId":"1:BASE_PARITY:9d94eb5cea51c04e","status":"fulfilled"}
13:31:50 [13:31:50.632]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:00f94bf465b2c7c7 settled=false
{"provingJobId":"1:BASE_PARITY:00f94bf465b2c7c7","status":"pending"}
13:31:50 [13:31:50.633]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.635]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a23d618aa709143 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a23d618aa709143"}
13:31:50 [13:31:50.636]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.637]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:43662f8bd4e6a275 epochNumber=1
{"provingJobId":"1:BASE_PARITY:43662f8bd4e6a275"}
13:31:50 [13:31:50.638]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:43662f8bd4e6a275: Error: db error: Error: db error
13:31:50 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:43662f8bd4e6a275"}
13:31:50 [13:31:50.639]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.639]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f3b4c395b46a223 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f3b4c395b46a223"}
13:31:50 [13:31:50.640]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7f3b4c395b46a223 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7f3b4c395b46a223"}
13:31:50 [13:31:50.640]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7f3b4c395b46a223 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7f3b4c395b46a223"}
13:31:50 [13:31:50.641]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.641]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:927670b823b61265 epochNumber=1
{"provingJobId":"1:BASE_PARITY:927670b823b61265"}
13:31:50 [13:31:50.642]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:927670b823b61265 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:927670b823b61265"}
13:31:50 [13:31:50.642]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:927670b823b61265 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:927670b823b61265"}
13:31:50 [13:31:50.644]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:927670b823b61265: Error: db error
13:31:50 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:927670b823b61265"}
13:31:50 [13:31:50.646]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.646]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:43edbb7bc1cae01a epochNumber=1
{"provingJobId":"1:BASE_PARITY:43edbb7bc1cae01a"}
13:31:50 [13:31:50.647]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:43edbb7bc1cae01a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:43edbb7bc1cae01a"}
13:31:50 [13:31:50.647]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:43edbb7bc1cae01a type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:43edbb7bc1cae01a"}
13:31:50 [13:31:50.649]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:078d65de428d8d2b epochNumber=1
{"provingJobId":"1:BASE_PARITY:078d65de428d8d2b"}
13:31:50 [13:31:50.649]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:078d65de428d8d2b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:078d65de428d8d2b"}
13:31:50 [13:31:50.649]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:078d65de428d8d2b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:078d65de428d8d2b"}
13:31:50 [13:31:50.650]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:078d65de428d8d2b jobErr=test error: Error: db error
13:31:50 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:078d65de428d8d2b"}
13:31:50 [13:31:50.651]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.651]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1068f7de4b4c166d not found
{"provingJobId":"1:BASE_PARITY:1068f7de4b4c166d"}
13:31:50 [13:31:50.653]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.653]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:6caa6f23776fa474 err=test error
{"provingJoId":"1:BASE_PARITY:6caa6f23776fa474"}
13:31:50 [13:31:50.654]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.665]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:70c61e60e27ccdb9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:70c61e60e27ccdb9"}
13:31:50 [13:31:50.666]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:70c61e60e27ccdb9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:70c61e60e27ccdb9"}
13:31:50 [13:31:50.667]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:70c61e60e27ccdb9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:70c61e60e27ccdb9"}
13:31:50 [13:31:50.677]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:2b4fbeed58f9f310 epochNumber=2
{"provingJobId":"2:BASE_PARITY:2b4fbeed58f9f310"}
13:31:50 [13:31:50.678]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:2b4fbeed58f9f310 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:2b4fbeed58f9f310"}
13:31:50 [13:31:50.678]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:2b4fbeed58f9f310 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:2b4fbeed58f9f310"}
13:31:50 [13:31:50.738]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:097a6ef38cce4826 epochNumber=3
{"provingJobId":"3:BASE_PARITY:097a6ef38cce4826"}
13:31:50 [13:31:50.756]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:50 [13:31:50.801]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:3c0a2ea9b15cd48e epochNumber=4
{"provingJobId":"4:BASE_PARITY:3c0a2ea9b15cd48e"}
13:31:50 [13:31:50.810]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:50 [13:31:50.864]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:f610d5af99490dcd epochNumber=5
{"provingJobId":"5:BASE_PARITY:f610d5af99490dcd"}
13:31:50 [13:31:50.890]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:50 [13:31:50.922]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d2fd9d3ec8bd0a4 epochNumber=42
{"provingJobId":"1:BASE_PARITY:7d2fd9d3ec8bd0a4"}
13:31:50 [13:31:50.924]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testDBw4JS/42 with map size 1073741824
13:31:50 [13:31:50.928]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:50 [13:31:50.957]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:dc52d3659b538586 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc52d3659b538586"}
13:31:50 [13:31:50.961]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.962]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ab759664c82e027 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ab759664c82e027"}
13:31:50 [13:31:50.962]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLOGt4v/1 with map size 1073741824
13:31:50 [13:31:50.964]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:50 [13:31:50.984]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68e6002ef8cc8ef0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68e6002ef8cc8ef0"}
13:31:50 [13:31:50.996]
INFO:
prover-client:proving-broker Proving Broker started
13:31:50 [13:31:50.997]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:635305fec2b83a5e epochNumber=1
{"provingJobId":"1:BASE_PARITY:635305fec2b83a5e"}
13:31:50 [13:31:50.998]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl5lXrr/1 with map size 1073741824
13:31:51 [13:31:51.002]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.021]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:635305fec2b83a5e epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:635305fec2b83a5e"}
13:31:51 [13:31:51.026]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.027]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae04120feae66ceb epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae04120feae66ceb"}
13:31:51 [13:31:51.028]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testelz9iJ/1 with map size 1073741824
13:31:51 [13:31:51.029]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.056]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ae04120feae66ceb epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ae04120feae66ceb"}
13:31:51 [13:31:51.057]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ae04120feae66ceb type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ae04120feae66ceb"}
13:31:51 [13:31:51.064]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ae04120feae66ceb epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ae04120feae66ceb"}
13:31:51 [13:31:51.068]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.069]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0c017d4687dd41fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:0c017d4687dd41fc"}
13:31:51 [13:31:51.070]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeHwJQP/1 with map size 1073741824
13:31:51 [13:31:51.073]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.106]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.110]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.111]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c94060fdb65de59b epochNumber=1
{"provingJobId":"1:BASE_PARITY:c94060fdb65de59b"}
13:31:51 [13:31:51.111]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test16Esdv/1 with map size 1073741824
13:31:51 [13:31:51.113]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.134]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:c94060fdb65de59b
{"provingJobId":"1:BASE_PARITY:c94060fdb65de59b"}
13:31:51 [13:31:51.135]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c94060fdb65de59b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c94060fdb65de59b"}
13:31:51 [13:31:51.135]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c94060fdb65de59b type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:c94060fdb65de59b"}
13:31:51 [13:31:51.144]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.145]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eea32229f66c0aec epochNumber=1
{"provingJobId":"1:BASE_PARITY:eea32229f66c0aec"}
13:31:51 [13:31:51.145]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAbHRWg/1 with map size 1073741824
13:31:51 [13:31:51.148]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.166]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:eea32229f66c0aec
{"provingJobId":"1:BASE_PARITY:eea32229f66c0aec"}
13:31:51 [13:31:51.166]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:eea32229f66c0aec type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:eea32229f66c0aec"}
13:31:51 [13:31:51.176]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.176]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ea95c96a521f4d14 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ea95c96a521f4d14"}
13:31:51 [13:31:51.177]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testP0BTlL/1 with map size 1073741824
13:31:51 [13:31:51.181]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.207]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ea95c96a521f4d14 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ea95c96a521f4d14"}
13:31:51 [13:31:51.207]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ea95c96a521f4d14 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ea95c96a521f4d14"}
13:31:51 [13:31:51.218]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.218]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:93f1a7d49dcd2ce5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:93f1a7d49dcd2ce5"}
13:31:51 [13:31:51.219]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testr3xsWw/1 with map size 1073741824
13:31:51 [13:31:51.221]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.237]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:93f1a7d49dcd2ce5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:93f1a7d49dcd2ce5"}
13:31:51 [13:31:51.237]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:93f1a7d49dcd2ce5 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:93f1a7d49dcd2ce5"}
13:31:51 [13:31:51.245]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.245]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dceeb8606cc67801 epochNumber=0
{"provingJobId":"1:BASE_PARITY:dceeb8606cc67801"}
13:31:51 [13:31:51.247]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testSV9aOK/0 with map size 1073741824
13:31:51 [13:31:51.250]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.268]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dceeb8606cc67801 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:dceeb8606cc67801"}
13:31:51 [13:31:51.269]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dd31c137f215bf45 epochNumber=0
{"provingJobId":"1:BASE_PARITY:dd31c137f215bf45"}
13:31:51 [13:31:51.269]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dceeb8606cc67801 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:dceeb8606cc67801"}
13:31:51 [13:31:51.270]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de472cee0a579e01 epochNumber=0
{"provingJobId":"1:BASE_PARITY:de472cee0a579e01"}
13:31:51 [13:31:51.300]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.307]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.308]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d97b104208fc688 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3d97b104208fc688"}
13:31:51 [13:31:51.309]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLUIbVD/2 with map size 1073741824
13:31:51 [13:31:51.311]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.329]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9cdbfa2c3656c38f epochNumber=1
{"provingJobId":"1:BASE_PARITY:9cdbfa2c3656c38f"}
13:31:51 [13:31:51.330]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLUIbVD/1 with map size 1073741824
13:31:51 [13:31:51.332]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.350]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.351]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5e5ca443b3d7cca4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5e5ca443b3d7cca4"}
13:31:51 [13:31:51.352]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaLUza8/1 with map size 1073741824
13:31:51 [13:31:51.356]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.379]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.379]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:33d060a077ff2f49 epochNumber=1
{"provingJobId":"1:BASE_PARITY:33d060a077ff2f49"}
13:31:51 [13:31:51.380]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOjETTF/1 with map size 1073741824
13:31:51 [13:31:51.391]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.417]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bbef337eaca1aca3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bbef337eaca1aca3"}
13:31:51 [13:31:51.424]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:74c3a3dfeba146e1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:74c3a3dfeba146e1"}
13:31:51 [13:31:51.425]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOjETTF/2 with map size 1073741824
13:31:51 [13:31:51.427]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.448]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fb1d3a67f2e91777 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fb1d3a67f2e91777"}
13:31:51 [13:31:51.462]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.463]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cd5c23eb7aeaeb1b epochNumber=1
{"provingJobId":"1:BASE_PARITY:cd5c23eb7aeaeb1b"}
13:31:51 [13:31:51.467]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPjMC34/1 with map size 1073741824
13:31:51 [13:31:51.472]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.489]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:070137969bfaa988 epochNumber=1
{"provingJobId":"1:BASE_PARITY:070137969bfaa988"}
13:31:51 [13:31:51.494]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:adc7c835d531f619 epochNumber=2
{"provingJobId":"1:BASE_PARITY:adc7c835d531f619"}
13:31:51 [13:31:51.495]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPjMC34/2 with map size 1073741824
13:31:51 [13:31:51.497]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8e6d1e9704d4539f epochNumber=1
{"provingJobId":"1:BASE_PARITY:8e6d1e9704d4539f"}
13:31:51 [13:31:51.523]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.524]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:48dbee46b96223b8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:48dbee46b96223b8"}
13:31:51 [13:31:51.524]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRqArmS/1 with map size 1073741824
13:31:51 [13:31:51.526]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.544]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c88845c5541b922 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c88845c5541b922"}
13:31:51 [13:31:51.551]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:88f1b213ed41d954 epochNumber=2
{"provingJobId":"1:BASE_PARITY:88f1b213ed41d954"}
13:31:51 [13:31:51.552]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testRqArmS/2 with map size 1073741824
13:31:51 [13:31:51.554]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.572]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2a8306fed8304fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2a8306fed8304fe"}
13:31:51 [13:31:51.580]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.581]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7429fcef376c49d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7429fcef376c49d7"}
13:31:51 [13:31:51.581]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3W2nlM/1 with map size 1073741824
13:31:51 [13:31:51.584]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.603]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e9bc04ddc2f61a7d epochNumber=1
{"provingJobId":"1:BASE_PARITY:e9bc04ddc2f61a7d"}
13:31:51 [13:31:51.610]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7429fcef376c49d7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7429fcef376c49d7"}
13:31:51 [13:31:51.620]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.621]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0dd08a2cef98bf00 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0dd08a2cef98bf00"}
13:31:51 [13:31:51.621]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYU6QXK/1 with map size 1073741824
13:31:51 [13:31:51.623]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.639]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f26de44150475279 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f26de44150475279"}
13:31:51 [13:31:51.646]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0dd08a2cef98bf00 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:0dd08a2cef98bf00"}
13:31:51 [13:31:51.654]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.655]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd3a5456536937ce epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd3a5456536937ce"}
13:31:51 [13:31:51.656]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testavOhud/1 with map size 1073741824
13:31:51 [13:31:51.657]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.674]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7886f1c99b84ed6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7886f1c99b84ed6"}
13:31:51 [13:31:51.680]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bd3a5456536937ce type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:bd3a5456536937ce"}
13:31:51 [13:31:51.686]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.687]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb9caf65aa643b10 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bb9caf65aa643b10"}
13:31:51 [13:31:51.688]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1ZMeJY/1 with map size 1073741824
13:31:51 [13:31:51.690]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.706]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:bb9caf65aa643b10
{"provingJobId":"1:BASE_PARITY:bb9caf65aa643b10"}
13:31:51 [13:31:51.707]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bb9caf65aa643b10 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:bb9caf65aa643b10"}
13:31:51 [13:31:51.712]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bcfdc36a95012715 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bcfdc36a95012715"}
13:31:51 [13:31:51.719]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bb9caf65aa643b10 has already been completed
{"provingJobId":"1:BASE_PARITY:bb9caf65aa643b10"}
13:31:51 [13:31:51.722]
INFO:
prover-client:proving-broker Proving Broker started
13:31:51 [13:31:51.722]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:83cbccc482f4278c epochNumber=1
{"provingJobId":"1:BASE_PARITY:83cbccc482f4278c"}
13:31:51 [13:31:51.723]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testv4kXII/1 with map size 1073741824
13:31:51 [13:31:51.724]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:51 [13:31:51.740]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:930d980b0d3e4808 epochNumber=2
{"provingJobId":"1:BASE_PARITY:930d980b0d3e4808"}
13:31:51 [13:31:51.741]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testv4kXII/2 with map size 1073741824
13:31:51 [13:31:51.743]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:52 [13:31:52.811]
INFO:
prover-client:proving-broker Proving Broker started
13:31:52 [13:31:52.819]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:83cbccc482f4278c settled=false
{"provingJobId":"1:BASE_PARITY:83cbccc482f4278c","status":"pending"}
13:31:52 [13:31:52.821]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:930d980b0d3e4808 settled=false
{"provingJobId":"1:BASE_PARITY:930d980b0d3e4808","status":"pending"}
13:31:52 [13:31:52.822]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:83cbccc482f4278c type=BASE_PARITY startedAt=1750080711759 older agent has taken job
{"provingJobId":"1:BASE_PARITY:83cbccc482f4278c"}
13:31:52 [13:31:52.823]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:83cbccc482f4278c type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:83cbccc482f4278c"}
13:31:52 [13:31:52.829]
INFO:
prover-client:proving-broker Proving Broker started
13:31:52 [13:31:52.830]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7462114f0b71017b epochNumber=1
{"provingJobId":"1:BASE_PARITY:7462114f0b71017b"}
13:31:52 [13:31:52.831]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpHCQ4f/1 with map size 1073741824
13:31:52 [13:31:52.834]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:52 [13:31:52.854]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:00ca19b3d1cbfa0e epochNumber=2
{"provingJobId":"1:BASE_PARITY:00ca19b3d1cbfa0e"}
13:31:52 [13:31:52.855]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpHCQ4f/2 with map size 1073741824
13:31:52 [13:31:52.859]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:53 [13:31:53.880]
INFO:
prover-client:proving-broker Proving Broker started
13:31:53 [13:31:53.886]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7462114f0b71017b settled=false
{"provingJobId":"1:BASE_PARITY:7462114f0b71017b","status":"pending"}
13:31:53 [13:31:53.891]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:00ca19b3d1cbfa0e settled=false
{"provingJobId":"1:BASE_PARITY:00ca19b3d1cbfa0e","status":"pending"}
13:31:53 [13:31:53.892]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7462114f0b71017b type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:7462114f0b71017b"}
13:31:53 [13:31:53.902]
INFO:
prover-client:proving-broker Proving Broker started
13:31:53 [13:31:53.903]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:03e6535f8444a67c epochNumber=1
{"provingJobId":"1:BASE_PARITY:03e6535f8444a67c"}
13:31:53 [13:31:53.905]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKKopos/1 with map size 1073741824
13:31:53 [13:31:53.906]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:53 [13:31:53.923]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e69f598bb32b7b73 epochNumber=2
{"provingJobId":"1:BASE_PARITY:e69f598bb32b7b73"}
13:31:53 [13:31:53.925]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testKKopos/2 with map size 1073741824
13:31:53 [13:31:53.927]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:54 [13:31:54.946]
INFO:
prover-client:proving-broker Proving Broker started
13:31:54 [13:31:54.948]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:03e6535f8444a67c settled=false
{"provingJobId":"1:BASE_PARITY:03e6535f8444a67c","status":"pending"}
13:31:54 [13:31:54.951]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e69f598bb32b7b73 settled=false
{"provingJobId":"1:BASE_PARITY:e69f598bb32b7b73","status":"pending"}
13:31:54 [13:31:54.953]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:03e6535f8444a67c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:03e6535f8444a67c"}
13:31:54 [13:31:54.953]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:03e6535f8444a67c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:03e6535f8444a67c"}
13:31:54 [13:31:54.962]
INFO:
prover-client:proving-broker Proving Broker started
13:31:54 [13:31:54.963]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:52731e36c07c7e0f epochNumber=1
{"provingJobId":"1:BASE_PARITY:52731e36c07c7e0f"}
13:31:54 [13:31:54.964]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmhzlFr/1 with map size 1073741824
13:31:54 [13:31:54.965]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:54 [13:31:54.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de1107f34c53cd3e epochNumber=2
{"provingJobId":"1:BASE_PARITY:de1107f34c53cd3e"}
13:31:54 [13:31:54.987]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testmhzlFr/2 with map size 1073741824
13:31:54 [13:31:54.991]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.009]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:52731e36c07c7e0f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:52731e36c07c7e0f"}
13:31:55 [13:31:55.017]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:de1107f34c53cd3e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:de1107f34c53cd3e"}
13:31:55 [13:31:55.025]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.026]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eede1ba2a1b89a76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:eede1ba2a1b89a76"}
13:31:55 [13:31:55.026]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMmaOBZ/1 with map size 1073741824
13:31:55 [13:31:55.028]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.043]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d1c43fdb6148f67 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1d1c43fdb6148f67"}
13:31:55 [13:31:55.044]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testMmaOBZ/2 with map size 1073741824
13:31:55 [13:31:55.046]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.063]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eede1ba2a1b89a76 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:eede1ba2a1b89a76"}
13:31:55 [13:31:55.064]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:eede1ba2a1b89a76 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:eede1ba2a1b89a76"}
13:31:55 [13:31:55.069]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1d1c43fdb6148f67 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1d1c43fdb6148f67"}
13:31:55 [13:31:55.080]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.081]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c25b01de596199be err=test error
{"provingJoId":"1:BASE_PARITY:c25b01de596199be"}
13:31:55 [13:31:55.090]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.092]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8010517ace6995a5 not found
{"provingJobId":"1:BASE_PARITY:8010517ace6995a5"}
13:31:55 [13:31:55.097]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.098]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:beebf356d066e9bb epochNumber=1
{"provingJobId":"1:BASE_PARITY:beebf356d066e9bb"}
13:31:55 [13:31:55.099]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgv50os/1 with map size 1073741824
13:31:55 [13:31:55.102]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.123]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.125]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5f0ba3871f9b1481 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5f0ba3871f9b1481"}
13:31:55 [13:31:55.128]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtqEWaw/1 with map size 1073741824
13:31:55 [13:31:55.130]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.253]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5f0ba3871f9b1481 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:5f0ba3871f9b1481"}
13:31:55 [13:31:55.258]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe349b1068a07d69 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe349b1068a07d69"}
13:31:55 [13:31:55.260]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMaJJxN/1 with map size 1073741824
13:31:55 [13:31:55.262]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.386]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fe349b1068a07d69 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:fe349b1068a07d69"}
13:31:55 [13:31:55.392]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f836cca5a4c78b28 epochNumber=10
{"provingJobId":"1:BASE_PARITY:f836cca5a4c78b28"}
13:31:55 [13:31:55.395]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testMaJJxN/10 with map size 1073741824
13:31:55 [13:31:55.397]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.439]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:55 [13:31:55.439]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
13:31:55 [13:31:55.552]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.554]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b720914eed42ed64 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b720914eed42ed64"}
13:31:55 [13:31:55.557]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqDHsiL/1 with map size 1073741824
13:31:55 [13:31:55.562]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.756]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b720914eed42ed64 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b720914eed42ed64"}
13:31:55 [13:31:55.795]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.796]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e9388a783f335df epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e9388a783f335df"}
13:31:55 [13:31:55.796]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuhlhe2/1 with map size 1073741824
13:31:55 [13:31:55.802]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.819]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0e9388a783f335df type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:0e9388a783f335df"}
13:31:55 [13:31:55.824]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.825]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:057e944d8a044c7f epochNumber=1
{"provingJobId":"1:BASE_PARITY:057e944d8a044c7f"}
13:31:55 [13:31:55.825]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test35U6aV/1 with map size 1073741824
13:31:55 [13:31:55.828]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.845]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:057e944d8a044c7f type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:057e944d8a044c7f"}
13:31:55 [13:31:55.846]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:057e944d8a044c7f type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:057e944d8a044c7f"}
13:31:55 [13:31:55.859]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.861]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e295bf1e6ddb1afc epochNumber=1
{"provingJobId":"1:BASE_PARITY:e295bf1e6ddb1afc"}
13:31:55 [13:31:55.862]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXTISbQ/1 with map size 1073741824
13:31:55 [13:31:55.864]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.890]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e295bf1e6ddb1afc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e295bf1e6ddb1afc"}
13:31:55 [13:31:55.904]
INFO:
prover-client:proving-broker Proving Broker started
13:31:55 [13:31:55.905]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c94352cdc1454649 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c94352cdc1454649"}
13:31:55 [13:31:55.906]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjmmWPd/1 with map size 1073741824
13:31:55 [13:31:55.909]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.931]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c94352cdc1454649 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c94352cdc1454649"}
13:31:55 [13:31:55.932]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ca9db14d03efc4d4 epochNumber=3
{"provingJobId":"1:BASE_PARITY:ca9db14d03efc4d4"}
13:31:55 [13:31:55.933]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testjmmWPd/3 with map size 1073741824
13:31:55 [13:31:55.935]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:55 [13:31:55.954]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:55 [13:31:55.955]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
13:31:55 [13:31:55.978]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c94352cdc1454649 err=test error
{"provingJoId":"1:BASE_PARITY:c94352cdc1454649"}
13:31:55 [13:31:55.982]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlArNFQ/1 with map size 1073741824
13:31:55 [13:31:55.990]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.011]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testlArNFQ/2 with map size 1073741824
13:31:56 [13:31:56.014]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.038]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.040]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6e4311a2cee86ae2 settled=false
{"provingJobId":"1:BASE_PARITY:6e4311a2cee86ae2","status":"pending"}
13:31:56 [13:31:56.042]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:28cf9b88696706d1 settled=false
{"provingJobId":"1:BASE_PARITY:28cf9b88696706d1","status":"pending"}
13:31:56 [13:31:56.049]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5nCsYL/1 with map size 1073741824
13:31:56 [13:31:56.052]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.073]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5nCsYL/2 with map size 1073741824
13:31:56 [13:31:56.076]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.124]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.126]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:01d7cd689e93a80c settled=true
{"provingJobId":"1:BASE_PARITY:01d7cd689e93a80c","status":"fulfilled"}
13:31:56 [13:31:56.127]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:fb5ccfa25e38c944 settled=true
{"provingJobId":"2:BASE_PARITY:fb5ccfa25e38c944","status":"fulfilled"}
13:31:56 [13:31:56.135]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrpcNXB/1 with map size 1073741824
13:31:56 [13:31:56.138]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.167]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrpcNXB/2 with map size 1073741824
13:31:56 [13:31:56.170]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.198]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.205]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ff8efffe49f826da settled=true
{"provingJobId":"1:BASE_PARITY:ff8efffe49f826da","status":"fulfilled"}
13:31:56 [13:31:56.209]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:49ef496866c88b7e settled=false
{"provingJobId":"1:BASE_PARITY:49ef496866c88b7e","status":"pending"}
13:31:56 [13:31:56.218]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.219]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:32f94e549848efaa epochNumber=1
{"provingJobId":"1:BASE_PARITY:32f94e549848efaa"}
13:31:56 [13:31:56.220]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYvGUQC/1 with map size 1073741824
13:31:56 [13:31:56.224]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.250]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.251]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b746c27055bfaf0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b746c27055bfaf0"}
13:31:56 [13:31:56.252]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:8b746c27055bfaf0: Error: db error: Error: db error
13:31:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:8b746c27055bfaf0"}
13:31:56 [13:31:56.260]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.261]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:719637c00f605330 epochNumber=1
{"provingJobId":"1:BASE_PARITY:719637c00f605330"}
13:31:56 [13:31:56.261]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnOtITX/1 with map size 1073741824
13:31:56 [13:31:56.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.290]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:719637c00f605330 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:719637c00f605330"}
13:31:56 [13:31:56.291]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:719637c00f605330 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:719637c00f605330"}
13:31:56 [13:31:56.305]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.306]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12a50ffdff9642d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:12a50ffdff9642d9"}
13:31:56 [13:31:56.308]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBIq4nM/1 with map size 1073741824
13:31:56 [13:31:56.311]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.338]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:12a50ffdff9642d9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:12a50ffdff9642d9"}
13:31:56 [13:31:56.339]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:12a50ffdff9642d9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:12a50ffdff9642d9"}
13:31:56 [13:31:56.340]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:12a50ffdff9642d9: Error: db error
13:31:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:12a50ffdff9642d9"}
13:31:56 [13:31:56.345]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.346]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:59d5468f3ef31da2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:59d5468f3ef31da2"}
13:31:56 [13:31:56.347]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNUn4Nu/1 with map size 1073741824
13:31:56 [13:31:56.354]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.377]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:59d5468f3ef31da2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:59d5468f3ef31da2"}
13:31:56 [13:31:56.377]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:59d5468f3ef31da2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:59d5468f3ef31da2"}
13:31:56 [13:31:56.394]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.395]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ee4a7fb0980a2202 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ee4a7fb0980a2202"}
13:31:56 [13:31:56.397]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOHqWW0/1 with map size 1073741824
13:31:56 [13:31:56.403]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.425]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ee4a7fb0980a2202 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ee4a7fb0980a2202"}
13:31:56 [13:31:56.426]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ee4a7fb0980a2202 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ee4a7fb0980a2202"}
13:31:56 [13:31:56.427]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:ee4a7fb0980a2202 jobErr=test error: Error: db error
13:31:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:ee4a7fb0980a2202"}
13:31:56 [13:31:56.433]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.434]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bf2557f4454ebdf5 not found
{"provingJobId":"1:BASE_PARITY:bf2557f4454ebdf5"}
13:31:56 [13:31:56.439]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.439]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:778d44e6d1c2b9b4 err=test error
{"provingJoId":"1:BASE_PARITY:778d44e6d1c2b9b4"}
13:31:56 [13:31:56.446]
INFO:
prover-client:proving-broker Proving Broker started
13:31:56 [13:31:56.457]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:22bea727632743dc epochNumber=1
{"provingJobId":"1:BASE_PARITY:22bea727632743dc"}
13:31:56 [13:31:56.458]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdvuIdv/1 with map size 1073741824
13:31:56 [13:31:56.464]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.491]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:22bea727632743dc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:22bea727632743dc"}
13:31:56 [13:31:56.491]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:22bea727632743dc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:22bea727632743dc"}
13:31:56 [13:31:56.508]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:1a91fa2410a5dd47 epochNumber=2
{"provingJobId":"2:BASE_PARITY:1a91fa2410a5dd47"}
13:31:56 [13:31:56.510]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testdvuIdv/2 with map size 1073741824
13:31:56 [13:31:56.513]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.535]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:1a91fa2410a5dd47 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:1a91fa2410a5dd47"}
13:31:56 [13:31:56.535]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:1a91fa2410a5dd47 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:1a91fa2410a5dd47"}
13:31:56 [13:31:56.604]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:ff7fa4c044d857b6 epochNumber=3
{"provingJobId":"3:BASE_PARITY:ff7fa4c044d857b6"}
13:31:56 [13:31:56.605]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testdvuIdv/3 with map size 1073741824
13:31:56 [13:31:56.609]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.650]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:56 [13:31:56.651]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
13:31:56 [13:31:56.687]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:d5b73a93d3443fc5 epochNumber=4
{"provingJobId":"4:BASE_PARITY:d5b73a93d3443fc5"}
13:31:56 [13:31:56.688]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testdvuIdv/4 with map size 1073741824
13:31:56 [13:31:56.695]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.731]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:56 [13:31:56.732]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
13:31:56 [13:31:56.777]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:199760a26f49432a epochNumber=5
{"provingJobId":"5:BASE_PARITY:199760a26f49432a"}
13:31:56 [13:31:56.779]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testdvuIdv/5 with map size 1073741824
13:31:56 [13:31:56.782]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
13:31:56 [13:31:56.810]
INFO:
prover-client:proving-broker Cleaned up jobs=1
13:31:56 [13:31:56.811]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
13:31:56
PASS src/proving_broker/proving_broker.test.ts (
13.476 s)
13:31:56 ProvingBroker
13:31:56 Producer API
13:31:56
✓ refuses stale jobs (52 ms)
13:31:56
✓ enqueues jobs (5 ms)
13:31:56
✓ ignores duplicate jobs (4 ms)
13:31:56
✓ reports correct status when enqueuing repeat jobs (5 ms)
13:31:56
✓ throws an error in case of duplicate job IDs (33 ms)
13:31:56
✓ returns not-found status for non-existing jobs (1 ms)
13:31:56
✓ cancels jobs in queue (2 ms)
13:31:56
✓ cancels jobs in-progress (1 ms)
13:31:56
✓ returns job result if successful (2 ms)
13:31:56
✓ returns job error if failed (2 ms)
13:31:56
✓ correctly returns job status for concurrent writes (2 ms)
13:31:56
✓ refuses stale jobs (43 ms)
13:31:56
✓ enqueues jobs (33 ms)
13:31:56
✓ ignores duplicate jobs (31 ms)
13:31:56
✓ reports correct status when enqueuing repeat jobs (42 ms)
13:31:56
✓ throws an error in case of duplicate job IDs (35 ms)
13:31:56
✓ returns not-found status for non-existing jobs (6 ms)
13:31:56
✓ cancels jobs in queue (35 ms)
13:31:56
✓ cancels jobs in-progress (31 ms)
13:31:56
✓ returns job result if successful (40 ms)
13:31:56
✓ returns job error if failed (30 ms)
13:31:56
✓ correctly returns job status for concurrent writes (53 ms)
13:31:56 Consumer API
13:31:56
✓ returns undefined if no jobs are available (1 ms)
13:31:56
✓ returns jobs in priority order (1 ms)
13:31:56
✓ returns undefined if no jobs are available for the given allowList (1 ms)
13:31:56
✓ returns a job if it is in the allowList (2 ms)
13:31:56
✓ returns the most important job if it is in the allowList (3 ms)
13:31:56
✓ returns any job if filter is empty (2 ms)
13:31:56
✓ returns a new job when reporting job success (3 ms)
13:31:56
✓ returns a new job when reporting permanent error (2 ms)
13:31:56
✓ returns a new job when reporting retry-able error (2 ms)
13:31:56
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
13:31:56
✓ returns a new job if job is already in progress elsewhere (1056 ms)
13:31:56
✓ avoids sending the same job to a new agent after a restart (1013 ms)
13:31:56
✓ avoids sending a completed job to a new agent after a restart (1010 ms)
13:31:56
✓ tracks job result if in progress (3 ms)
13:31:56
✓ tracks job result even if job is in queue (3 ms)
13:31:56
✓ ignores reported job error if unknown job (2 ms)
13:31:56
✓ ignores job result if unknown job (1 ms)
13:31:56
✓ returns undefined if no jobs are available (4 ms)
13:31:56
✓ returns jobs in priority order (46 ms)
13:31:56
✓ returns undefined if no jobs are available for the given allowList (28 ms)
13:31:56
✓ returns a job if it is in the allowList (82 ms)
13:31:56
✓ returns the most important job if it is in the allowList (62 ms)
13:31:56
✓ returns any job if filter is empty (56 ms)
13:31:56
✓ returns a new job when reporting job success (39 ms)
13:31:56
✓ returns a new job when reporting permanent error (35 ms)
13:31:56
✓ returns a new job when reporting retry-able error (29 ms)
13:31:56
✓ returns a new job when reporting progress if current one is cancelled (37 ms)
13:31:56
✓ returns a new job if job is already in progress elsewhere (1105 ms)
13:31:56
✓ avoids sending the same job to a new agent after a restart (1072 ms)
13:31:56
✓ avoids sending a completed job to a new agent after a restart (1063 ms)
13:31:56
✓ tracks job result if in progress (62 ms)
13:31:56
✓ tracks job result even if job is in queue (53 ms)
13:31:56
✓ ignores reported job error if unknown job (4 ms)
13:31:56
✓ ignores job result if unknown job (7 ms)
13:31:56 Timeouts
13:31:56
✓ tracks in progress jobs (2 ms)
13:31:56
✓ re-enqueues jobs that time out (109 ms)
13:31:56
✓ cancel stale jobs that time out (236 ms)
13:31:56
✓ keeps the jobs in progress while it is alive (203 ms)
13:31:56
✓ tracks in progress jobs (27 ms)
13:31:56
✓ re-enqueues jobs that time out (135 ms)
13:31:56
✓ cancel stale jobs that time out (293 ms)
13:31:56
✓ keeps the jobs in progress while it is alive (243 ms)
13:31:56 Retries
13:31:56
✓ retries jobs (3 ms)
13:31:56
✓ retries up to a maximum number of times (3 ms)
13:31:56
✓ passing retry=false does not retry (7 ms)
13:31:56
✓ does not retry if job is stale (31 ms)
13:31:56
✓ retries jobs (27 ms)
13:31:56
✓ retries up to a maximum number of times (33 ms)
13:31:56
✓ passing retry=false does not retry (44 ms)
13:31:56
✓ does not retry if job is stale (81 ms)
13:31:56 Database management
13:31:56
✓ re-enqueues proof requests on start (3 ms)
13:31:56
✓ restores proof results on start (2 ms)
13:31:56
✓ only re-enqueues unfinished jobs (2 ms)
13:31:56
✓ saves job when enqueued (3 ms)
13:31:56
✓ does not retain job if database fails to save (2 ms)
13:31:56
✓ saves job result (2 ms)
13:31:56
✓ saves result even if database fails to save (4 ms)
13:31:56
✓ saves job error (3 ms)
13:31:56
✓ saves job error even if database fails to save (2 ms)
13:31:56
✓ does not save job result if job is unknown (1 ms)
13:31:56
✓ does not save job error if job is unknown (2 ms)
13:31:56
✓ cleans up old jobs periodically (261 ms)
13:31:56
✓ re-enqueues proof requests on start (67 ms)
13:31:56
✓ restores proof results on start (83 ms)
13:31:56
✓ only re-enqueues unfinished jobs (82 ms)
13:31:56
✓ saves job when enqueued (32 ms)
13:31:56
✓ does not retain job if database fails to save (6 ms)
13:31:56
✓ saves job result (49 ms)
13:31:56
✓ saves result even if database fails to save (39 ms)
13:31:56
✓ saves job error (48 ms)
13:31:56
✓ saves job error even if database fails to save (38 ms)
13:31:56
✓ does not save job result if job is unknown (5 ms)
13:31:56
✓ does not save job error if job is unknown (5 ms)
13:31:56
✓ cleans up old jobs periodically (416 ms)
13:31:56
13:31:56
Test Suites: 1 passed, 1 total
13:31:56
Tests: 96 passed, 96 total
13:31:56
Snapshots: 0 total
13:31:56
Time: 13.581 s
13:31:56
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
13:31:56
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?