Parent Log:
http://ci.aztec-labs.com/345c5ed524d6c3dd
Command: 557c108a8ed544a3 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/ed4f6d71e2ea98f43186114eb914d54b87dd1183
Env: REF_NAME=gh-readonly-queue/next/pr-15116-433ba6218aed45169029991a493e4232b30b8ec6 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Tue Jun 17 19:34:21 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15116_amd64_x3-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
19:34:25 [19:34:25.863]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.870]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cadd7e29f1ffe03d epochNumber=42
{"provingJobId":"1:BASE_PARITY:cadd7e29f1ffe03d"}
19:34:25 [19:34:25.873]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:3e3c03551e2d0e2b epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e3c03551e2d0e2b"}
19:34:25 [19:34:25.899]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.900]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae6461568adcc635 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae6461568adcc635"}
19:34:25 [19:34:25.902]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:50e8534a156970d1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:50e8534a156970d1"}
19:34:25 [19:34:25.904]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.905]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5e382465b476d26b epochNumber=1
{"provingJobId":"1:BASE_PARITY:5e382465b476d26b"}
19:34:25 [19:34:25.906]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:5e382465b476d26b epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:5e382465b476d26b"}
19:34:25 [19:34:25.908]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.909]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a978eff26c0bf2fb epochNumber=1
{"provingJobId":"1:BASE_PARITY:a978eff26c0bf2fb"}
19:34:25 [19:34:25.910]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a978eff26c0bf2fb epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a978eff26c0bf2fb"}
19:34:25 [19:34:25.911]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a978eff26c0bf2fb type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a978eff26c0bf2fb"}
19:34:25 [19:34:25.912]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a978eff26c0bf2fb epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a978eff26c0bf2fb"}
19:34:25 [19:34:25.913]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.914]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dda446226ac4c9df epochNumber=1
{"provingJobId":"1:BASE_PARITY:dda446226ac4c9df"}
19:34:25 [19:34:25.944]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.946]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.947]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81f3b4961925efbf epochNumber=1
{"provingJobId":"1:BASE_PARITY:81f3b4961925efbf"}
19:34:25 [19:34:25.948]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:81f3b4961925efbf
{"provingJobId":"1:BASE_PARITY:81f3b4961925efbf"}
19:34:25 [19:34:25.948]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:81f3b4961925efbf type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:81f3b4961925efbf"}
19:34:25 [19:34:25.948]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:81f3b4961925efbf type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:81f3b4961925efbf"}
19:34:25 [19:34:25.950]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.951]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f955f57ba942db87 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f955f57ba942db87"}
19:34:25 [19:34:25.951]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:f955f57ba942db87
{"provingJobId":"1:BASE_PARITY:f955f57ba942db87"}
19:34:25 [19:34:25.952]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f955f57ba942db87 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:f955f57ba942db87"}
19:34:25 [19:34:25.953]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:efa25a954c27148e epochNumber=1
{"provingJobId":"1:BASE_PARITY:efa25a954c27148e"}
19:34:25 [19:34:25.955]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:efa25a954c27148e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:efa25a954c27148e"}
19:34:25 [19:34:25.955]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:efa25a954c27148e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:efa25a954c27148e"}
19:34:25 [19:34:25.957]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.957]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e109b6aa04e82070 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e109b6aa04e82070"}
19:34:25 [19:34:25.958]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e109b6aa04e82070 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e109b6aa04e82070"}
19:34:25 [19:34:25.958]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e109b6aa04e82070 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e109b6aa04e82070"}
19:34:25 [19:34:25.959]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60838ba5a757c1e6 epochNumber=0
{"provingJobId":"1:BASE_PARITY:60838ba5a757c1e6"}
19:34:25 [19:34:25.960]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:60838ba5a757c1e6 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:60838ba5a757c1e6"}
19:34:25 [19:34:25.960]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57951cb87676f1bd epochNumber=0
{"provingJobId":"1:BASE_PARITY:57951cb87676f1bd"}
19:34:25 [19:34:25.960]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:60838ba5a757c1e6 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:60838ba5a757c1e6"}
19:34:25 [19:34:25.961]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:970d23253c70709e epochNumber=0
{"provingJobId":"1:BASE_PARITY:970d23253c70709e"}
19:34:25 [19:34:25.962]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.970]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.971]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:900c20c445fa3a1f epochNumber=2
{"provingJobId":"1:BASE_PARITY:900c20c445fa3a1f"}
19:34:25 [19:34:25.971]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d6558a0d276eba3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d6558a0d276eba3"}
19:34:25 [19:34:25.973]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.973]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac2b2d001503eb31 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ac2b2d001503eb31"}
19:34:25 [19:34:25.974]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.975]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1e0dbfa89179067c epochNumber=1
{"provingJobId":"1:BASE_PARITY:1e0dbfa89179067c"}
19:34:25 [19:34:25.975]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2a35a3d72043fa3b epochNumber=1
{"provingJobId":"1:BASE_PARITY:2a35a3d72043fa3b"}
19:34:25 [19:34:25.975]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b90b7a87ef294742 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b90b7a87ef294742"}
19:34:25 [19:34:25.976]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0968a9a9027fb70 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0968a9a9027fb70"}
19:34:25 [19:34:25.977]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.977]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:86a0bcaaa982e341 epochNumber=1
{"provingJobId":"1:BASE_PARITY:86a0bcaaa982e341"}
19:34:25 [19:34:25.977]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c98bcfc30e464297 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c98bcfc30e464297"}
19:34:25 [19:34:25.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a31d79b50cd35615 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a31d79b50cd35615"}
19:34:25 [19:34:25.978]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f5885161f91b10fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:f5885161f91b10fe"}
19:34:25 [19:34:25.979]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.979]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:17aaa902937e1e99 epochNumber=1
{"provingJobId":"1:BASE_PARITY:17aaa902937e1e99"}
19:34:25 [19:34:25.979]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3afd9759404b9e50 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3afd9759404b9e50"}
19:34:25 [19:34:25.980]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d4d0115d1291f9ad epochNumber=2
{"provingJobId":"1:BASE_PARITY:d4d0115d1291f9ad"}
19:34:25 [19:34:25.980]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6b1e45902ea4d50c epochNumber=1
{"provingJobId":"1:BASE_PARITY:6b1e45902ea4d50c"}
19:34:25 [19:34:25.981]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.981]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ad2d1588f3c131d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ad2d1588f3c131d9"}
19:34:25 [19:34:25.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:315bf96e77d9a69e epochNumber=1
{"provingJobId":"1:BASE_PARITY:315bf96e77d9a69e"}
19:34:25 [19:34:25.982]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ad2d1588f3c131d9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ad2d1588f3c131d9"}
19:34:25 [19:34:25.983]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.984]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b59020ebb0339041 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b59020ebb0339041"}
19:34:25 [19:34:25.984]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e2e18931378ff215 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e2e18931378ff215"}
19:34:25 [19:34:25.984]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b59020ebb0339041 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:b59020ebb0339041"}
19:34:25 [19:34:25.985]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.986]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cbb7c21332da2dd7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cbb7c21332da2dd7"}
19:34:25 [19:34:25.986]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:59c664aab75722a8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:59c664aab75722a8"}
19:34:25 [19:34:25.986]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:cbb7c21332da2dd7 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:cbb7c21332da2dd7"}
19:34:25 [19:34:25.987]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.988]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a271f54cb4101a87 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a271f54cb4101a87"}
19:34:25 [19:34:25.988]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:a271f54cb4101a87
{"provingJobId":"1:BASE_PARITY:a271f54cb4101a87"}
19:34:25 [19:34:25.988]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a271f54cb4101a87 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:a271f54cb4101a87"}
19:34:25 [19:34:25.989]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf290d7007466e01 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bf290d7007466e01"}
19:34:25 [19:34:25.989]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a271f54cb4101a87 has already been completed
{"provingJobId":"1:BASE_PARITY:a271f54cb4101a87"}
19:34:25 [19:34:25.990]
INFO:
prover-client:proving-broker Proving Broker started
19:34:25 [19:34:25.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ccf3c1430e4ae31 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3ccf3c1430e4ae31"}
19:34:25 [19:34:25.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7f117880507b816 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b7f117880507b816"}
19:34:27 [19:34:27.043]
INFO:
prover-client:proving-broker Proving Broker started
19:34:27 [19:34:27.044]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3ccf3c1430e4ae31 settled=false
{"provingJobId":"1:BASE_PARITY:3ccf3c1430e4ae31","status":"pending"}
19:34:27 [19:34:27.045]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b7f117880507b816 settled=false
{"provingJobId":"1:BASE_PARITY:b7f117880507b816","status":"pending"}
19:34:27 [19:34:27.046]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3ccf3c1430e4ae31 type=BASE_PARITY startedAt=1750188865991 older agent has taken job
{"provingJobId":"1:BASE_PARITY:3ccf3c1430e4ae31"}
19:34:27 [19:34:27.047]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3ccf3c1430e4ae31 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:3ccf3c1430e4ae31"}
19:34:27 [19:34:27.049]
INFO:
prover-client:proving-broker Proving Broker started
19:34:27 [19:34:27.050]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d00b0cf3bf0fae0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d00b0cf3bf0fae0"}
19:34:27 [19:34:27.051]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:70327f80a339b09b epochNumber=2
{"provingJobId":"1:BASE_PARITY:70327f80a339b09b"}
19:34:28 [19:34:28.052]
INFO:
prover-client:proving-broker Proving Broker started
19:34:28 [19:34:28.052]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0d00b0cf3bf0fae0 settled=false
{"provingJobId":"1:BASE_PARITY:0d00b0cf3bf0fae0","status":"pending"}
19:34:28 [19:34:28.053]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:70327f80a339b09b settled=false
{"provingJobId":"1:BASE_PARITY:70327f80a339b09b","status":"pending"}
19:34:28 [19:34:28.053]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0d00b0cf3bf0fae0 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:0d00b0cf3bf0fae0"}
19:34:28 [19:34:28.054]
INFO:
prover-client:proving-broker Proving Broker started
19:34:28 [19:34:28.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e45e2a58739daa2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e45e2a58739daa2"}
19:34:28 [19:34:28.055]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ebc40a062f7de3c0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ebc40a062f7de3c0"}
19:34:29 [19:34:29.056]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.056]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3e45e2a58739daa2 settled=false
{"provingJobId":"1:BASE_PARITY:3e45e2a58739daa2","status":"pending"}
19:34:29 [19:34:29.057]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ebc40a062f7de3c0 settled=false
{"provingJobId":"1:BASE_PARITY:ebc40a062f7de3c0","status":"pending"}
19:34:29 [19:34:29.057]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3e45e2a58739daa2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3e45e2a58739daa2"}
19:34:29 [19:34:29.057]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3e45e2a58739daa2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3e45e2a58739daa2"}
19:34:29 [19:34:29.059]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.059]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4ec278e38a65e51a epochNumber=1
{"provingJobId":"1:BASE_PARITY:4ec278e38a65e51a"}
19:34:29 [19:34:29.059]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a51adc652344504 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3a51adc652344504"}
19:34:29 [19:34:29.060]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4ec278e38a65e51a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4ec278e38a65e51a"}
19:34:29 [19:34:29.060]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3a51adc652344504 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3a51adc652344504"}
19:34:29 [19:34:29.062]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.062]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3099b0b1c341fbcc epochNumber=1
{"provingJobId":"1:BASE_PARITY:3099b0b1c341fbcc"}
19:34:29 [19:34:29.063]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2edd9374c8f29bc0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:2edd9374c8f29bc0"}
19:34:29 [19:34:29.063]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3099b0b1c341fbcc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3099b0b1c341fbcc"}
19:34:29 [19:34:29.063]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3099b0b1c341fbcc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3099b0b1c341fbcc"}
19:34:29 [19:34:29.063]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2edd9374c8f29bc0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2edd9374c8f29bc0"}
19:34:29 [19:34:29.065]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.065]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:038d4855af6bc478 err=test error
{"provingJoId":"1:BASE_PARITY:038d4855af6bc478"}
19:34:29 [19:34:29.066]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.067]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8a38ea02a392768e not found
{"provingJobId":"1:BASE_PARITY:8a38ea02a392768e"}
19:34:29 [19:34:29.068]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.069]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9007d984116f0c50 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9007d984116f0c50"}
19:34:29 [19:34:29.070]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.071]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c3521293dcd0f4f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c3521293dcd0f4f5"}
19:34:29 [19:34:29.173]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c3521293dcd0f4f5 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:c3521293dcd0f4f5"}
19:34:29 [19:34:29.180]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.181]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a84f30b2017f47cc epochNumber=1
{"provingJobId":"1:BASE_PARITY:a84f30b2017f47cc"}
19:34:29 [19:34:29.282]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a84f30b2017f47cc timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:a84f30b2017f47cc"}
19:34:29 [19:34:29.284]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3dc903c309918ba8 epochNumber=10
{"provingJobId":"1:BASE_PARITY:3dc903c309918ba8"}
19:34:29 [19:34:29.310]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:29 [19:34:29.411]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.412]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9186aa74c0e9207c epochNumber=1
{"provingJobId":"1:BASE_PARITY:9186aa74c0e9207c"}
19:34:29 [19:34:29.564]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9186aa74c0e9207c timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:9186aa74c0e9207c"}
19:34:29 [19:34:29.615]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.616]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7c47df387f42e0a epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7c47df387f42e0a"}
19:34:29 [19:34:29.618]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a7c47df387f42e0a type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a7c47df387f42e0a"}
19:34:29 [19:34:29.619]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.620]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:012b5eef3f82993f epochNumber=1
{"provingJobId":"1:BASE_PARITY:012b5eef3f82993f"}
19:34:29 [19:34:29.620]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:012b5eef3f82993f type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:012b5eef3f82993f"}
19:34:29 [19:34:29.621]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:012b5eef3f82993f type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:012b5eef3f82993f"}
19:34:29 [19:34:29.622]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.623]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:542360fa55f35813 epochNumber=1
{"provingJobId":"1:BASE_PARITY:542360fa55f35813"}
19:34:29 [19:34:29.623]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:542360fa55f35813 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:542360fa55f35813"}
19:34:29 [19:34:29.624]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.624]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d9cb988e509a9c7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3d9cb988e509a9c7"}
19:34:29 [19:34:29.625]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3d9cb988e509a9c7 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3d9cb988e509a9c7"}
19:34:29 [19:34:29.625]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:15f2c57e6787b7df epochNumber=3
{"provingJobId":"1:BASE_PARITY:15f2c57e6787b7df"}
19:34:29 [19:34:29.649]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:29 [19:34:29.651]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:3d9cb988e509a9c7 err=test error
{"provingJoId":"1:BASE_PARITY:3d9cb988e509a9c7"}
19:34:29 [19:34:29.653]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.653]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2bbaab9cfb555f83 settled=false
{"provingJobId":"1:BASE_PARITY:2bbaab9cfb555f83","status":"pending"}
19:34:29 [19:34:29.653]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b3bbd38a5dfa483f settled=false
{"provingJobId":"1:BASE_PARITY:b3bbd38a5dfa483f","status":"pending"}
19:34:29 [19:34:29.656]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.656]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:356e4764ac9cefdb settled=true
{"provingJobId":"1:BASE_PARITY:356e4764ac9cefdb","status":"fulfilled"}
19:34:29 [19:34:29.656]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:1dae162c0cd5ad30 settled=true
{"provingJobId":"2:BASE_PARITY:1dae162c0cd5ad30","status":"fulfilled"}
19:34:29 [19:34:29.657]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.657]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8eb2cf7b65980fd9 settled=true
{"provingJobId":"1:BASE_PARITY:8eb2cf7b65980fd9","status":"fulfilled"}
19:34:29 [19:34:29.658]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d436d4b6cbc74243 settled=false
{"provingJobId":"1:BASE_PARITY:d436d4b6cbc74243","status":"pending"}
19:34:29 [19:34:29.659]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb746fb58fb11e6b epochNumber=1
{"provingJobId":"1:BASE_PARITY:bb746fb58fb11e6b"}
19:34:29 [19:34:29.662]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.662]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c4b10f998cc4a2ec epochNumber=1
{"provingJobId":"1:BASE_PARITY:c4b10f998cc4a2ec"}
19:34:29 [19:34:29.663]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:c4b10f998cc4a2ec: Error: db error: Error: db error
19:34:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:c4b10f998cc4a2ec"}
19:34:29 [19:34:29.664]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.665]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:189d9a4577ab9fd6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:189d9a4577ab9fd6"}
19:34:29 [19:34:29.665]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:189d9a4577ab9fd6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:189d9a4577ab9fd6"}
19:34:29 [19:34:29.665]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:189d9a4577ab9fd6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:189d9a4577ab9fd6"}
19:34:29 [19:34:29.666]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.667]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:af163c4e877be456 epochNumber=1
{"provingJobId":"1:BASE_PARITY:af163c4e877be456"}
19:34:29 [19:34:29.667]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:af163c4e877be456 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:af163c4e877be456"}
19:34:29 [19:34:29.667]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:af163c4e877be456 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:af163c4e877be456"}
19:34:29 [19:34:29.668]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:af163c4e877be456: Error: db error
19:34:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:af163c4e877be456"}
19:34:29 [19:34:29.669]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.669]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a5a60c8bcf3f8792 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a5a60c8bcf3f8792"}
19:34:29 [19:34:29.670]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a5a60c8bcf3f8792 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a5a60c8bcf3f8792"}
19:34:29 [19:34:29.670]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a5a60c8bcf3f8792 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a5a60c8bcf3f8792"}
19:34:29 [19:34:29.671]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.672]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9031480b1b302dfa epochNumber=1
{"provingJobId":"1:BASE_PARITY:9031480b1b302dfa"}
19:34:29 [19:34:29.672]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9031480b1b302dfa type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9031480b1b302dfa"}
19:34:29 [19:34:29.672]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9031480b1b302dfa type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9031480b1b302dfa"}
19:34:29 [19:34:29.673]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:9031480b1b302dfa jobErr=test error: Error: db error
19:34:29 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:9031480b1b302dfa"}
19:34:29 [19:34:29.674]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.674]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7cf3dbdd77af41b6 not found
{"provingJobId":"1:BASE_PARITY:7cf3dbdd77af41b6"}
19:34:29 [19:34:29.675]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.676]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1cb806086b46699d err=test error
{"provingJoId":"1:BASE_PARITY:1cb806086b46699d"}
19:34:29 [19:34:29.677]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.687]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2465620b8257e1fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:2465620b8257e1fe"}
19:34:29 [19:34:29.688]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2465620b8257e1fe type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2465620b8257e1fe"}
19:34:29 [19:34:29.689]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2465620b8257e1fe type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2465620b8257e1fe"}
19:34:29 [19:34:29.699]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:4558adb72eebc716 epochNumber=2
{"provingJobId":"2:BASE_PARITY:4558adb72eebc716"}
19:34:29 [19:34:29.700]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:4558adb72eebc716 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:4558adb72eebc716"}
19:34:29 [19:34:29.700]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:4558adb72eebc716 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:4558adb72eebc716"}
19:34:29 [19:34:29.762]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:9930e040631fb8be epochNumber=3
{"provingJobId":"3:BASE_PARITY:9930e040631fb8be"}
19:34:29 [19:34:29.779]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:29 [19:34:29.824]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:0eb76c87f5ca734a epochNumber=4
{"provingJobId":"4:BASE_PARITY:0eb76c87f5ca734a"}
19:34:29 [19:34:29.831]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:29 [19:34:29.886]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:f5eefd1a23724cc2 epochNumber=5
{"provingJobId":"5:BASE_PARITY:f5eefd1a23724cc2"}
19:34:29 [19:34:29.908]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:29 [19:34:29.944]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.945]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a818396344424484 epochNumber=42
{"provingJobId":"1:BASE_PARITY:a818396344424484"}
19:34:29 [19:34:29.947]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-test724K6D/42 with map size 1073741824
19:34:29 [19:34:29.953]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:29 [19:34:29.981]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:8b8a7dfc80fb2bf1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b8a7dfc80fb2bf1"}
19:34:29 [19:34:29.990]
INFO:
prover-client:proving-broker Proving Broker started
19:34:29 [19:34:29.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8fbc086664b863fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:8fbc086664b863fd"}
19:34:29 [19:34:29.992]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5wbxS1/1 with map size 1073741824
19:34:29 [19:34:29.994]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.016]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57ceb024ca6863f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:57ceb024ca6863f9"}
19:34:30 [19:34:30.026]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.027]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:82681cb4d0571a3f epochNumber=1
{"provingJobId":"1:BASE_PARITY:82681cb4d0571a3f"}
19:34:30 [19:34:30.028]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test78SmLV/1 with map size 1073741824
19:34:30 [19:34:30.032]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.053]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:82681cb4d0571a3f epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:82681cb4d0571a3f"}
19:34:30 [19:34:30.057]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.058]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cf89c813e6bbe8c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cf89c813e6bbe8c0"}
19:34:30 [19:34:30.058]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2KIGa8/1 with map size 1073741824
19:34:30 [19:34:30.062]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.082]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:cf89c813e6bbe8c0 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:cf89c813e6bbe8c0"}
19:34:30 [19:34:30.083]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:cf89c813e6bbe8c0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:cf89c813e6bbe8c0"}
19:34:30 [19:34:30.090]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:cf89c813e6bbe8c0 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:cf89c813e6bbe8c0"}
19:34:30 [19:34:30.095]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.096]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eabe93638913c317 epochNumber=1
{"provingJobId":"1:BASE_PARITY:eabe93638913c317"}
19:34:30 [19:34:30.097]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZ7b6yc/1 with map size 1073741824
19:34:30 [19:34:30.099]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.127]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.132]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.132]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c5d4ba4e4dcaf7e epochNumber=1
{"provingJobId":"1:BASE_PARITY:4c5d4ba4e4dcaf7e"}
19:34:30 [19:34:30.134]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWUVNFk/1 with map size 1073741824
19:34:30 [19:34:30.138]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.158]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:4c5d4ba4e4dcaf7e
{"provingJobId":"1:BASE_PARITY:4c5d4ba4e4dcaf7e"}
19:34:30 [19:34:30.159]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4c5d4ba4e4dcaf7e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4c5d4ba4e4dcaf7e"}
19:34:30 [19:34:30.159]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4c5d4ba4e4dcaf7e type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:4c5d4ba4e4dcaf7e"}
19:34:30 [19:34:30.169]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.170]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5035d0c8e61e1ed2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5035d0c8e61e1ed2"}
19:34:30 [19:34:30.170]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqxjfFB/1 with map size 1073741824
19:34:30 [19:34:30.172]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.190]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5035d0c8e61e1ed2
{"provingJobId":"1:BASE_PARITY:5035d0c8e61e1ed2"}
19:34:30 [19:34:30.191]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5035d0c8e61e1ed2 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:5035d0c8e61e1ed2"}
19:34:30 [19:34:30.200]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.201]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:25f2bd2e575d2df9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:25f2bd2e575d2df9"}
19:34:30 [19:34:30.202]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMrjAC1/1 with map size 1073741824
19:34:30 [19:34:30.207]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.224]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:25f2bd2e575d2df9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:25f2bd2e575d2df9"}
19:34:30 [19:34:30.225]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:25f2bd2e575d2df9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:25f2bd2e575d2df9"}
19:34:30 [19:34:30.233]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.233]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2903547f423ec9d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2903547f423ec9d7"}
19:34:30 [19:34:30.234]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testeOJE4u/1 with map size 1073741824
19:34:30 [19:34:30.236]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.252]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2903547f423ec9d7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2903547f423ec9d7"}
19:34:30 [19:34:30.253]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2903547f423ec9d7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2903547f423ec9d7"}
19:34:30 [19:34:30.264]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.264]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:978d2568a371b791 epochNumber=0
{"provingJobId":"1:BASE_PARITY:978d2568a371b791"}
19:34:30 [19:34:30.266]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testYq2tK8/0 with map size 1073741824
19:34:30 [19:34:30.271]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.294]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:978d2568a371b791 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:978d2568a371b791"}
19:34:30 [19:34:30.296]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:820b74164ea00681 epochNumber=0
{"provingJobId":"1:BASE_PARITY:820b74164ea00681"}
19:34:30 [19:34:30.296]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:978d2568a371b791 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:978d2568a371b791"}
19:34:30 [19:34:30.297]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9a0245242b143a16 epochNumber=0
{"provingJobId":"1:BASE_PARITY:9a0245242b143a16"}
19:34:30 [19:34:30.316]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.320]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.321]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7453d505a2017f95 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7453d505a2017f95"}
19:34:30 [19:34:30.321]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqJgwwW/2 with map size 1073741824
19:34:30 [19:34:30.324]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4adf60c5c96557a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4adf60c5c96557a3"}
19:34:30 [19:34:30.341]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqJgwwW/1 with map size 1073741824
19:34:30 [19:34:30.343]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.360]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.361]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dda8c6cc411fd3be epochNumber=1
{"provingJobId":"1:BASE_PARITY:dda8c6cc411fd3be"}
19:34:30 [19:34:30.361]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMDgEr0/1 with map size 1073741824
19:34:30 [19:34:30.363]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.380]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.380]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e26b11a65f1eeb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3e26b11a65f1eeb1"}
19:34:30 [19:34:30.381]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdee2L9/1 with map size 1073741824
19:34:30 [19:34:30.383]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4ea5abe052dc8cd epochNumber=1
{"provingJobId":"1:BASE_PARITY:e4ea5abe052dc8cd"}
19:34:30 [19:34:30.410]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49f07d5b34c1d262 epochNumber=2
{"provingJobId":"1:BASE_PARITY:49f07d5b34c1d262"}
19:34:30 [19:34:30.411]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testdee2L9/2 with map size 1073741824
19:34:30 [19:34:30.414]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.431]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa0fc4cdd844dab5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa0fc4cdd844dab5"}
19:34:30 [19:34:30.442]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.443]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:23750443cca5c173 epochNumber=1
{"provingJobId":"1:BASE_PARITY:23750443cca5c173"}
19:34:30 [19:34:30.444]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testxeB6MA/1 with map size 1073741824
19:34:30 [19:34:30.446]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.463]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a7bc9a70399a9c9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5a7bc9a70399a9c9"}
19:34:30 [19:34:30.470]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c09ba69ddaac8156 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c09ba69ddaac8156"}
19:34:30 [19:34:30.471]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testxeB6MA/2 with map size 1073741824
19:34:30 [19:34:30.473]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.489]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3b94349938e9fe86 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3b94349938e9fe86"}
19:34:30 [19:34:30.505]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.506]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c52750edcf15ddfc epochNumber=1
{"provingJobId":"1:BASE_PARITY:c52750edcf15ddfc"}
19:34:30 [19:34:30.508]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXsJMGN/1 with map size 1073741824
19:34:30 [19:34:30.509]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.528]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71ed7f8579323e27 epochNumber=1
{"provingJobId":"1:BASE_PARITY:71ed7f8579323e27"}
19:34:30 [19:34:30.534]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eafc0e4ce2034814 epochNumber=2
{"provingJobId":"1:BASE_PARITY:eafc0e4ce2034814"}
19:34:30 [19:34:30.535]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXsJMGN/2 with map size 1073741824
19:34:30 [19:34:30.538]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.558]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51a7e1999c9a741c epochNumber=1
{"provingJobId":"1:BASE_PARITY:51a7e1999c9a741c"}
19:34:30 [19:34:30.570]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.570]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2ee96276c52f23da epochNumber=1
{"provingJobId":"1:BASE_PARITY:2ee96276c52f23da"}
19:34:30 [19:34:30.571]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmZl4iR/1 with map size 1073741824
19:34:30 [19:34:30.573]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.592]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e65cf92be763e2d0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e65cf92be763e2d0"}
19:34:30 [19:34:30.602]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2ee96276c52f23da type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2ee96276c52f23da"}
19:34:30 [19:34:30.617]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.618]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91ba346752cd2ef4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:91ba346752cd2ef4"}
19:34:30 [19:34:30.620]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFVZtHP/1 with map size 1073741824
19:34:30 [19:34:30.625]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.642]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2edeb32f0d7924ae epochNumber=1
{"provingJobId":"1:BASE_PARITY:2edeb32f0d7924ae"}
19:34:30 [19:34:30.649]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:91ba346752cd2ef4 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:91ba346752cd2ef4"}
19:34:30 [19:34:30.659]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dbcd034c90343b3d epochNumber=1
{"provingJobId":"1:BASE_PARITY:dbcd034c90343b3d"}
19:34:30 [19:34:30.661]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXuSPOa/1 with map size 1073741824
19:34:30 [19:34:30.663]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.680]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d4e7ac637d2bbff epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d4e7ac637d2bbff"}
19:34:30 [19:34:30.689]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:dbcd034c90343b3d type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:dbcd034c90343b3d"}
19:34:30 [19:34:30.693]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.694]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6671471b849aaad epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6671471b849aaad"}
19:34:30 [19:34:30.695]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIUpYu5/1 with map size 1073741824
19:34:30 [19:34:30.700]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.720]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e6671471b849aaad
{"provingJobId":"1:BASE_PARITY:e6671471b849aaad"}
19:34:30 [19:34:30.721]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e6671471b849aaad type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:e6671471b849aaad"}
19:34:30 [19:34:30.727]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:feecbfb31fd87b4c epochNumber=1
{"provingJobId":"1:BASE_PARITY:feecbfb31fd87b4c"}
19:34:30 [19:34:30.735]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e6671471b849aaad has already been completed
{"provingJobId":"1:BASE_PARITY:e6671471b849aaad"}
19:34:30 [19:34:30.740]
INFO:
prover-client:proving-broker Proving Broker started
19:34:30 [19:34:30.741]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:50f08af94ca81de9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:50f08af94ca81de9"}
19:34:30 [19:34:30.741]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiTuuuG/1 with map size 1073741824
19:34:30 [19:34:30.743]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:30 [19:34:30.759]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1577f901f645dea1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1577f901f645dea1"}
19:34:30 [19:34:30.760]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testiTuuuG/2 with map size 1073741824
19:34:30 [19:34:30.763]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:31 [19:34:31.830]
INFO:
prover-client:proving-broker Proving Broker started
19:34:31 [19:34:31.838]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:50f08af94ca81de9 settled=false
{"provingJobId":"1:BASE_PARITY:50f08af94ca81de9","status":"pending"}
19:34:31 [19:34:31.840]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1577f901f645dea1 settled=false
{"provingJobId":"1:BASE_PARITY:1577f901f645dea1","status":"pending"}
19:34:31 [19:34:31.841]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:50f08af94ca81de9 type=BASE_PARITY startedAt=1750188870778 older agent has taken job
{"provingJobId":"1:BASE_PARITY:50f08af94ca81de9"}
19:34:31 [19:34:31.841]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:50f08af94ca81de9 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:50f08af94ca81de9"}
19:34:31 [19:34:31.846]
INFO:
prover-client:proving-broker Proving Broker started
19:34:31 [19:34:31.846]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:22512b05a502195d epochNumber=1
{"provingJobId":"1:BASE_PARITY:22512b05a502195d"}
19:34:31 [19:34:31.847]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testggGa4E/1 with map size 1073741824
19:34:31 [19:34:31.849]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:31 [19:34:31.865]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0754e5296deefb00 epochNumber=2
{"provingJobId":"1:BASE_PARITY:0754e5296deefb00"}
19:34:31 [19:34:31.866]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testggGa4E/2 with map size 1073741824
19:34:31 [19:34:31.871]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:32 [19:34:32.889]
INFO:
prover-client:proving-broker Proving Broker started
19:34:32 [19:34:32.891]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:22512b05a502195d settled=false
{"provingJobId":"1:BASE_PARITY:22512b05a502195d","status":"pending"}
19:34:32 [19:34:32.893]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0754e5296deefb00 settled=false
{"provingJobId":"1:BASE_PARITY:0754e5296deefb00","status":"pending"}
19:34:32 [19:34:32.894]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:22512b05a502195d type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:22512b05a502195d"}
19:34:32 [19:34:32.903]
INFO:
prover-client:proving-broker Proving Broker started
19:34:32 [19:34:32.904]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4fa6b7f7fc6e9941 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4fa6b7f7fc6e9941"}
19:34:32 [19:34:32.908]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8wFoz2/1 with map size 1073741824
19:34:32 [19:34:32.917]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:32 [19:34:32.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:29d001ff2ab2a926 epochNumber=2
{"provingJobId":"1:BASE_PARITY:29d001ff2ab2a926"}
19:34:32 [19:34:32.938]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8wFoz2/2 with map size 1073741824
19:34:32 [19:34:32.940]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:33 [19:34:33.966]
INFO:
prover-client:proving-broker Proving Broker started
19:34:33 [19:34:33.969]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4fa6b7f7fc6e9941 settled=false
{"provingJobId":"1:BASE_PARITY:4fa6b7f7fc6e9941","status":"pending"}
19:34:33 [19:34:33.971]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:29d001ff2ab2a926 settled=false
{"provingJobId":"1:BASE_PARITY:29d001ff2ab2a926","status":"pending"}
19:34:33 [19:34:33.972]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4fa6b7f7fc6e9941 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4fa6b7f7fc6e9941"}
19:34:33 [19:34:33.972]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4fa6b7f7fc6e9941 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4fa6b7f7fc6e9941"}
19:34:33 [19:34:33.988]
INFO:
prover-client:proving-broker Proving Broker started
19:34:33 [19:34:33.988]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:12544cf46983886e epochNumber=1
{"provingJobId":"1:BASE_PARITY:12544cf46983886e"}
19:34:33 [19:34:33.989]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzXRNdY/1 with map size 1073741824
19:34:33 [19:34:33.991]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.021]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36baee59c553f775 epochNumber=2
{"provingJobId":"1:BASE_PARITY:36baee59c553f775"}
19:34:34 [19:34:34.022]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzXRNdY/2 with map size 1073741824
19:34:34 [19:34:34.026]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.047]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:12544cf46983886e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:12544cf46983886e"}
19:34:34 [19:34:34.053]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:36baee59c553f775 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:36baee59c553f775"}
19:34:34 [19:34:34.064]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.065]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1b82a06dfc8898ce epochNumber=1
{"provingJobId":"1:BASE_PARITY:1b82a06dfc8898ce"}
19:34:34 [19:34:34.066]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbRoakA/1 with map size 1073741824
19:34:34 [19:34:34.070]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.091]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ea175aa45cd4f74f epochNumber=2
{"provingJobId":"1:BASE_PARITY:ea175aa45cd4f74f"}
19:34:34 [19:34:34.092]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testbRoakA/2 with map size 1073741824
19:34:34 [19:34:34.094]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.118]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1b82a06dfc8898ce type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:1b82a06dfc8898ce"}
19:34:34 [19:34:34.119]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1b82a06dfc8898ce type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1b82a06dfc8898ce"}
19:34:34 [19:34:34.127]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ea175aa45cd4f74f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ea175aa45cd4f74f"}
19:34:34 [19:34:34.140]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.141]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7c346764b08438ba err=test error
{"provingJoId":"1:BASE_PARITY:7c346764b08438ba"}
19:34:34 [19:34:34.146]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.146]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cd00ab0523e401d5 not found
{"provingJobId":"1:BASE_PARITY:cd00ab0523e401d5"}
19:34:34 [19:34:34.149]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.150]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5dbbd406d8123a5a epochNumber=1
{"provingJobId":"1:BASE_PARITY:5dbbd406d8123a5a"}
19:34:34 [19:34:34.151]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuuj5jS/1 with map size 1073741824
19:34:34 [19:34:34.154]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.184]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.185]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a7fec976b995dbda epochNumber=1
{"provingJobId":"1:BASE_PARITY:a7fec976b995dbda"}
19:34:34 [19:34:34.187]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4Wjh9I/1 with map size 1073741824
19:34:34 [19:34:34.191]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.340]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a7fec976b995dbda timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:a7fec976b995dbda"}
19:34:34 [19:34:34.346]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.347]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:213b54c6674de744 epochNumber=1
{"provingJobId":"1:BASE_PARITY:213b54c6674de744"}
19:34:34 [19:34:34.347]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTWrifL/1 with map size 1073741824
19:34:34 [19:34:34.349]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.476]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:213b54c6674de744 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:213b54c6674de744"}
19:34:34 [19:34:34.484]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:34e99f017f8eae03 epochNumber=10
{"provingJobId":"1:BASE_PARITY:34e99f017f8eae03"}
19:34:34 [19:34:34.485]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testTWrifL/10 with map size 1073741824
19:34:34 [19:34:34.488]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.529]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:34 [19:34:34.529]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
19:34:34 [19:34:34.643]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.644]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6fef9240a2f37e4b epochNumber=1
{"provingJobId":"1:BASE_PARITY:6fef9240a2f37e4b"}
19:34:34 [19:34:34.646]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVQMlWH/1 with map size 1073741824
19:34:34 [19:34:34.648]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.821]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6fef9240a2f37e4b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:6fef9240a2f37e4b"}
19:34:34 [19:34:34.871]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.872]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:feeab0ec5e732758 epochNumber=1
{"provingJobId":"1:BASE_PARITY:feeab0ec5e732758"}
19:34:34 [19:34:34.872]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdodnC6/1 with map size 1073741824
19:34:34 [19:34:34.874]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.895]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:feeab0ec5e732758 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:feeab0ec5e732758"}
19:34:34 [19:34:34.902]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.903]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c3aa6eb3060b9c20 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c3aa6eb3060b9c20"}
19:34:34 [19:34:34.906]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test916wP7/1 with map size 1073741824
19:34:34 [19:34:34.912]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.934]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c3aa6eb3060b9c20 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c3aa6eb3060b9c20"}
19:34:34 [19:34:34.935]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c3aa6eb3060b9c20 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:c3aa6eb3060b9c20"}
19:34:34 [19:34:34.946]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.946]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:33cc4e0521548152 epochNumber=1
{"provingJobId":"1:BASE_PARITY:33cc4e0521548152"}
19:34:34 [19:34:34.947]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZRCl3i/1 with map size 1073741824
19:34:34 [19:34:34.949]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:34 [19:34:34.969]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:33cc4e0521548152 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:33cc4e0521548152"}
19:34:34 [19:34:34.980]
INFO:
prover-client:proving-broker Proving Broker started
19:34:34 [19:34:34.980]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e51f158c02067361 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e51f158c02067361"}
19:34:34 [19:34:34.981]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtxafsH/1 with map size 1073741824
19:34:34 [19:34:34.986]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.010]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e51f158c02067361 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e51f158c02067361"}
19:34:35 [19:34:35.013]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:828065330f2ca1e7 epochNumber=3
{"provingJobId":"1:BASE_PARITY:828065330f2ca1e7"}
19:34:35 [19:34:35.014]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testtxafsH/3 with map size 1073741824
19:34:35 [19:34:35.019]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.056]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:35 [19:34:35.056]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
19:34:35 [19:34:35.064]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e51f158c02067361 err=test error
{"provingJoId":"1:BASE_PARITY:e51f158c02067361"}
19:34:35 [19:34:35.073]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5VAc5s/1 with map size 1073741824
19:34:35 [19:34:35.076]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.106]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5VAc5s/2 with map size 1073741824
19:34:35 [19:34:35.112]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.132]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.133]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6f7af43364de13ba settled=false
{"provingJobId":"1:BASE_PARITY:6f7af43364de13ba","status":"pending"}
19:34:35 [19:34:35.139]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e19a8f4d0a582a73 settled=false
{"provingJobId":"1:BASE_PARITY:e19a8f4d0a582a73","status":"pending"}
19:34:35 [19:34:35.148]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testk2jGLs/1 with map size 1073741824
19:34:35 [19:34:35.152]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.178]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testk2jGLs/2 with map size 1073741824
19:34:35 [19:34:35.183]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.231]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.235]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:30ed1686d63f27d5 settled=true
{"provingJobId":"1:BASE_PARITY:30ed1686d63f27d5","status":"fulfilled"}
19:34:35 [19:34:35.236]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:f4f2251a01c6ed93 settled=true
{"provingJobId":"2:BASE_PARITY:f4f2251a01c6ed93","status":"fulfilled"}
19:34:35 [19:34:35.245]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQMlw5j/1 with map size 1073741824
19:34:35 [19:34:35.248]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.279]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testQMlw5j/2 with map size 1073741824
19:34:35 [19:34:35.282]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.313]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.315]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:807035418d00fec1 settled=true
{"provingJobId":"1:BASE_PARITY:807035418d00fec1","status":"fulfilled"}
19:34:35 [19:34:35.319]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:70a0a6cdf32792d9 settled=false
{"provingJobId":"1:BASE_PARITY:70a0a6cdf32792d9","status":"pending"}
19:34:35 [19:34:35.327]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.327]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ff75f6f2e387143a epochNumber=1
{"provingJobId":"1:BASE_PARITY:ff75f6f2e387143a"}
19:34:35 [19:34:35.328]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZHGbh8/1 with map size 1073741824
19:34:35 [19:34:35.331]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.352]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.353]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5eec2da259a7fc31 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5eec2da259a7fc31"}
19:34:35 [19:34:35.353]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:5eec2da259a7fc31: Error: db error: Error: db error
19:34:35 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:5eec2da259a7fc31"}
19:34:35 [19:34:35.360]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bacb0359a3ea9f21 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bacb0359a3ea9f21"}
19:34:35 [19:34:35.362]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testp0vYJ4/1 with map size 1073741824
19:34:35 [19:34:35.365]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.385]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bacb0359a3ea9f21 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:bacb0359a3ea9f21"}
19:34:35 [19:34:35.385]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bacb0359a3ea9f21 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:bacb0359a3ea9f21"}
19:34:35 [19:34:35.401]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c69aff6d062e6e0d epochNumber=1
{"provingJobId":"1:BASE_PARITY:c69aff6d062e6e0d"}
19:34:35 [19:34:35.403]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test95GwPf/1 with map size 1073741824
19:34:35 [19:34:35.407]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.429]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c69aff6d062e6e0d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c69aff6d062e6e0d"}
19:34:35 [19:34:35.431]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c69aff6d062e6e0d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c69aff6d062e6e0d"}
19:34:35 [19:34:35.432]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:c69aff6d062e6e0d: Error: db error
19:34:35 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:c69aff6d062e6e0d"}
19:34:35 [19:34:35.437]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.438]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:acee365b6edb0454 epochNumber=1
{"provingJobId":"1:BASE_PARITY:acee365b6edb0454"}
19:34:35 [19:34:35.440]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testka36MZ/1 with map size 1073741824
19:34:35 [19:34:35.443]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.460]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:acee365b6edb0454 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:acee365b6edb0454"}
19:34:35 [19:34:35.460]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:acee365b6edb0454 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:acee365b6edb0454"}
19:34:35 [19:34:35.472]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.473]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3695e68cadf7e3a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3695e68cadf7e3a3"}
19:34:35 [19:34:35.473]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testH7ED03/1 with map size 1073741824
19:34:35 [19:34:35.477]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.498]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3695e68cadf7e3a3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3695e68cadf7e3a3"}
19:34:35 [19:34:35.498]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3695e68cadf7e3a3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3695e68cadf7e3a3"}
19:34:35 [19:34:35.501]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:3695e68cadf7e3a3 jobErr=test error: Error: db error
19:34:35 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:3695e68cadf7e3a3"}
19:34:35 [19:34:35.510]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.511]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0b36aaaceada4be7 not found
{"provingJobId":"1:BASE_PARITY:0b36aaaceada4be7"}
19:34:35 [19:34:35.518]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.518]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:640ea44d7b781e16 err=test error
{"provingJoId":"1:BASE_PARITY:640ea44d7b781e16"}
19:34:35 [19:34:35.524]
INFO:
prover-client:proving-broker Proving Broker started
19:34:35 [19:34:35.535]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f62a8257e3e97e17 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f62a8257e3e97e17"}
19:34:35 [19:34:35.537]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testK8vcpg/1 with map size 1073741824
19:34:35 [19:34:35.540]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.558]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f62a8257e3e97e17 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f62a8257e3e97e17"}
19:34:35 [19:34:35.558]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f62a8257e3e97e17 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f62a8257e3e97e17"}
19:34:35 [19:34:35.574]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:62d4e994c3f0d0f8 epochNumber=2
{"provingJobId":"2:BASE_PARITY:62d4e994c3f0d0f8"}
19:34:35 [19:34:35.576]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testK8vcpg/2 with map size 1073741824
19:34:35 [19:34:35.578]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.599]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:62d4e994c3f0d0f8 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:62d4e994c3f0d0f8"}
19:34:35 [19:34:35.600]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:62d4e994c3f0d0f8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:62d4e994c3f0d0f8"}
19:34:35 [19:34:35.673]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:3fe79a39330dc824 epochNumber=3
{"provingJobId":"3:BASE_PARITY:3fe79a39330dc824"}
19:34:35 [19:34:35.675]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testK8vcpg/3 with map size 1073741824
19:34:35 [19:34:35.677]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.702]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:35 [19:34:35.703]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
19:34:35 [19:34:35.757]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:89719e59aed1025f epochNumber=4
{"provingJobId":"4:BASE_PARITY:89719e59aed1025f"}
19:34:35 [19:34:35.758]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testK8vcpg/4 with map size 1073741824
19:34:35 [19:34:35.764]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.784]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:35 [19:34:35.785]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
19:34:35 [19:34:35.845]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:e606278e74108bd8 epochNumber=5
{"provingJobId":"5:BASE_PARITY:e606278e74108bd8"}
19:34:35 [19:34:35.847]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testK8vcpg/5 with map size 1073741824
19:34:35 [19:34:35.849]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
19:34:35 [19:34:35.889]
INFO:
prover-client:proving-broker Cleaned up jobs=1
19:34:35 [19:34:35.890]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
19:34:35
PASS src/proving_broker/proving_broker.test.ts (
13.663 s)
19:34:35 ProvingBroker
19:34:35 Producer API
19:34:35
✓ refuses stale jobs (38 ms)
19:34:35
✓ enqueues jobs (5 ms)
19:34:35
✓ ignores duplicate jobs (4 ms)
19:34:35
✓ reports correct status when enqueuing repeat jobs (5 ms)
19:34:35
✓ throws an error in case of duplicate job IDs (30 ms)
19:34:35
✓ returns not-found status for non-existing jobs (1 ms)
19:34:35
✓ cancels jobs in queue (3 ms)
19:34:35
✓ cancels jobs in-progress (2 ms)
19:34:35
✓ returns job result if successful (3 ms)
19:34:35
✓ returns job error if failed (2 ms)
19:34:35
✓ correctly returns job status for concurrent writes (2 ms)
19:34:35
✓ refuses stale jobs (47 ms)
19:34:35
✓ enqueues jobs (38 ms)
19:34:35
✓ ignores duplicate jobs (31 ms)
19:34:35
✓ reports correct status when enqueuing repeat jobs (37 ms)
19:34:35
✓ throws an error in case of duplicate job IDs (31 ms)
19:34:35
✓ returns not-found status for non-existing jobs (4 ms)
19:34:35
✓ cancels jobs in queue (38 ms)
19:34:35
✓ cancels jobs in-progress (31 ms)
19:34:35
✓ returns job result if successful (33 ms)
19:34:35
✓ returns job error if failed (31 ms)
19:34:35
✓ correctly returns job status for concurrent writes (52 ms)
19:34:35 Consumer API
19:34:35
✓ returns undefined if no jobs are available (8 ms)
19:34:35
✓ returns jobs in priority order (2 ms)
19:34:35
✓ returns undefined if no jobs are available for the given allowList (2 ms)
19:34:35
✓ returns a job if it is in the allowList (2 ms)
19:34:35
✓ returns the most important job if it is in the allowList (2 ms)
19:34:35
✓ returns any job if filter is empty (2 ms)
19:34:35
✓ returns a new job when reporting job success (2 ms)
19:34:35
✓ returns a new job when reporting permanent error (2 ms)
19:34:35
✓ returns a new job when reporting retry-able error (2 ms)
19:34:35
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
19:34:35
✓ returns a new job if job is already in progress elsewhere (1058 ms)
19:34:35
✓ avoids sending the same job to a new agent after a restart (1006 ms)
19:34:35
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
19:34:35
✓ tracks job result if in progress (3 ms)
19:34:35
✓ tracks job result even if job is in queue (3 ms)
19:34:35
✓ ignores reported job error if unknown job (2 ms)
19:34:35
✓ ignores job result if unknown job (1 ms)
19:34:35
✓ returns undefined if no jobs are available (4 ms)
19:34:35
✓ returns jobs in priority order (41 ms)
19:34:35
✓ returns undefined if no jobs are available for the given allowList (19 ms)
19:34:35
✓ returns a job if it is in the allowList (61 ms)
19:34:35
✓ returns the most important job if it is in the allowList (63 ms)
19:34:35
✓ returns any job if filter is empty (65 ms)
19:34:35
✓ returns a new job when reporting job success (42 ms)
19:34:35
✓ returns a new job when reporting permanent error (46 ms)
19:34:35
✓ returns a new job when reporting retry-able error (34 ms)
19:34:35
✓ returns a new job when reporting progress if current one is cancelled (45 ms)
19:34:35
✓ returns a new job if job is already in progress elsewhere (1105 ms)
19:34:35
✓ avoids sending the same job to a new agent after a restart (1056 ms)
19:34:35
✓ avoids sending a completed job to a new agent after a restart (1084 ms)
19:34:35
✓ tracks job result if in progress (75 ms)
19:34:35
✓ tracks job result even if job is in queue (75 ms)
19:34:35
✓ ignores reported job error if unknown job (4 ms)
19:34:35
✓ ignores job result if unknown job (5 ms)
19:34:35 Timeouts
19:34:35
✓ tracks in progress jobs (2 ms)
19:34:35
✓ re-enqueues jobs that time out (110 ms)
19:34:35
✓ cancel stale jobs that time out (231 ms)
19:34:35
✓ keeps the jobs in progress while it is alive (203 ms)
19:34:35
✓ tracks in progress jobs (33 ms)
19:34:35
✓ re-enqueues jobs that time out (160 ms)
19:34:35
✓ cancel stale jobs that time out (297 ms)
19:34:35
✓ keeps the jobs in progress while it is alive (228 ms)
19:34:35 Retries
19:34:35
✓ retries jobs (3 ms)
19:34:35
✓ retries up to a maximum number of times (3 ms)
19:34:35
✓ passing retry=false does not retry (1 ms)
19:34:35
✓ does not retry if job is stale (27 ms)
19:34:35
✓ retries jobs (30 ms)
19:34:35
✓ retries up to a maximum number of times (44 ms)
19:34:35
✓ passing retry=false does not retry (33 ms)
19:34:35
✓ does not retry if job is stale (89 ms)
19:34:35 Database management
19:34:35
✓ re-enqueues proof requests on start (3 ms)
19:34:35
✓ restores proof results on start (2 ms)
19:34:35
✓ only re-enqueues unfinished jobs (1 ms)
19:34:35
✓ saves job when enqueued (2 ms)
19:34:35
✓ does not retain job if database fails to save (3 ms)
19:34:35
✓ saves job result (2 ms)
19:34:35
✓ saves result even if database fails to save (2 ms)
19:34:35
✓ saves job error (3 ms)
19:34:35
✓ saves job error even if database fails to save (2 ms)
19:34:35
✓ does not save job result if job is unknown (2 ms)
19:34:35
✓ does not save job error if job is unknown (1 ms)
19:34:35
✓ cleans up old jobs periodically (262 ms)
19:34:35
✓ re-enqueues proof requests on start (76 ms)
19:34:35
✓ restores proof results on start (95 ms)
19:34:35
✓ only re-enqueues unfinished jobs (85 ms)
19:34:35
✓ saves job when enqueued (26 ms)
19:34:35
✓ does not retain job if database fails to save (5 ms)
19:34:35
✓ saves job result (39 ms)
19:34:35
✓ saves result even if database fails to save (41 ms)
19:34:35
✓ saves job error (34 ms)
19:34:35
✓ saves job error even if database fails to save (34 ms)
19:34:35
✓ does not save job result if job is unknown (9 ms)
19:34:35
✓ does not save job error if job is unknown (6 ms)
19:34:35
✓ cleans up old jobs periodically (399 ms)
19:34:35
19:34:35
Test Suites: 1 passed, 1 total
19:34:35
Tests: 96 passed, 96 total
19:34:35
Snapshots: 0 total
19:34:35
Time: 13.775 s
19:34:35
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
19:34:35
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?