Parent Log:
http://ci.aztec-labs.com/4451244b7f022098
Command: 731d98bdb3377842 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/4110b08ca823c6af4399e58e54e0483d6e3459ed
Env: REF_NAME=gh-readonly-queue/next/pr-15064-daebab2c4bace59ef25ddb29040a040926f095e6 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 16 09:31:40 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15064_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
09:31:44 [09:31:44.602]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.609]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:28c42dbbb62f91b0 epochNumber=42
{"provingJobId":"1:BASE_PARITY:28c42dbbb62f91b0"}
09:31:44 [09:31:44.613]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:f1f19dffaeff7aa0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f1f19dffaeff7aa0"}
09:31:44 [09:31:44.647]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b41562ec99bf87a7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b41562ec99bf87a7"}
09:31:44 [09:31:44.650]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:280d1e3da0a76535 epochNumber=1
{"provingJobId":"1:BASE_PARITY:280d1e3da0a76535"}
09:31:44 [09:31:44.652]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.654]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:691ddf8564a90bf3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:691ddf8564a90bf3"}
09:31:44 [09:31:44.655]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:691ddf8564a90bf3 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:691ddf8564a90bf3"}
09:31:44 [09:31:44.658]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.659]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4fd8ec2739482ba4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4fd8ec2739482ba4"}
09:31:44 [09:31:44.661]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4fd8ec2739482ba4 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:4fd8ec2739482ba4"}
09:31:44 [09:31:44.662]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4fd8ec2739482ba4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4fd8ec2739482ba4"}
09:31:44 [09:31:44.663]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4fd8ec2739482ba4 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:4fd8ec2739482ba4"}
09:31:44 [09:31:44.664]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.665]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62f0c59c32d1ecbc epochNumber=1
{"provingJobId":"1:BASE_PARITY:62f0c59c32d1ecbc"}
09:31:44 [09:31:44.691]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.692]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.693]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e81a230792e6a5ef epochNumber=1
{"provingJobId":"1:BASE_PARITY:e81a230792e6a5ef"}
09:31:44 [09:31:44.694]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e81a230792e6a5ef
{"provingJobId":"1:BASE_PARITY:e81a230792e6a5ef"}
09:31:44 [09:31:44.694]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e81a230792e6a5ef type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e81a230792e6a5ef"}
09:31:44 [09:31:44.695]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e81a230792e6a5ef type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:e81a230792e6a5ef"}
09:31:44 [09:31:44.697]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.697]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:589365c58c2d4f1c epochNumber=1
{"provingJobId":"1:BASE_PARITY:589365c58c2d4f1c"}
09:31:44 [09:31:44.698]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:589365c58c2d4f1c
{"provingJobId":"1:BASE_PARITY:589365c58c2d4f1c"}
09:31:44 [09:31:44.698]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:589365c58c2d4f1c type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:589365c58c2d4f1c"}
09:31:44 [09:31:44.708]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.709]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e28c2d6ea897cacd epochNumber=1
{"provingJobId":"1:BASE_PARITY:e28c2d6ea897cacd"}
09:31:44 [09:31:44.710]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e28c2d6ea897cacd type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e28c2d6ea897cacd"}
09:31:44 [09:31:44.710]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e28c2d6ea897cacd type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e28c2d6ea897cacd"}
09:31:44 [09:31:44.712]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.712]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e08417cc26c7e85 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e08417cc26c7e85"}
09:31:44 [09:31:44.713]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0e08417cc26c7e85 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0e08417cc26c7e85"}
09:31:44 [09:31:44.713]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0e08417cc26c7e85 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0e08417cc26c7e85"}
09:31:44 [09:31:44.714]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.715]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d01356096c6eea3d epochNumber=0
{"provingJobId":"1:BASE_PARITY:d01356096c6eea3d"}
09:31:44 [09:31:44.715]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d01356096c6eea3d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d01356096c6eea3d"}
09:31:44 [09:31:44.715]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b2749bb8ed7acbd epochNumber=0
{"provingJobId":"1:BASE_PARITY:7b2749bb8ed7acbd"}
09:31:44 [09:31:44.716]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d01356096c6eea3d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d01356096c6eea3d"}
09:31:44 [09:31:44.716]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5dfcb2f4bf47c46f epochNumber=0
{"provingJobId":"1:BASE_PARITY:5dfcb2f4bf47c46f"}
09:31:44 [09:31:44.717]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.719]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.719]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4008c0bad69eacd epochNumber=2
{"provingJobId":"1:BASE_PARITY:a4008c0bad69eacd"}
09:31:44 [09:31:44.719]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f954a71bed41ed7d epochNumber=1
{"provingJobId":"1:BASE_PARITY:f954a71bed41ed7d"}
09:31:44 [09:31:44.720]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.721]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:670f06229bd47454 epochNumber=1
{"provingJobId":"1:BASE_PARITY:670f06229bd47454"}
09:31:44 [09:31:44.722]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6f45425289b655f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6f45425289b655f5"}
09:31:44 [09:31:44.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:22ff543b609cafd5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:22ff543b609cafd5"}
09:31:44 [09:31:44.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31a4c461f2a216c4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:31a4c461f2a216c4"}
09:31:44 [09:31:44.724]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57995801ae0ced0e epochNumber=1
{"provingJobId":"1:BASE_PARITY:57995801ae0ced0e"}
09:31:44 [09:31:44.725]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.725]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:126d5de3eff096ac epochNumber=1
{"provingJobId":"1:BASE_PARITY:126d5de3eff096ac"}
09:31:44 [09:31:44.726]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5315379516103d1f epochNumber=1
{"provingJobId":"1:BASE_PARITY:5315379516103d1f"}
09:31:44 [09:31:44.726]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec9ac133913427d9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ec9ac133913427d9"}
09:31:44 [09:31:44.726]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7157044496704220 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7157044496704220"}
09:31:44 [09:31:44.727]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.727]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:32dcedc97f9d118d epochNumber=1
{"provingJobId":"1:BASE_PARITY:32dcedc97f9d118d"}
09:31:44 [09:31:44.728]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c877f0f6de969f7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6c877f0f6de969f7"}
09:31:44 [09:31:44.728]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:50b4fd6594c786d2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:50b4fd6594c786d2"}
09:31:44 [09:31:44.728]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:29382c15f2329db2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:29382c15f2329db2"}
09:31:44 [09:31:44.729]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.730]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6f300d2ee387896 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6f300d2ee387896"}
09:31:44 [09:31:44.730]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7faf0845f25a33d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7faf0845f25a33d7"}
09:31:44 [09:31:44.731]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e6f300d2ee387896 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e6f300d2ee387896"}
09:31:44 [09:31:44.732]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.732]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac009c4661a80afe epochNumber=1
{"provingJobId":"1:BASE_PARITY:ac009c4661a80afe"}
09:31:44 [09:31:44.733]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ca630e15d29b9a8f epochNumber=1
{"provingJobId":"1:BASE_PARITY:ca630e15d29b9a8f"}
09:31:44 [09:31:44.733]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ac009c4661a80afe type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:ac009c4661a80afe"}
09:31:44 [09:31:44.734]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.735]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:16cfbe4169971c8d epochNumber=1
{"provingJobId":"1:BASE_PARITY:16cfbe4169971c8d"}
09:31:44 [09:31:44.735]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8f0ba893ff6a5a0e epochNumber=1
{"provingJobId":"1:BASE_PARITY:8f0ba893ff6a5a0e"}
09:31:44 [09:31:44.735]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:16cfbe4169971c8d type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:16cfbe4169971c8d"}
09:31:44 [09:31:44.736]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.737]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1410264d04cae9e9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1410264d04cae9e9"}
09:31:44 [09:31:44.737]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:1410264d04cae9e9
{"provingJobId":"1:BASE_PARITY:1410264d04cae9e9"}
09:31:44 [09:31:44.738]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1410264d04cae9e9 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:1410264d04cae9e9"}
09:31:44 [09:31:44.738]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fea87525eee30f51 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fea87525eee30f51"}
09:31:44 [09:31:44.738]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1410264d04cae9e9 has already been completed
{"provingJobId":"1:BASE_PARITY:1410264d04cae9e9"}
09:31:44 [09:31:44.739]
INFO:
prover-client:proving-broker Proving Broker started
09:31:44 [09:31:44.740]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f89c85974711d851 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f89c85974711d851"}
09:31:44 [09:31:44.740]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:363bfb5af3a0983b epochNumber=2
{"provingJobId":"1:BASE_PARITY:363bfb5af3a0983b"}
09:31:45 [09:31:45.792]
INFO:
prover-client:proving-broker Proving Broker started
09:31:45 [09:31:45.792]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f89c85974711d851 settled=false
{"provingJobId":"1:BASE_PARITY:f89c85974711d851","status":"pending"}
09:31:45 [09:31:45.793]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:363bfb5af3a0983b settled=false
{"provingJobId":"1:BASE_PARITY:363bfb5af3a0983b","status":"pending"}
09:31:45 [09:31:45.793]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f89c85974711d851 type=BASE_PARITY startedAt=1750066304740 older agent has taken job
{"provingJobId":"1:BASE_PARITY:f89c85974711d851"}
09:31:45 [09:31:45.794]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f89c85974711d851 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:f89c85974711d851"}
09:31:45 [09:31:45.795]
INFO:
prover-client:proving-broker Proving Broker started
09:31:45 [09:31:45.795]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b08de1354583fd53 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b08de1354583fd53"}
09:31:45 [09:31:45.795]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a6b010070f74b9b epochNumber=2
{"provingJobId":"1:BASE_PARITY:7a6b010070f74b9b"}
09:31:46 [09:31:46.796]
INFO:
prover-client:proving-broker Proving Broker started
09:31:46 [09:31:46.797]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b08de1354583fd53 settled=false
{"provingJobId":"1:BASE_PARITY:b08de1354583fd53","status":"pending"}
09:31:46 [09:31:46.797]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7a6b010070f74b9b settled=false
{"provingJobId":"1:BASE_PARITY:7a6b010070f74b9b","status":"pending"}
09:31:46 [09:31:46.797]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b08de1354583fd53 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:b08de1354583fd53"}
09:31:46 [09:31:46.799]
INFO:
prover-client:proving-broker Proving Broker started
09:31:46 [09:31:46.800]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b6867af1b6a5f5f epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b6867af1b6a5f5f"}
09:31:46 [09:31:46.800]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:82f370f80f3fa52c epochNumber=2
{"provingJobId":"1:BASE_PARITY:82f370f80f3fa52c"}
09:31:47 [09:31:47.801]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.801]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2b6867af1b6a5f5f settled=false
{"provingJobId":"1:BASE_PARITY:2b6867af1b6a5f5f","status":"pending"}
09:31:47 [09:31:47.802]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:82f370f80f3fa52c settled=false
{"provingJobId":"1:BASE_PARITY:82f370f80f3fa52c","status":"pending"}
09:31:47 [09:31:47.803]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b6867af1b6a5f5f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2b6867af1b6a5f5f"}
09:31:47 [09:31:47.803]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2b6867af1b6a5f5f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2b6867af1b6a5f5f"}
09:31:47 [09:31:47.804]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.804]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fcefbd5bf0e2d21e epochNumber=1
{"provingJobId":"1:BASE_PARITY:fcefbd5bf0e2d21e"}
09:31:47 [09:31:47.805]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2bcfc2c6be212775 epochNumber=2
{"provingJobId":"1:BASE_PARITY:2bcfc2c6be212775"}
09:31:47 [09:31:47.805]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:fcefbd5bf0e2d21e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:fcefbd5bf0e2d21e"}
09:31:47 [09:31:47.806]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2bcfc2c6be212775 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2bcfc2c6be212775"}
09:31:47 [09:31:47.807]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.807]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:63f9f9ae60aa7ba5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:63f9f9ae60aa7ba5"}
09:31:47 [09:31:47.808]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a6472f0ecd91544 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7a6472f0ecd91544"}
09:31:47 [09:31:47.809]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:63f9f9ae60aa7ba5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:63f9f9ae60aa7ba5"}
09:31:47 [09:31:47.809]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:63f9f9ae60aa7ba5 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:63f9f9ae60aa7ba5"}
09:31:47 [09:31:47.810]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7a6472f0ecd91544 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7a6472f0ecd91544"}
09:31:47 [09:31:47.811]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.812]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:84d0b487c1c43167 err=test error
{"provingJoId":"1:BASE_PARITY:84d0b487c1c43167"}
09:31:47 [09:31:47.813]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.813]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b3bb87eeaafe99e not found
{"provingJobId":"1:BASE_PARITY:2b3bb87eeaafe99e"}
09:31:47 [09:31:47.814]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.815]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d6a07ac01d0c082c epochNumber=1
{"provingJobId":"1:BASE_PARITY:d6a07ac01d0c082c"}
09:31:47 [09:31:47.816]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.816]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d6a06c88ff3c2b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d6a06c88ff3c2b4"}
09:31:47 [09:31:47.918]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7d6a06c88ff3c2b4 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:7d6a06c88ff3c2b4"}
09:31:47 [09:31:47.924]
INFO:
prover-client:proving-broker Proving Broker started
09:31:47 [09:31:47.925]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3f662eb0842ff16 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d3f662eb0842ff16"}
09:31:48 [09:31:48.026]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d3f662eb0842ff16 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:d3f662eb0842ff16"}
09:31:48 [09:31:48.028]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9c75135345c29fb4 epochNumber=10
{"provingJobId":"1:BASE_PARITY:9c75135345c29fb4"}
09:31:48 [09:31:48.052]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:48 [09:31:48.157]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.159]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b6333bdc5285200 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b6333bdc5285200"}
09:31:48 [09:31:48.311]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b6333bdc5285200 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2b6333bdc5285200"}
09:31:48 [09:31:48.362]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.363]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:40ef891e2a3e304b epochNumber=1
{"provingJobId":"1:BASE_PARITY:40ef891e2a3e304b"}
09:31:48 [09:31:48.364]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:40ef891e2a3e304b type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:40ef891e2a3e304b"}
09:31:48 [09:31:48.365]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.366]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81b11d4e5b851c53 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81b11d4e5b851c53"}
09:31:48 [09:31:48.367]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:81b11d4e5b851c53 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:81b11d4e5b851c53"}
09:31:48 [09:31:48.368]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:81b11d4e5b851c53 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:81b11d4e5b851c53"}
09:31:48 [09:31:48.370]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.370]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:77cc51b42f2a685c epochNumber=1
{"provingJobId":"1:BASE_PARITY:77cc51b42f2a685c"}
09:31:48 [09:31:48.371]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:77cc51b42f2a685c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:77cc51b42f2a685c"}
09:31:48 [09:31:48.372]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.372]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38ee24a9aa8f7f17 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38ee24a9aa8f7f17"}
09:31:48 [09:31:48.373]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:38ee24a9aa8f7f17 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:38ee24a9aa8f7f17"}
09:31:48 [09:31:48.374]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31e2c22ff09de82d epochNumber=3
{"provingJobId":"1:BASE_PARITY:31e2c22ff09de82d"}
09:31:48 [09:31:48.398]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:48 [09:31:48.399]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:38ee24a9aa8f7f17 err=test error
{"provingJoId":"1:BASE_PARITY:38ee24a9aa8f7f17"}
09:31:48 [09:31:48.400]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.401]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:217da3673be8576a settled=false
{"provingJobId":"1:BASE_PARITY:217da3673be8576a","status":"pending"}
09:31:48 [09:31:48.401]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ac4510ef07178022 settled=false
{"provingJobId":"1:BASE_PARITY:ac4510ef07178022","status":"pending"}
09:31:48 [09:31:48.403]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.404]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:445defffdab524c9 settled=true
{"provingJobId":"1:BASE_PARITY:445defffdab524c9","status":"fulfilled"}
09:31:48 [09:31:48.404]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:08f67c897a4cad23 settled=true
{"provingJobId":"2:BASE_PARITY:08f67c897a4cad23","status":"fulfilled"}
09:31:48 [09:31:48.406]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.406]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4abbca0794fd6363 settled=true
{"provingJobId":"1:BASE_PARITY:4abbca0794fd6363","status":"fulfilled"}
09:31:48 [09:31:48.407]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:667f60d05511c228 settled=false
{"provingJobId":"1:BASE_PARITY:667f60d05511c228","status":"pending"}
09:31:48 [09:31:48.408]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.408]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f79dbd72d4a9171 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f79dbd72d4a9171"}
09:31:48 [09:31:48.410]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.410]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae17327c40451d19 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae17327c40451d19"}
09:31:48 [09:31:48.411]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:ae17327c40451d19: Error: db error: Error: db error
09:31:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:ae17327c40451d19"}
09:31:48 [09:31:48.412]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.412]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f97054eb513bd91f epochNumber=1
{"provingJobId":"1:BASE_PARITY:f97054eb513bd91f"}
09:31:48 [09:31:48.412]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f97054eb513bd91f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f97054eb513bd91f"}
09:31:48 [09:31:48.413]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f97054eb513bd91f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f97054eb513bd91f"}
09:31:48 [09:31:48.414]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.414]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b872ae6e13406448 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b872ae6e13406448"}
09:31:48 [09:31:48.414]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b872ae6e13406448 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b872ae6e13406448"}
09:31:48 [09:31:48.414]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b872ae6e13406448 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b872ae6e13406448"}
09:31:48 [09:31:48.415]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:b872ae6e13406448: Error: db error
09:31:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:b872ae6e13406448"}
09:31:48 [09:31:48.416]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.416]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d13ae1eb2238abbc epochNumber=1
{"provingJobId":"1:BASE_PARITY:d13ae1eb2238abbc"}
09:31:48 [09:31:48.416]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d13ae1eb2238abbc type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d13ae1eb2238abbc"}
09:31:48 [09:31:48.416]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d13ae1eb2238abbc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d13ae1eb2238abbc"}
09:31:48 [09:31:48.417]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.417]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6bacf91a44fb8f54 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6bacf91a44fb8f54"}
09:31:48 [09:31:48.418]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6bacf91a44fb8f54 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6bacf91a44fb8f54"}
09:31:48 [09:31:48.418]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6bacf91a44fb8f54 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6bacf91a44fb8f54"}
09:31:48 [09:31:48.418]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:6bacf91a44fb8f54 jobErr=test error: Error: db error
09:31:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:6bacf91a44fb8f54"}
09:31:48 [09:31:48.419]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.419]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7c940d6b9c4e8b20 not found
{"provingJobId":"1:BASE_PARITY:7c940d6b9c4e8b20"}
09:31:48 [09:31:48.420]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.421]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c970ba1ba485891c err=test error
{"provingJoId":"1:BASE_PARITY:c970ba1ba485891c"}
09:31:48 [09:31:48.422]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.433]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30f08a9d66aba2f1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:30f08a9d66aba2f1"}
09:31:48 [09:31:48.434]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:30f08a9d66aba2f1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:30f08a9d66aba2f1"}
09:31:48 [09:31:48.434]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:30f08a9d66aba2f1 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:30f08a9d66aba2f1"}
09:31:48 [09:31:48.445]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:48b340e4913a660e epochNumber=2
{"provingJobId":"2:BASE_PARITY:48b340e4913a660e"}
09:31:48 [09:31:48.446]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:48b340e4913a660e type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:48b340e4913a660e"}
09:31:48 [09:31:48.446]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:48b340e4913a660e type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:48b340e4913a660e"}
09:31:48 [09:31:48.507]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:edd03341d6adad91 epochNumber=3
{"provingJobId":"3:BASE_PARITY:edd03341d6adad91"}
09:31:48 [09:31:48.524]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:48 [09:31:48.569]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:79d72a3de7733c1a epochNumber=4
{"provingJobId":"4:BASE_PARITY:79d72a3de7733c1a"}
09:31:48 [09:31:48.579]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:48 [09:31:48.632]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:c1455404e2942cac epochNumber=5
{"provingJobId":"5:BASE_PARITY:c1455404e2942cac"}
09:31:48 [09:31:48.656]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:48 [09:31:48.687]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.687]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c06c1591525296e1 epochNumber=42
{"provingJobId":"1:BASE_PARITY:c06c1591525296e1"}
09:31:48 [09:31:48.689]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testdQIVdD/42 with map size 1073741824
09:31:48 [09:31:48.692]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.717]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:ffbd611312129e93 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ffbd611312129e93"}
09:31:48 [09:31:48.723]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.724]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:593e148a538cac8b epochNumber=1
{"provingJobId":"1:BASE_PARITY:593e148a538cac8b"}
09:31:48 [09:31:48.725]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3aPgxQ/1 with map size 1073741824
09:31:48 [09:31:48.729]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.746]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09a9430d1fd54f86 epochNumber=1
{"provingJobId":"1:BASE_PARITY:09a9430d1fd54f86"}
09:31:48 [09:31:48.761]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.762]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6b7884eec8989416 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6b7884eec8989416"}
09:31:48 [09:31:48.767]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUQxI8V/1 with map size 1073741824
09:31:48 [09:31:48.774]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.792]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6b7884eec8989416 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6b7884eec8989416"}
09:31:48 [09:31:48.797]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.799]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2b4415600138eb1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2b4415600138eb1"}
09:31:48 [09:31:48.800]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJOjX1C/1 with map size 1073741824
09:31:48 [09:31:48.803]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.822]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a2b4415600138eb1 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a2b4415600138eb1"}
09:31:48 [09:31:48.823]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a2b4415600138eb1 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a2b4415600138eb1"}
09:31:48 [09:31:48.829]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a2b4415600138eb1 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a2b4415600138eb1"}
09:31:48 [09:31:48.833]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.834]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55d544da92f4bd14 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55d544da92f4bd14"}
09:31:48 [09:31:48.835]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpXn7Kw/1 with map size 1073741824
09:31:48 [09:31:48.838]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.865]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.868]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.869]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4db6f45b1d1da95d epochNumber=1
{"provingJobId":"1:BASE_PARITY:4db6f45b1d1da95d"}
09:31:48 [09:31:48.872]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3q4m7j/1 with map size 1073741824
09:31:48 [09:31:48.874]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.893]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:4db6f45b1d1da95d
{"provingJobId":"1:BASE_PARITY:4db6f45b1d1da95d"}
09:31:48 [09:31:48.894]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4db6f45b1d1da95d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4db6f45b1d1da95d"}
09:31:48 [09:31:48.894]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4db6f45b1d1da95d type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:4db6f45b1d1da95d"}
09:31:48 [09:31:48.903]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.904]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:396d58049813cb43 epochNumber=1
{"provingJobId":"1:BASE_PARITY:396d58049813cb43"}
09:31:48 [09:31:48.905]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMfJbbc/1 with map size 1073741824
09:31:48 [09:31:48.908]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.926]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:396d58049813cb43
{"provingJobId":"1:BASE_PARITY:396d58049813cb43"}
09:31:48 [09:31:48.926]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:396d58049813cb43 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:396d58049813cb43"}
09:31:48 [09:31:48.937]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.938]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0fbe6d03b016b44f epochNumber=1
{"provingJobId":"1:BASE_PARITY:0fbe6d03b016b44f"}
09:31:48 [09:31:48.938]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHvfWP8/1 with map size 1073741824
09:31:48 [09:31:48.940]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.964]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0fbe6d03b016b44f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0fbe6d03b016b44f"}
09:31:48 [09:31:48.964]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0fbe6d03b016b44f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0fbe6d03b016b44f"}
09:31:48 [09:31:48.973]
INFO:
prover-client:proving-broker Proving Broker started
09:31:48 [09:31:48.973]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:beb8ddaec7cad498 epochNumber=1
{"provingJobId":"1:BASE_PARITY:beb8ddaec7cad498"}
09:31:48 [09:31:48.974]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOEQQ58/1 with map size 1073741824
09:31:48 [09:31:48.976]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:48 [09:31:48.996]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:beb8ddaec7cad498 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:beb8ddaec7cad498"}
09:31:48 [09:31:48.996]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:beb8ddaec7cad498 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:beb8ddaec7cad498"}
09:31:49 [09:31:49.004]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.004]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ab5df3c678801a6d epochNumber=0
{"provingJobId":"1:BASE_PARITY:ab5df3c678801a6d"}
09:31:49 [09:31:49.005]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testTcsNa1/0 with map size 1073741824
09:31:49 [09:31:49.009]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.027]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ab5df3c678801a6d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ab5df3c678801a6d"}
09:31:49 [09:31:49.028]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3f9c49dfcda0af17 epochNumber=0
{"provingJobId":"1:BASE_PARITY:3f9c49dfcda0af17"}
09:31:49 [09:31:49.028]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ab5df3c678801a6d epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:ab5df3c678801a6d"}
09:31:49 [09:31:49.029]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81ce95c4aed9334f epochNumber=0
{"provingJobId":"1:BASE_PARITY:81ce95c4aed9334f"}
09:31:49 [09:31:49.046]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.050]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.051]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:61c6902a04efa374 epochNumber=2
{"provingJobId":"1:BASE_PARITY:61c6902a04efa374"}
09:31:49 [09:31:49.052]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXbBgjm/2 with map size 1073741824
09:31:49 [09:31:49.059]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.079]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a41fa401ee5cd02d epochNumber=1
{"provingJobId":"1:BASE_PARITY:a41fa401ee5cd02d"}
09:31:49 [09:31:49.080]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXbBgjm/1 with map size 1073741824
09:31:49 [09:31:49.082]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.105]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.106]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c89dd61b6aa3be35 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c89dd61b6aa3be35"}
09:31:49 [09:31:49.107]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3BrtAl/1 with map size 1073741824
09:31:49 [09:31:49.111]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.135]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.136]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:87da3a52101d259c epochNumber=1
{"provingJobId":"1:BASE_PARITY:87da3a52101d259c"}
09:31:49 [09:31:49.137]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3AUY1N/1 with map size 1073741824
09:31:49 [09:31:49.139]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.160]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ba77e6907317363 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9ba77e6907317363"}
09:31:49 [09:31:49.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8f39e4cf70eae330 epochNumber=2
{"provingJobId":"1:BASE_PARITY:8f39e4cf70eae330"}
09:31:49 [09:31:49.175]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3AUY1N/2 with map size 1073741824
09:31:49 [09:31:49.177]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.199]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2356212ef3c71c1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2356212ef3c71c1"}
09:31:49 [09:31:49.208]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.209]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fadc8cd9e5bc65c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fadc8cd9e5bc65c3"}
09:31:49 [09:31:49.210]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTmYTSS/1 with map size 1073741824
09:31:49 [09:31:49.211]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.230]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3a60943faef6525e epochNumber=1
{"provingJobId":"1:BASE_PARITY:3a60943faef6525e"}
09:31:49 [09:31:49.236]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:827eb0d94a61bbef epochNumber=2
{"provingJobId":"1:BASE_PARITY:827eb0d94a61bbef"}
09:31:49 [09:31:49.237]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTmYTSS/2 with map size 1073741824
09:31:49 [09:31:49.240]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.264]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f092bba2c782bf9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f092bba2c782bf9"}
09:31:49 [09:31:49.277]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.278]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b8a14a96c5a6cefe epochNumber=1
{"provingJobId":"1:BASE_PARITY:b8a14a96c5a6cefe"}
09:31:49 [09:31:49.279]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM7ZKPQ/1 with map size 1073741824
09:31:49 [09:31:49.280]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.296]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36cbb49d86fc0dda epochNumber=1
{"provingJobId":"1:BASE_PARITY:36cbb49d86fc0dda"}
09:31:49 [09:31:49.303]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:639bbbc4414699fa epochNumber=2
{"provingJobId":"1:BASE_PARITY:639bbbc4414699fa"}
09:31:49 [09:31:49.304]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testM7ZKPQ/2 with map size 1073741824
09:31:49 [09:31:49.306]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.323]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1847bcb93fa17392 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1847bcb93fa17392"}
09:31:49 [09:31:49.333]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.334]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5aff25151f10316d epochNumber=1
{"provingJobId":"1:BASE_PARITY:5aff25151f10316d"}
09:31:49 [09:31:49.334]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGnRKLl/1 with map size 1073741824
09:31:49 [09:31:49.336]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.355]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b9be64ee002719c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b9be64ee002719c2"}
09:31:49 [09:31:49.363]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5aff25151f10316d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5aff25151f10316d"}
09:31:49 [09:31:49.377]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.378]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b7afe5fbf6decea epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b7afe5fbf6decea"}
09:31:49 [09:31:49.379]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMZMlZ1/1 with map size 1073741824
09:31:49 [09:31:49.381]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.401]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:778fb75c3575b29e epochNumber=1
{"provingJobId":"1:BASE_PARITY:778fb75c3575b29e"}
09:31:49 [09:31:49.407]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8b7afe5fbf6decea type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:8b7afe5fbf6decea"}
09:31:49 [09:31:49.416]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.417]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:825c160766e6606e epochNumber=1
{"provingJobId":"1:BASE_PARITY:825c160766e6606e"}
09:31:49 [09:31:49.418]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtwbKoU/1 with map size 1073741824
09:31:49 [09:31:49.419]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.435]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:99bcb00b6a72e395 epochNumber=1
{"provingJobId":"1:BASE_PARITY:99bcb00b6a72e395"}
09:31:49 [09:31:49.441]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:825c160766e6606e type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:825c160766e6606e"}
09:31:49 [09:31:49.444]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f10553bed2a5a640 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f10553bed2a5a640"}
09:31:49 [09:31:49.446]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdmCQbV/1 with map size 1073741824
09:31:49 [09:31:49.447]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.468]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:f10553bed2a5a640
{"provingJobId":"1:BASE_PARITY:f10553bed2a5a640"}
09:31:49 [09:31:49.469]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f10553bed2a5a640 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:f10553bed2a5a640"}
09:31:49 [09:31:49.476]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81782b50b6f43d18 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81782b50b6f43d18"}
09:31:49 [09:31:49.483]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f10553bed2a5a640 has already been completed
{"provingJobId":"1:BASE_PARITY:f10553bed2a5a640"}
09:31:49 [09:31:49.491]
INFO:
prover-client:proving-broker Proving Broker started
09:31:49 [09:31:49.492]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2b192dac81b39a21 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2b192dac81b39a21"}
09:31:49 [09:31:49.493]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testotqMJk/1 with map size 1073741824
09:31:49 [09:31:49.495]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:49 [09:31:49.512]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:34f7477849b492cc epochNumber=2
{"provingJobId":"1:BASE_PARITY:34f7477849b492cc"}
09:31:49 [09:31:49.513]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testotqMJk/2 with map size 1073741824
09:31:49 [09:31:49.518]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:50 [09:31:50.590]
INFO:
prover-client:proving-broker Proving Broker started
09:31:50 [09:31:50.596]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2b192dac81b39a21 settled=false
{"provingJobId":"1:BASE_PARITY:2b192dac81b39a21","status":"pending"}
09:31:50 [09:31:50.597]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:34f7477849b492cc settled=false
{"provingJobId":"1:BASE_PARITY:34f7477849b492cc","status":"pending"}
09:31:50 [09:31:50.598]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b192dac81b39a21 type=BASE_PARITY startedAt=1750066309538 older agent has taken job
{"provingJobId":"1:BASE_PARITY:2b192dac81b39a21"}
09:31:50 [09:31:50.598]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2b192dac81b39a21 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:2b192dac81b39a21"}
09:31:50 [09:31:50.603]
INFO:
prover-client:proving-broker Proving Broker started
09:31:50 [09:31:50.604]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fb98263b42570202 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fb98263b42570202"}
09:31:50 [09:31:50.605]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6J7EVq/1 with map size 1073741824
09:31:50 [09:31:50.607]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:50 [09:31:50.630]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d3b7b2dc2156bf2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3d3b7b2dc2156bf2"}
09:31:50 [09:31:50.630]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test6J7EVq/2 with map size 1073741824
09:31:50 [09:31:50.632]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:51 [09:31:51.650]
INFO:
prover-client:proving-broker Proving Broker started
09:31:51 [09:31:51.651]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:fb98263b42570202 settled=false
{"provingJobId":"1:BASE_PARITY:fb98263b42570202","status":"pending"}
09:31:51 [09:31:51.653]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3d3b7b2dc2156bf2 settled=false
{"provingJobId":"1:BASE_PARITY:3d3b7b2dc2156bf2","status":"pending"}
09:31:51 [09:31:51.653]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fb98263b42570202 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:fb98263b42570202"}
09:31:51 [09:31:51.657]
INFO:
prover-client:proving-broker Proving Broker started
09:31:51 [09:31:51.658]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0802380c5b3b5204 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0802380c5b3b5204"}
09:31:51 [09:31:51.660]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNVLspV/1 with map size 1073741824
09:31:51 [09:31:51.662]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:51 [09:31:51.678]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2da759c5a31936cf epochNumber=2
{"provingJobId":"1:BASE_PARITY:2da759c5a31936cf"}
09:31:51 [09:31:51.679]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testNVLspV/2 with map size 1073741824
09:31:51 [09:31:51.681]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.699]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.707]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0802380c5b3b5204 settled=false
{"provingJobId":"1:BASE_PARITY:0802380c5b3b5204","status":"pending"}
09:31:52 [09:31:52.709]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2da759c5a31936cf settled=false
{"provingJobId":"1:BASE_PARITY:2da759c5a31936cf","status":"pending"}
09:31:52 [09:31:52.711]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0802380c5b3b5204 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:0802380c5b3b5204"}
09:31:52 [09:31:52.711]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0802380c5b3b5204 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0802380c5b3b5204"}
09:31:52 [09:31:52.721]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.722]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d566c54acb717c8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d566c54acb717c8"}
09:31:52 [09:31:52.723]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSUWTO6/1 with map size 1073741824
09:31:52 [09:31:52.731]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.750]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e8a2d0317f01cae3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:e8a2d0317f01cae3"}
09:31:52 [09:31:52.751]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSUWTO6/2 with map size 1073741824
09:31:52 [09:31:52.756]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.782]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0d566c54acb717c8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0d566c54acb717c8"}
09:31:52 [09:31:52.789]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e8a2d0317f01cae3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e8a2d0317f01cae3"}
09:31:52 [09:31:52.802]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.803]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7db592e4c2c7bb02 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7db592e4c2c7bb02"}
09:31:52 [09:31:52.804]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPGttWk/1 with map size 1073741824
09:31:52 [09:31:52.806]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.823]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4bcee895db0388fb epochNumber=2
{"provingJobId":"1:BASE_PARITY:4bcee895db0388fb"}
09:31:52 [09:31:52.824]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPGttWk/2 with map size 1073741824
09:31:52 [09:31:52.825]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.847]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7db592e4c2c7bb02 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7db592e4c2c7bb02"}
09:31:52 [09:31:52.847]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7db592e4c2c7bb02 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7db592e4c2c7bb02"}
09:31:52 [09:31:52.860]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4bcee895db0388fb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4bcee895db0388fb"}
09:31:52 [09:31:52.875]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.876]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a070d83c5c8dd248 err=test error
{"provingJoId":"1:BASE_PARITY:a070d83c5c8dd248"}
09:31:52 [09:31:52.881]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.882]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c217b6be20af55f0 not found
{"provingJobId":"1:BASE_PARITY:c217b6be20af55f0"}
09:31:52 [09:31:52.886]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.887]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b90ca8e5f49c903 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7b90ca8e5f49c903"}
09:31:52 [09:31:52.889]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKRJxJZ/1 with map size 1073741824
09:31:52 [09:31:52.893]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:52 [09:31:52.923]
INFO:
prover-client:proving-broker Proving Broker started
09:31:52 [09:31:52.924]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:95b6bceb77bb3b2b epochNumber=1
{"provingJobId":"1:BASE_PARITY:95b6bceb77bb3b2b"}
09:31:52 [09:31:52.926]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzfu23t/1 with map size 1073741824
09:31:52 [09:31:52.929]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.077]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:95b6bceb77bb3b2b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:95b6bceb77bb3b2b"}
09:31:53 [09:31:53.081]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.082]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:328f0e175c3ed96e epochNumber=1
{"provingJobId":"1:BASE_PARITY:328f0e175c3ed96e"}
09:31:53 [09:31:53.083]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGnfMgH/1 with map size 1073741824
09:31:53 [09:31:53.084]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.232]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:328f0e175c3ed96e timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:328f0e175c3ed96e"}
09:31:53 [09:31:53.239]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:75ff606b93138e61 epochNumber=10
{"provingJobId":"1:BASE_PARITY:75ff606b93138e61"}
09:31:53 [09:31:53.241]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testGnfMgH/10 with map size 1073741824
09:31:53 [09:31:53.245]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.284]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:53 [09:31:53.284]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:31:53 [09:31:53.401]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.402]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:85db55d50ce130a2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:85db55d50ce130a2"}
09:31:53 [09:31:53.403]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD9GOZK/1 with map size 1073741824
09:31:53 [09:31:53.405]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.584]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:85db55d50ce130a2 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:85db55d50ce130a2"}
09:31:53 [09:31:53.636]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.636]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:29966977cda8b2d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:29966977cda8b2d5"}
09:31:53 [09:31:53.637]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTi0bGU/1 with map size 1073741824
09:31:53 [09:31:53.639]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.663]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:29966977cda8b2d5 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:29966977cda8b2d5"}
09:31:53 [09:31:53.674]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.675]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3abde7186e1f6c71 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3abde7186e1f6c71"}
09:31:53 [09:31:53.675]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0lYl5s/1 with map size 1073741824
09:31:53 [09:31:53.678]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.702]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3abde7186e1f6c71 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3abde7186e1f6c71"}
09:31:53 [09:31:53.703]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3abde7186e1f6c71 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:3abde7186e1f6c71"}
09:31:53 [09:31:53.715]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.716]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fdc21799453dc97b epochNumber=1
{"provingJobId":"1:BASE_PARITY:fdc21799453dc97b"}
09:31:53 [09:31:53.717]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmYk6WM/1 with map size 1073741824
09:31:53 [09:31:53.720]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.738]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fdc21799453dc97b type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fdc21799453dc97b"}
09:31:53 [09:31:53.749]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.750]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:691ed03a7c34892f epochNumber=1
{"provingJobId":"1:BASE_PARITY:691ed03a7c34892f"}
09:31:53 [09:31:53.752]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBlfWAJ/1 with map size 1073741824
09:31:53 [09:31:53.757]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.777]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:691ed03a7c34892f type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:691ed03a7c34892f"}
09:31:53 [09:31:53.778]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5797ce2c060aa999 epochNumber=3
{"provingJobId":"1:BASE_PARITY:5797ce2c060aa999"}
09:31:53 [09:31:53.779]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testBlfWAJ/3 with map size 1073741824
09:31:53 [09:31:53.782]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.826]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:53 [09:31:53.827]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:31:53 [09:31:53.829]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:691ed03a7c34892f err=test error
{"provingJoId":"1:BASE_PARITY:691ed03a7c34892f"}
09:31:53 [09:31:53.835]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testut9wDg/1 with map size 1073741824
09:31:53 [09:31:53.840]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.860]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testut9wDg/2 with map size 1073741824
09:31:53 [09:31:53.863]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.889]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.890]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:07878b8e71ab6e35 settled=false
{"provingJobId":"1:BASE_PARITY:07878b8e71ab6e35","status":"pending"}
09:31:53 [09:31:53.892]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6313b150f0298860 settled=false
{"provingJobId":"1:BASE_PARITY:6313b150f0298860","status":"pending"}
09:31:53 [09:31:53.898]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJVXaxx/1 with map size 1073741824
09:31:53 [09:31:53.899]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.922]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testJVXaxx/2 with map size 1073741824
09:31:53 [09:31:53.924]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:53 [09:31:53.962]
INFO:
prover-client:proving-broker Proving Broker started
09:31:53 [09:31:53.972]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3be0284a7799e479 settled=true
{"provingJobId":"1:BASE_PARITY:3be0284a7799e479","status":"fulfilled"}
09:31:53 [09:31:53.975]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:089fb9e38dda4491 settled=true
{"provingJobId":"2:BASE_PARITY:089fb9e38dda4491","status":"fulfilled"}
09:31:53 [09:31:53.980]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testltH42a/1 with map size 1073741824
09:31:53 [09:31:53.984]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.016]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testltH42a/2 with map size 1073741824
09:31:54 [09:31:54.020]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.034]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.036]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8e287c61cc15bdff settled=true
{"provingJobId":"1:BASE_PARITY:8e287c61cc15bdff","status":"fulfilled"}
09:31:54 [09:31:54.039]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ad6ab0c679b80d86 settled=false
{"provingJobId":"1:BASE_PARITY:ad6ab0c679b80d86","status":"pending"}
09:31:54 [09:31:54.045]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.045]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3ff054a0d4d388db epochNumber=1
{"provingJobId":"1:BASE_PARITY:3ff054a0d4d388db"}
09:31:54 [09:31:54.046]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6zcCwb/1 with map size 1073741824
09:31:54 [09:31:54.048]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.079]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.080]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:350996f5e92e1535 epochNumber=1
{"provingJobId":"1:BASE_PARITY:350996f5e92e1535"}
09:31:54 [09:31:54.081]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:350996f5e92e1535: Error: db error: Error: db error
09:31:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:350996f5e92e1535"}
09:31:54 [09:31:54.089]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.090]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e44c7cdfc7213cd6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e44c7cdfc7213cd6"}
09:31:54 [09:31:54.091]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHTyv5T/1 with map size 1073741824
09:31:54 [09:31:54.096]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.119]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e44c7cdfc7213cd6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e44c7cdfc7213cd6"}
09:31:54 [09:31:54.120]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e44c7cdfc7213cd6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e44c7cdfc7213cd6"}
09:31:54 [09:31:54.129]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.130]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9f4a017f6f692224 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9f4a017f6f692224"}
09:31:54 [09:31:54.131]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsy3KuO/1 with map size 1073741824
09:31:54 [09:31:54.134]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.154]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9f4a017f6f692224 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9f4a017f6f692224"}
09:31:54 [09:31:54.155]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9f4a017f6f692224 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9f4a017f6f692224"}
09:31:54 [09:31:54.156]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:9f4a017f6f692224: Error: db error
09:31:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:9f4a017f6f692224"}
09:31:54 [09:31:54.161]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.162]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac4e820f6cbf91f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ac4e820f6cbf91f9"}
09:31:54 [09:31:54.163]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPIjPmC/1 with map size 1073741824
09:31:54 [09:31:54.165]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.193]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ac4e820f6cbf91f9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ac4e820f6cbf91f9"}
09:31:54 [09:31:54.194]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ac4e820f6cbf91f9 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ac4e820f6cbf91f9"}
09:31:54 [09:31:54.207]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.208]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:191b0092b784a2c3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:191b0092b784a2c3"}
09:31:54 [09:31:54.212]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1KANZw/1 with map size 1073741824
09:31:54 [09:31:54.214]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.240]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:191b0092b784a2c3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:191b0092b784a2c3"}
09:31:54 [09:31:54.241]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:191b0092b784a2c3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:191b0092b784a2c3"}
09:31:54 [09:31:54.241]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:191b0092b784a2c3 jobErr=test error: Error: db error
09:31:54 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:191b0092b784a2c3"}
09:31:54 [09:31:54.249]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.250]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b032d796e1690430 not found
{"provingJobId":"1:BASE_PARITY:b032d796e1690430"}
09:31:54 [09:31:54.254]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.255]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f19f146e0c0d2a30 err=test error
{"provingJoId":"1:BASE_PARITY:f19f146e0c0d2a30"}
09:31:54 [09:31:54.262]
INFO:
prover-client:proving-broker Proving Broker started
09:31:54 [09:31:54.274]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f4666a04a48aefd7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f4666a04a48aefd7"}
09:31:54 [09:31:54.275]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2PCWp7/1 with map size 1073741824
09:31:54 [09:31:54.277]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.309]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f4666a04a48aefd7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f4666a04a48aefd7"}
09:31:54 [09:31:54.309]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f4666a04a48aefd7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f4666a04a48aefd7"}
09:31:54 [09:31:54.327]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:259fe5cd6a1001ff epochNumber=2
{"provingJobId":"2:BASE_PARITY:259fe5cd6a1001ff"}
09:31:54 [09:31:54.330]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test2PCWp7/2 with map size 1073741824
09:31:54 [09:31:54.334]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.363]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:259fe5cd6a1001ff type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:259fe5cd6a1001ff"}
09:31:54 [09:31:54.363]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:259fe5cd6a1001ff type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:259fe5cd6a1001ff"}
09:31:54 [09:31:54.436]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:cbf508b4a3683655 epochNumber=3
{"provingJobId":"3:BASE_PARITY:cbf508b4a3683655"}
09:31:54 [09:31:54.437]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test2PCWp7/3 with map size 1073741824
09:31:54 [09:31:54.439]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.495]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:54 [09:31:54.495]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
09:31:54 [09:31:54.532]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:566be2491d92ed31 epochNumber=4
{"provingJobId":"4:BASE_PARITY:566be2491d92ed31"}
09:31:54 [09:31:54.533]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-test2PCWp7/4 with map size 1073741824
09:31:54 [09:31:54.539]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.580]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:54 [09:31:54.581]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
09:31:54 [09:31:54.630]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:32b7b32980e9212a epochNumber=5
{"provingJobId":"5:BASE_PARITY:32b7b32980e9212a"}
09:31:54 [09:31:54.633]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-test2PCWp7/5 with map size 1073741824
09:31:54 [09:31:54.637]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
09:31:54 [09:31:54.685]
INFO:
prover-client:proving-broker Cleaned up jobs=1
09:31:54 [09:31:54.685]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
09:31:54
PASS src/proving_broker/proving_broker.test.ts (
13.327 s)
09:31:54 ProvingBroker
09:31:54 Producer API
09:31:54
✓ refuses stale jobs (46 ms)
09:31:54
✓ enqueues jobs (5 ms)
09:31:54
✓ ignores duplicate jobs (5 ms)
09:31:54
✓ reports correct status when enqueuing repeat jobs (5 ms)
09:31:54
✓ throws an error in case of duplicate job IDs (26 ms)
09:31:54
✓ returns not-found status for non-existing jobs (2 ms)
09:31:54
✓ cancels jobs in queue (4 ms)
09:31:54
✓ cancels jobs in-progress (10 ms)
09:31:54
✓ returns job result if successful (4 ms)
09:31:54
✓ returns job error if failed (3 ms)
09:31:54
✓ correctly returns job status for concurrent writes (3 ms)
09:31:54
✓ refuses stale jobs (35 ms)
09:31:54
✓ enqueues jobs (38 ms)
09:31:54
✓ ignores duplicate jobs (36 ms)
09:31:54
✓ reports correct status when enqueuing repeat jobs (36 ms)
09:31:54
✓ throws an error in case of duplicate job IDs (30 ms)
09:31:54
✓ returns not-found status for non-existing jobs (6 ms)
09:31:54
✓ cancels jobs in queue (34 ms)
09:31:54
✓ cancels jobs in-progress (31 ms)
09:31:54
✓ returns job result if successful (38 ms)
09:31:54
✓ returns job error if failed (31 ms)
09:31:54
✓ correctly returns job status for concurrent writes (40 ms)
09:31:54 Consumer API
09:31:54
✓ returns undefined if no jobs are available (1 ms)
09:31:54
✓ returns jobs in priority order (2 ms)
09:31:54
✓ returns undefined if no jobs are available for the given allowList (1 ms)
09:31:54
✓ returns a job if it is in the allowList (2 ms)
09:31:54
✓ returns the most important job if it is in the allowList (3 ms)
09:31:54
✓ returns any job if filter is empty (2 ms)
09:31:54
✓ returns a new job when reporting job success (2 ms)
09:31:54
✓ returns a new job when reporting permanent error (2 ms)
09:31:54
✓ returns a new job when reporting retry-able error (2 ms)
09:31:54
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
09:31:54
✓ returns a new job if job is already in progress elsewhere (1055 ms)
09:31:54
✓ avoids sending the same job to a new agent after a restart (1004 ms)
09:31:54
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
09:31:54
✓ tracks job result if in progress (2 ms)
09:31:54
✓ tracks job result even if job is in queue (4 ms)
09:31:54
✓ ignores reported job error if unknown job (1 ms)
09:31:54
✓ ignores job result if unknown job (2 ms)
09:31:54
✓ returns undefined if no jobs are available (5 ms)
09:31:54
✓ returns jobs in priority order (55 ms)
09:31:54
✓ returns undefined if no jobs are available for the given allowList (31 ms)
09:31:54
✓ returns a job if it is in the allowList (72 ms)
09:31:54
✓ returns the most important job if it is in the allowList (68 ms)
09:31:54
✓ returns any job if filter is empty (55 ms)
09:31:54
✓ returns a new job when reporting job success (44 ms)
09:31:54
✓ returns a new job when reporting permanent error (38 ms)
09:31:54
✓ returns a new job when reporting retry-able error (29 ms)
09:31:54
✓ returns a new job when reporting progress if current one is cancelled (45 ms)
09:31:54
✓ returns a new job if job is already in progress elsewhere (1113 ms)
09:31:54
✓ avoids sending the same job to a new agent after a restart (1053 ms)
09:31:54
✓ avoids sending a completed job to a new agent after a restart (1064 ms)
09:31:54
✓ tracks job result if in progress (80 ms)
09:31:54
✓ tracks job result even if job is in queue (73 ms)
09:31:54
✓ ignores reported job error if unknown job (3 ms)
09:31:54
✓ ignores job result if unknown job (6 ms)
09:31:54 Timeouts
09:31:54
✓ tracks in progress jobs (1 ms)
09:31:54
✓ re-enqueues jobs that time out (108 ms)
09:31:54
✓ cancel stale jobs that time out (231 ms)
09:31:54
✓ keeps the jobs in progress while it is alive (204 ms)
09:31:54
✓ tracks in progress jobs (35 ms)
09:31:54
✓ re-enqueues jobs that time out (159 ms)
09:31:54
✓ cancel stale jobs that time out (319 ms)
09:31:54
✓ keeps the jobs in progress while it is alive (235 ms)
09:31:54 Retries
09:31:54
✓ retries jobs (4 ms)
09:31:54
✓ retries up to a maximum number of times (4 ms)
09:31:54
✓ passing retry=false does not retry (2 ms)
09:31:54
✓ does not retry if job is stale (27 ms)
09:31:54
✓ retries jobs (32 ms)
09:31:54
✓ retries up to a maximum number of times (42 ms)
09:31:54
✓ passing retry=false does not retry (35 ms)
09:31:54
✓ does not retry if job is stale (83 ms)
09:31:54 Database management
09:31:54
✓ re-enqueues proof requests on start (2 ms)
09:31:54
✓ restores proof results on start (2 ms)
09:31:54
✓ only re-enqueues unfinished jobs (1 ms)
09:31:54
✓ saves job when enqueued (3 ms)
09:31:54
✓ does not retain job if database fails to save (1 ms)
09:31:54
✓ saves job result (1 ms)
09:31:54
✓ saves result even if database fails to save (2 ms)
09:31:54
✓ saves job error (2 ms)
09:31:54
✓ saves job error even if database fails to save (1 ms)
09:31:54
✓ does not save job result if job is unknown (1 ms)
09:31:54
✓ does not save job error if job is unknown (1 ms)
09:31:54
✓ cleans up old jobs periodically (263 ms)
09:31:54
✓ re-enqueues proof requests on start (64 ms)
09:31:54
✓ restores proof results on start (82 ms)
09:31:54
✓ only re-enqueues unfinished jobs (64 ms)
09:31:54
✓ saves job when enqueued (34 ms)
09:31:54
✓ does not retain job if database fails to save (7 ms)
09:31:54
✓ saves job result (43 ms)
09:31:54
✓ saves result even if database fails to save (31 ms)
09:31:54
✓ saves job error (45 ms)
09:31:54
✓ saves job error even if database fails to save (41 ms)
09:31:54
✓ does not save job result if job is unknown (8 ms)
09:31:54
✓ does not save job error if job is unknown (4 ms)
09:31:54
✓ cleans up old jobs periodically (458 ms)
09:31:54
09:31:54
Test Suites: 1 passed, 1 total
09:31:54
Tests: 96 passed, 96 total
09:31:54
Snapshots: 0 total
09:31:54
Time: 13.428 s
09:31:54
Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i
.
09:31:54
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?