Parent Log: http://ci.aztec-labs.com/5285bbb6426f906e Command: 7ad4152ecf607d72 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/15c121440ebf929fded7bd0c31960fb154a3fa7a Env: REF_NAME=gh-readonly-queue/next/pr-14554-befc8c4c9ab253911288e1ef4bdd56c5ae8eeb9b CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Wed Jun 11 22:59:19 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-14554_amd64_x2-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 22:59:22 [22:59:22.821] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.831] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:49f770a16e7b6460 epochNumber=42 {"provingJobId":"1:BASE_PARITY:49f770a16e7b6460"} 22:59:22 [22:59:22.835] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:7bbeb49311a8fd62 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7bbeb49311a8fd62"} 22:59:22 [22:59:22.883] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.884] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e8a752205a591f9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e8a752205a591f9"} 22:59:22 [22:59:22.886] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:365c6ae38334c4b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:365c6ae38334c4b2"} 22:59:22 [22:59:22.888] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.889] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0e1a720af6f66942 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0e1a720af6f66942"} 22:59:22 [22:59:22.890] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0e1a720af6f66942 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:0e1a720af6f66942"} 22:59:22 [22:59:22.892] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.893] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a6af16da2f8b67f epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a6af16da2f8b67f"} 22:59:22 [22:59:22.894] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2a6af16da2f8b67f epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2a6af16da2f8b67f"} 22:59:22 [22:59:22.895] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2a6af16da2f8b67f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2a6af16da2f8b67f"} 22:59:22 [22:59:22.895] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2a6af16da2f8b67f epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2a6af16da2f8b67f"} 22:59:22 [22:59:22.897] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.897] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1718fb05c432b52e epochNumber=1 {"provingJobId":"1:BASE_PARITY:1718fb05c432b52e"} 22:59:22 [22:59:22.927] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.930] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.930] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:25df3adda2ba796e epochNumber=1 {"provingJobId":"1:BASE_PARITY:25df3adda2ba796e"} 22:59:22 [22:59:22.931] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:25df3adda2ba796e {"provingJobId":"1:BASE_PARITY:25df3adda2ba796e"} 22:59:22 [22:59:22.932] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:25df3adda2ba796e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:25df3adda2ba796e"} 22:59:22 [22:59:22.933] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:25df3adda2ba796e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:25df3adda2ba796e"} 22:59:22 [22:59:22.934] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.935] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e44deb8b59990878 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e44deb8b59990878"} 22:59:22 [22:59:22.936] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e44deb8b59990878 {"provingJobId":"1:BASE_PARITY:e44deb8b59990878"} 22:59:22 [22:59:22.936] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e44deb8b59990878 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:e44deb8b59990878"} 22:59:22 [22:59:22.938] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:62fd44e6033bc8fc epochNumber=1 {"provingJobId":"1:BASE_PARITY:62fd44e6033bc8fc"} 22:59:22 [22:59:22.939] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:62fd44e6033bc8fc type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:62fd44e6033bc8fc"} 22:59:22 [22:59:22.940] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:62fd44e6033bc8fc type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:62fd44e6033bc8fc"} 22:59:22 [22:59:22.941] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.942] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:55b0419df59e7fb1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:55b0419df59e7fb1"} 22:59:22 [22:59:22.942] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:55b0419df59e7fb1 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:55b0419df59e7fb1"} 22:59:22 [22:59:22.943] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:55b0419df59e7fb1 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:55b0419df59e7fb1"} 22:59:22 [22:59:22.944] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.945] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:40f0753ac5acf28d epochNumber=0 {"provingJobId":"1:BASE_PARITY:40f0753ac5acf28d"} 22:59:22 [22:59:22.945] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:40f0753ac5acf28d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:40f0753ac5acf28d"} 22:59:22 [22:59:22.945] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e4c56c7a5ec010ea epochNumber=0 {"provingJobId":"1:BASE_PARITY:e4c56c7a5ec010ea"} 22:59:22 [22:59:22.945] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:40f0753ac5acf28d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:40f0753ac5acf28d"} 22:59:22 [22:59:22.946] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:429393a0e6a940a6 epochNumber=0 {"provingJobId":"1:BASE_PARITY:429393a0e6a940a6"} 22:59:22 [22:59:22.947] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.948] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.949] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77169bff77b0c967 epochNumber=2 {"provingJobId":"1:BASE_PARITY:77169bff77b0c967"} 22:59:22 [22:59:22.949] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4dca3baf13a1998 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4dca3baf13a1998"} 22:59:22 [22:59:22.950] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.951] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:75f26cce0db83884 epochNumber=1 {"provingJobId":"1:BASE_PARITY:75f26cce0db83884"} 22:59:22 [22:59:22.961] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.961] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:33e1bbaa0c0fc55a epochNumber=1 {"provingJobId":"1:BASE_PARITY:33e1bbaa0c0fc55a"} 22:59:22 [22:59:22.962] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:26b4e236390fb452 epochNumber=1 {"provingJobId":"1:BASE_PARITY:26b4e236390fb452"} 22:59:22 [22:59:22.962] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:996a067c5f307dea epochNumber=2 {"provingJobId":"1:BASE_PARITY:996a067c5f307dea"} 22:59:22 [22:59:22.963] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:635c74b72ce0c8c5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:635c74b72ce0c8c5"} 22:59:22 [22:59:22.964] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.965] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1036a802a1806831 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1036a802a1806831"} 22:59:22 [22:59:22.965] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf8bb57412845ae0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bf8bb57412845ae0"} 22:59:22 [22:59:22.966] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:693e03f4cc68e0e0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:693e03f4cc68e0e0"} 22:59:22 [22:59:22.966] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3ca05423cc0a900 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e3ca05423cc0a900"} 22:59:22 [22:59:22.967] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.968] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2adf9a9761895f92 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2adf9a9761895f92"} 22:59:22 [22:59:22.968] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a17f4143a31716cd epochNumber=1 {"provingJobId":"1:BASE_PARITY:a17f4143a31716cd"} 22:59:22 [22:59:22.969] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f90f1bfa63e7ef4 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1f90f1bfa63e7ef4"} 22:59:22 [22:59:22.969] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:68129eed0aa44497 epochNumber=1 {"provingJobId":"1:BASE_PARITY:68129eed0aa44497"} 22:59:22 [22:59:22.971] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.971] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ec12472fe4f65f52 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ec12472fe4f65f52"} 22:59:22 [22:59:22.972] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aad7412bc0d0cfab epochNumber=1 {"provingJobId":"1:BASE_PARITY:aad7412bc0d0cfab"} 22:59:22 [22:59:22.972] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ec12472fe4f65f52 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ec12472fe4f65f52"} 22:59:22 [22:59:22.974] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.974] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b3838fb77422517a epochNumber=1 {"provingJobId":"1:BASE_PARITY:b3838fb77422517a"} 22:59:22 [22:59:22.975] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8819a83856af718d epochNumber=1 {"provingJobId":"1:BASE_PARITY:8819a83856af718d"} 22:59:22 [22:59:22.975] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b3838fb77422517a type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:b3838fb77422517a"} 22:59:22 [22:59:22.977] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.977] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2cfb4fbe125cdfbb epochNumber=1 {"provingJobId":"1:BASE_PARITY:2cfb4fbe125cdfbb"} 22:59:22 [22:59:22.978] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d8c412113545745 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6d8c412113545745"} 22:59:22 [22:59:22.978] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:2cfb4fbe125cdfbb type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:2cfb4fbe125cdfbb"} 22:59:22 [22:59:22.979] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.980] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:718d558d65a8cb12 epochNumber=1 {"provingJobId":"1:BASE_PARITY:718d558d65a8cb12"} 22:59:22 [22:59:22.980] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:718d558d65a8cb12 {"provingJobId":"1:BASE_PARITY:718d558d65a8cb12"} 22:59:22 [22:59:22.981] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:718d558d65a8cb12 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:718d558d65a8cb12"} 22:59:22 [22:59:22.981] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d7a225871af0c985 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d7a225871af0c985"} 22:59:22 [22:59:22.982] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:718d558d65a8cb12 has already been completed {"provingJobId":"1:BASE_PARITY:718d558d65a8cb12"} 22:59:22 [22:59:22.983] INFO: prover-client:proving-broker Proving Broker started 22:59:22 [22:59:22.984] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f8c8737d8e6471dd epochNumber=1 {"provingJobId":"1:BASE_PARITY:f8c8737d8e6471dd"} 22:59:22 [22:59:22.984] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9c5f9cda0936fc5b epochNumber=2 {"provingJobId":"1:BASE_PARITY:9c5f9cda0936fc5b"} 22:59:24 [22:59:24.036] INFO: prover-client:proving-broker Proving Broker started 22:59:24 [22:59:24.047] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f8c8737d8e6471dd settled=false {"provingJobId":"1:BASE_PARITY:f8c8737d8e6471dd","status":"pending"} 22:59:24 [22:59:24.051] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9c5f9cda0936fc5b settled=false {"provingJobId":"1:BASE_PARITY:9c5f9cda0936fc5b","status":"pending"} 22:59:24 [22:59:24.053] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:f8c8737d8e6471dd type=BASE_PARITY startedAt=1749682762984 older agent has taken job {"provingJobId":"1:BASE_PARITY:f8c8737d8e6471dd"} 22:59:24 [22:59:24.054] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f8c8737d8e6471dd type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:f8c8737d8e6471dd"} 22:59:24 [22:59:24.057] INFO: prover-client:proving-broker Proving Broker started 22:59:24 [22:59:24.059] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:69c997662b535aec epochNumber=1 {"provingJobId":"1:BASE_PARITY:69c997662b535aec"} 22:59:24 [22:59:24.059] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:065d4ad836c1a0ae epochNumber=2 {"provingJobId":"1:BASE_PARITY:065d4ad836c1a0ae"} 22:59:25 [22:59:25.061] INFO: prover-client:proving-broker Proving Broker started 22:59:25 [22:59:25.062] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:69c997662b535aec settled=false {"provingJobId":"1:BASE_PARITY:69c997662b535aec","status":"pending"} 22:59:25 [22:59:25.063] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:065d4ad836c1a0ae settled=false {"provingJobId":"1:BASE_PARITY:065d4ad836c1a0ae","status":"pending"} 22:59:25 [22:59:25.064] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:69c997662b535aec type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:69c997662b535aec"} 22:59:25 [22:59:25.067] INFO: prover-client:proving-broker Proving Broker started 22:59:25 [22:59:25.068] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15878a29da1444b0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:15878a29da1444b0"} 22:59:25 [22:59:25.069] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e3e1fd2f0246bc81 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e3e1fd2f0246bc81"} 22:59:26 [22:59:26.070] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.071] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:15878a29da1444b0 settled=false {"provingJobId":"1:BASE_PARITY:15878a29da1444b0","status":"pending"} 22:59:26 [22:59:26.072] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e3e1fd2f0246bc81 settled=false {"provingJobId":"1:BASE_PARITY:e3e1fd2f0246bc81","status":"pending"} 22:59:26 [22:59:26.073] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:15878a29da1444b0 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:15878a29da1444b0"} 22:59:26 [22:59:26.073] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:15878a29da1444b0 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:15878a29da1444b0"} 22:59:26 [22:59:26.075] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:16426e9438227e5a epochNumber=1 {"provingJobId":"1:BASE_PARITY:16426e9438227e5a"} 22:59:26 [22:59:26.077] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6a239c8d36665379 epochNumber=2 {"provingJobId":"1:BASE_PARITY:6a239c8d36665379"} 22:59:26 [22:59:26.077] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:16426e9438227e5a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:16426e9438227e5a"} 22:59:26 [22:59:26.078] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6a239c8d36665379 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:6a239c8d36665379"} 22:59:26 [22:59:26.081] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.081] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:49aa5e5157996fd5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:49aa5e5157996fd5"} 22:59:26 [22:59:26.082] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bf58fd12220b6988 epochNumber=2 {"provingJobId":"1:BASE_PARITY:bf58fd12220b6988"} 22:59:26 [22:59:26.082] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:49aa5e5157996fd5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:49aa5e5157996fd5"} 22:59:26 [22:59:26.082] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:49aa5e5157996fd5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:49aa5e5157996fd5"} 22:59:26 [22:59:26.083] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:bf58fd12220b6988 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:bf58fd12220b6988"} 22:59:26 [22:59:26.084] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.085] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:71819ed35dbe6501 err=test error {"provingJoId":"1:BASE_PARITY:71819ed35dbe6501"} 22:59:26 [22:59:26.087] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.087] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8bc0f0fa7d249673 not found {"provingJobId":"1:BASE_PARITY:8bc0f0fa7d249673"} 22:59:26 [22:59:26.089] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.090] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e6539aa9ecb0b1d epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e6539aa9ecb0b1d"} 22:59:26 [22:59:26.091] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.092] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7022b195d49af0f6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7022b195d49af0f6"} 22:59:26 [22:59:26.193] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7022b195d49af0f6 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:7022b195d49af0f6"} 22:59:26 [22:59:26.203] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:524d849f065afe9a epochNumber=1 {"provingJobId":"1:BASE_PARITY:524d849f065afe9a"} 22:59:26 [22:59:26.306] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:524d849f065afe9a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:524d849f065afe9a"} 22:59:26 [22:59:26.309] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d9ab54b286756ad4 epochNumber=10 {"provingJobId":"1:BASE_PARITY:d9ab54b286756ad4"} 22:59:26 [22:59:26.333] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:26 [22:59:26.438] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.439] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d4486bbdc57b9a11 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d4486bbdc57b9a11"} 22:59:26 [22:59:26.617] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d4486bbdc57b9a11 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d4486bbdc57b9a11"} 22:59:26 [22:59:26.647] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.648] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b50b55e9a9351cd6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b50b55e9a9351cd6"} 22:59:26 [22:59:26.650] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b50b55e9a9351cd6 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:b50b55e9a9351cd6"} 22:59:26 [22:59:26.652] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.653] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:491becd2664c366f epochNumber=1 {"provingJobId":"1:BASE_PARITY:491becd2664c366f"} 22:59:26 [22:59:26.655] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:491becd2664c366f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:491becd2664c366f"} 22:59:26 [22:59:26.657] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:491becd2664c366f type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:491becd2664c366f"} 22:59:26 [22:59:26.659] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.660] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a35af834134e67e7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a35af834134e67e7"} 22:59:26 [22:59:26.661] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a35af834134e67e7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:a35af834134e67e7"} 22:59:26 [22:59:26.663] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.664] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:292f15be605c6b44 epochNumber=1 {"provingJobId":"1:BASE_PARITY:292f15be605c6b44"} 22:59:26 [22:59:26.665] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:292f15be605c6b44 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:292f15be605c6b44"} 22:59:26 [22:59:26.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0e00eb48c2d6d445 epochNumber=3 {"provingJobId":"1:BASE_PARITY:0e00eb48c2d6d445"} 22:59:26 [22:59:26.689] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:26 [22:59:26.692] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:292f15be605c6b44 err=test error {"provingJoId":"1:BASE_PARITY:292f15be605c6b44"} 22:59:26 [22:59:26.696] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.696] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bb424cccd6e3f1be settled=false {"provingJobId":"1:BASE_PARITY:bb424cccd6e3f1be","status":"pending"} 22:59:26 [22:59:26.696] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e81be86f1fa0e4b7 settled=false {"provingJobId":"1:BASE_PARITY:e81be86f1fa0e4b7","status":"pending"} 22:59:26 [22:59:26.699] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.700] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6072e3eae6f28930 settled=true {"provingJobId":"1:BASE_PARITY:6072e3eae6f28930","status":"fulfilled"} 22:59:26 [22:59:26.700] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8fcc5708d24f8544 settled=true {"provingJobId":"2:BASE_PARITY:8fcc5708d24f8544","status":"fulfilled"} 22:59:26 [22:59:26.702] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.702] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b5d2fbb099103651 settled=true {"provingJobId":"1:BASE_PARITY:b5d2fbb099103651","status":"fulfilled"} 22:59:26 [22:59:26.703] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:948bfe42eef2509d settled=false {"provingJobId":"1:BASE_PARITY:948bfe42eef2509d","status":"pending"} 22:59:26 [22:59:26.705] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.708] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1181941c8d9852c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1181941c8d9852c8"} 22:59:26 [22:59:26.713] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.714] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4b15024b5edfbd66 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4b15024b5edfbd66"} 22:59:26 [22:59:26.716] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:4b15024b5edfbd66: Error: db error: Error: db error 22:59:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:4b15024b5edfbd66"} 22:59:26 [22:59:26.718] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.719] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:39bec05c157b5d1d epochNumber=1 {"provingJobId":"1:BASE_PARITY:39bec05c157b5d1d"} 22:59:26 [22:59:26.720] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:39bec05c157b5d1d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:39bec05c157b5d1d"} 22:59:26 [22:59:26.720] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:39bec05c157b5d1d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:39bec05c157b5d1d"} 22:59:26 [22:59:26.723] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.724] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:22eae5fd7a3b994e epochNumber=1 {"provingJobId":"1:BASE_PARITY:22eae5fd7a3b994e"} 22:59:26 [22:59:26.725] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:22eae5fd7a3b994e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:22eae5fd7a3b994e"} 22:59:26 [22:59:26.725] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:22eae5fd7a3b994e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:22eae5fd7a3b994e"} 22:59:26 [22:59:26.726] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:22eae5fd7a3b994e: Error: db error 22:59:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:22eae5fd7a3b994e"} 22:59:26 [22:59:26.729] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.729] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c7853a74d69de911 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c7853a74d69de911"} 22:59:26 [22:59:26.730] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c7853a74d69de911 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c7853a74d69de911"} 22:59:26 [22:59:26.730] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c7853a74d69de911 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c7853a74d69de911"} 22:59:26 [22:59:26.732] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.732] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:85ac159ebb8dc28e epochNumber=1 {"provingJobId":"1:BASE_PARITY:85ac159ebb8dc28e"} 22:59:26 [22:59:26.733] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:85ac159ebb8dc28e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:85ac159ebb8dc28e"} 22:59:26 [22:59:26.733] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:85ac159ebb8dc28e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:85ac159ebb8dc28e"} 22:59:26 [22:59:26.734] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:85ac159ebb8dc28e jobErr=test error: Error: db error 22:59:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:85ac159ebb8dc28e"} 22:59:26 [22:59:26.735] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.736] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7a3431f5995f2809 not found {"provingJobId":"1:BASE_PARITY:7a3431f5995f2809"} 22:59:26 [22:59:26.738] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.738] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a390335410a84512 err=test error {"provingJoId":"1:BASE_PARITY:a390335410a84512"} 22:59:26 [22:59:26.740] INFO: prover-client:proving-broker Proving Broker started 22:59:26 [22:59:26.751] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f82bc3fe13277e18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f82bc3fe13277e18"} 22:59:26 [22:59:26.752] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f82bc3fe13277e18 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f82bc3fe13277e18"} 22:59:26 [22:59:26.752] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f82bc3fe13277e18 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:f82bc3fe13277e18"} 22:59:26 [22:59:26.764] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:dbf8dc4ac62b29ab epochNumber=2 {"provingJobId":"2:BASE_PARITY:dbf8dc4ac62b29ab"} 22:59:26 [22:59:26.765] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:dbf8dc4ac62b29ab type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:dbf8dc4ac62b29ab"} 22:59:26 [22:59:26.765] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:dbf8dc4ac62b29ab type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:dbf8dc4ac62b29ab"} 22:59:26 [22:59:26.826] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:decb38d8c7361884 epochNumber=3 {"provingJobId":"3:BASE_PARITY:decb38d8c7361884"} 22:59:26 [22:59:26.842] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:26 [22:59:26.889] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:92c060c0d081efb0 epochNumber=4 {"provingJobId":"4:BASE_PARITY:92c060c0d081efb0"} 22:59:26 [22:59:26.892] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:26 [22:59:26.953] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:dd057fb68f9767d8 epochNumber=5 {"provingJobId":"5:BASE_PARITY:dd057fb68f9767d8"} 22:59:26 [22:59:26.969] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:27 [22:59:27.010] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.011] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b9dec54dbca91cd8 epochNumber=42 {"provingJobId":"1:BASE_PARITY:b9dec54dbca91cd8"} 22:59:27 [22:59:27.012] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testzFhzCG/42 with map size 1073741824 22:59:27 [22:59:27.018] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.048] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:b2b5e79ca3196314 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b2b5e79ca3196314"} 22:59:27 [22:59:27.056] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.057] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fd99fcd981238981 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fd99fcd981238981"} 22:59:27 [22:59:27.058] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjpg16t/1 with map size 1073741824 22:59:27 [22:59:27.060] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.081] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:38e822e88ff9b069 epochNumber=1 {"provingJobId":"1:BASE_PARITY:38e822e88ff9b069"} 22:59:27 [22:59:27.093] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.094] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4c5abb136ded5da7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4c5abb136ded5da7"} 22:59:27 [22:59:27.095] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvbsXNn/1 with map size 1073741824 22:59:27 [22:59:27.098] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.117] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4c5abb136ded5da7 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:4c5abb136ded5da7"} 22:59:27 [22:59:27.122] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.123] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:977a8d47ee291e05 epochNumber=1 {"provingJobId":"1:BASE_PARITY:977a8d47ee291e05"} 22:59:27 [22:59:27.124] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhr8MXW/1 with map size 1073741824 22:59:27 [22:59:27.126] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.145] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:977a8d47ee291e05 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:977a8d47ee291e05"} 22:59:27 [22:59:27.146] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:977a8d47ee291e05 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:977a8d47ee291e05"} 22:59:27 [22:59:27.156] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:977a8d47ee291e05 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:977a8d47ee291e05"} 22:59:27 [22:59:27.164] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5cd08334fd5ec63a epochNumber=1 {"provingJobId":"1:BASE_PARITY:5cd08334fd5ec63a"} 22:59:27 [22:59:27.166] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVEbeFo/1 with map size 1073741824 22:59:27 [22:59:27.168] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.193] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.197] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.197] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:322258baf494735b epochNumber=1 {"provingJobId":"1:BASE_PARITY:322258baf494735b"} 22:59:27 [22:59:27.198] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpKZZhX/1 with map size 1073741824 22:59:27 [22:59:27.201] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.222] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:322258baf494735b {"provingJobId":"1:BASE_PARITY:322258baf494735b"} 22:59:27 [22:59:27.223] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:322258baf494735b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:322258baf494735b"} 22:59:27 [22:59:27.223] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:322258baf494735b type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:322258baf494735b"} 22:59:27 [22:59:27.236] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.237] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d61bdb3310ef675f epochNumber=1 {"provingJobId":"1:BASE_PARITY:d61bdb3310ef675f"} 22:59:27 [22:59:27.240] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testE1T9Sf/1 with map size 1073741824 22:59:27 [22:59:27.245] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.266] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d61bdb3310ef675f {"provingJobId":"1:BASE_PARITY:d61bdb3310ef675f"} 22:59:27 [22:59:27.266] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d61bdb3310ef675f type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:d61bdb3310ef675f"} 22:59:27 [22:59:27.276] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.277] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:986456c476e91b8d epochNumber=1 {"provingJobId":"1:BASE_PARITY:986456c476e91b8d"} 22:59:27 [22:59:27.278] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQOSxA6/1 with map size 1073741824 22:59:27 [22:59:27.283] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.299] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:986456c476e91b8d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:986456c476e91b8d"} 22:59:27 [22:59:27.300] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:986456c476e91b8d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:986456c476e91b8d"} 22:59:27 [22:59:27.310] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.310] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0aaa3601ccc578c7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0aaa3601ccc578c7"} 22:59:27 [22:59:27.311] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7qV0WJ/1 with map size 1073741824 22:59:27 [22:59:27.314] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.335] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:0aaa3601ccc578c7 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:0aaa3601ccc578c7"} 22:59:27 [22:59:27.335] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0aaa3601ccc578c7 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:0aaa3601ccc578c7"} 22:59:27 [22:59:27.348] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.349] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:13255bc30fef00a7 epochNumber=0 {"provingJobId":"1:BASE_PARITY:13255bc30fef00a7"} 22:59:27 [22:59:27.349] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testdR6fK5/0 with map size 1073741824 22:59:27 [22:59:27.352] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.374] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:13255bc30fef00a7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:13255bc30fef00a7"} 22:59:27 [22:59:27.375] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c0cc65f5599db659 epochNumber=0 {"provingJobId":"1:BASE_PARITY:c0cc65f5599db659"} 22:59:27 [22:59:27.375] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:13255bc30fef00a7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:13255bc30fef00a7"} 22:59:27 [22:59:27.376] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afcaeaeb6b14053f epochNumber=0 {"provingJobId":"1:BASE_PARITY:afcaeaeb6b14053f"} 22:59:27 [22:59:27.392] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.396] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.397] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0c5bdec99deb4cef epochNumber=2 {"provingJobId":"1:BASE_PARITY:0c5bdec99deb4cef"} 22:59:27 [22:59:27.398] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOkkyUl/2 with map size 1073741824 22:59:27 [22:59:27.400] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.419] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa444d20206c89b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aa444d20206c89b5"} 22:59:27 [22:59:27.420] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOkkyUl/1 with map size 1073741824 22:59:27 [22:59:27.423] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.446] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.447] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:331599ae08be4b76 epochNumber=1 {"provingJobId":"1:BASE_PARITY:331599ae08be4b76"} 22:59:27 [22:59:27.449] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEUNArz/1 with map size 1073741824 22:59:27 [22:59:27.451] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.474] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.475] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cdc83a91cdc8e2ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:cdc83a91cdc8e2ee"} 22:59:27 [22:59:27.475] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testK6rRNz/1 with map size 1073741824 22:59:27 [22:59:27.478] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.498] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d68f0ace7ea61477 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d68f0ace7ea61477"} 22:59:27 [22:59:27.504] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f7e4c50268d8e17f epochNumber=2 {"provingJobId":"1:BASE_PARITY:f7e4c50268d8e17f"} 22:59:27 [22:59:27.505] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testK6rRNz/2 with map size 1073741824 22:59:27 [22:59:27.508] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.526] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:de93c7e48613408a epochNumber=1 {"provingJobId":"1:BASE_PARITY:de93c7e48613408a"} 22:59:27 [22:59:27.537] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.537] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7068fd08513a7a4a epochNumber=1 {"provingJobId":"1:BASE_PARITY:7068fd08513a7a4a"} 22:59:27 [22:59:27.538] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuXAYgp/1 with map size 1073741824 22:59:27 [22:59:27.542] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.563] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2e4f4e3c823b3d66 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2e4f4e3c823b3d66"} 22:59:27 [22:59:27.571] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1e05520cec2ef120 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1e05520cec2ef120"} 22:59:27 [22:59:27.572] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testuXAYgp/2 with map size 1073741824 22:59:27 [22:59:27.575] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.593] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:83184cd306a8b460 epochNumber=1 {"provingJobId":"1:BASE_PARITY:83184cd306a8b460"} 22:59:27 [22:59:27.603] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.604] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c1346f9f307e0eaa epochNumber=1 {"provingJobId":"1:BASE_PARITY:c1346f9f307e0eaa"} 22:59:27 [22:59:27.605] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYXwelr/1 with map size 1073741824 22:59:27 [22:59:27.607] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.624] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eb28246d642b77d8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:eb28246d642b77d8"} 22:59:27 [22:59:27.631] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:793077c20f97089f epochNumber=2 {"provingJobId":"1:BASE_PARITY:793077c20f97089f"} 22:59:27 [22:59:27.632] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testYXwelr/2 with map size 1073741824 22:59:27 [22:59:27.634] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.658] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:49af12ecd2e35550 epochNumber=1 {"provingJobId":"1:BASE_PARITY:49af12ecd2e35550"} 22:59:27 [22:59:27.671] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.671] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:07b6b2dbf2017036 epochNumber=1 {"provingJobId":"1:BASE_PARITY:07b6b2dbf2017036"} 22:59:27 [22:59:27.672] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZX35e0/1 with map size 1073741824 22:59:27 [22:59:27.674] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.691] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:79a8688bb2bcec35 epochNumber=1 {"provingJobId":"1:BASE_PARITY:79a8688bb2bcec35"} 22:59:27 [22:59:27.697] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:07b6b2dbf2017036 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:07b6b2dbf2017036"} 22:59:27 [22:59:27.707] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.708] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:74fd2d928b69fa25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:74fd2d928b69fa25"} 22:59:27 [22:59:27.709] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDU5SQI/1 with map size 1073741824 22:59:27 [22:59:27.711] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.728] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:830369451d16de10 epochNumber=1 {"provingJobId":"1:BASE_PARITY:830369451d16de10"} 22:59:27 [22:59:27.734] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:74fd2d928b69fa25 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:74fd2d928b69fa25"} 22:59:27 [22:59:27.756] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.758] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0833fd3d6eb872ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:0833fd3d6eb872ea"} 22:59:27 [22:59:27.759] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvsm6Gp/1 with map size 1073741824 22:59:27 [22:59:27.762] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.782] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:86c93ecae67b40d1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:86c93ecae67b40d1"} 22:59:27 [22:59:27.788] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0833fd3d6eb872ea type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:0833fd3d6eb872ea"} 22:59:27 [22:59:27.794] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.795] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa1f61d6b7d7d07a epochNumber=1 {"provingJobId":"1:BASE_PARITY:aa1f61d6b7d7d07a"} 22:59:27 [22:59:27.795] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test74opZ3/1 with map size 1073741824 22:59:27 [22:59:27.798] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.817] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:aa1f61d6b7d7d07a {"provingJobId":"1:BASE_PARITY:aa1f61d6b7d7d07a"} 22:59:27 [22:59:27.818] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:aa1f61d6b7d7d07a type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:aa1f61d6b7d7d07a"} 22:59:27 [22:59:27.825] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5baf75c449ba586 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b5baf75c449ba586"} 22:59:27 [22:59:27.831] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:aa1f61d6b7d7d07a has already been completed {"provingJobId":"1:BASE_PARITY:aa1f61d6b7d7d07a"} 22:59:27 [22:59:27.835] INFO: prover-client:proving-broker Proving Broker started 22:59:27 [22:59:27.836] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:12f9ff1081cc40d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:12f9ff1081cc40d3"} 22:59:27 [22:59:27.843] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHwTNB8/1 with map size 1073741824 22:59:27 [22:59:27.848] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:27 [22:59:27.868] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:275aa576cd113a96 epochNumber=2 {"provingJobId":"1:BASE_PARITY:275aa576cd113a96"} 22:59:27 [22:59:27.869] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testHwTNB8/2 with map size 1073741824 22:59:27 [22:59:27.872] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:28 [22:59:28.959] INFO: prover-client:proving-broker Proving Broker started 22:59:28 [22:59:28.968] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:12f9ff1081cc40d3 settled=false {"provingJobId":"1:BASE_PARITY:12f9ff1081cc40d3","status":"pending"} 22:59:28 [22:59:28.970] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:275aa576cd113a96 settled=false {"provingJobId":"1:BASE_PARITY:275aa576cd113a96","status":"pending"} 22:59:28 [22:59:28.972] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:12f9ff1081cc40d3 type=BASE_PARITY startedAt=1749682767891 older agent has taken job {"provingJobId":"1:BASE_PARITY:12f9ff1081cc40d3"} 22:59:28 [22:59:28.972] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:12f9ff1081cc40d3 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:12f9ff1081cc40d3"} 22:59:28 [22:59:28.976] INFO: prover-client:proving-broker Proving Broker started 22:59:28 [22:59:28.977] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:710bd561db139cfc epochNumber=1 {"provingJobId":"1:BASE_PARITY:710bd561db139cfc"} 22:59:28 [22:59:28.978] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSqjVq2/1 with map size 1073741824 22:59:28 [22:59:28.980] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:28 [22:59:28.998] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d7524f76f7a57c0a epochNumber=2 {"provingJobId":"1:BASE_PARITY:d7524f76f7a57c0a"} 22:59:28 [22:59:28.999] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSqjVq2/2 with map size 1073741824 22:59:29 [22:59:29.002] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:30 [22:59:30.020] INFO: prover-client:proving-broker Proving Broker started 22:59:30 [22:59:30.023] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:710bd561db139cfc settled=false {"provingJobId":"1:BASE_PARITY:710bd561db139cfc","status":"pending"} 22:59:30 [22:59:30.025] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d7524f76f7a57c0a settled=false {"provingJobId":"1:BASE_PARITY:d7524f76f7a57c0a","status":"pending"} 22:59:30 [22:59:30.026] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:710bd561db139cfc type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:710bd561db139cfc"} 22:59:30 [22:59:30.031] INFO: prover-client:proving-broker Proving Broker started 22:59:30 [22:59:30.031] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bbac940399ec0a1f epochNumber=1 {"provingJobId":"1:BASE_PARITY:bbac940399ec0a1f"} 22:59:30 [22:59:30.032] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testd470yk/1 with map size 1073741824 22:59:30 [22:59:30.034] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:30 [22:59:30.060] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:675a28b948d54667 epochNumber=2 {"provingJobId":"1:BASE_PARITY:675a28b948d54667"} 22:59:30 [22:59:30.061] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testd470yk/2 with map size 1073741824 22:59:30 [22:59:30.065] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.082] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.096] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bbac940399ec0a1f settled=false {"provingJobId":"1:BASE_PARITY:bbac940399ec0a1f","status":"pending"} 22:59:31 [22:59:31.100] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:675a28b948d54667 settled=false {"provingJobId":"1:BASE_PARITY:675a28b948d54667","status":"pending"} 22:59:31 [22:59:31.103] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:bbac940399ec0a1f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:bbac940399ec0a1f"} 22:59:31 [22:59:31.104] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:bbac940399ec0a1f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:bbac940399ec0a1f"} 22:59:31 [22:59:31.118] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.119] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3cc1716ad1f68d40 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3cc1716ad1f68d40"} 22:59:31 [22:59:31.120] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLeWdyS/1 with map size 1073741824 22:59:31 [22:59:31.123] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.149] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2c007ba38f95be9c epochNumber=2 {"provingJobId":"1:BASE_PARITY:2c007ba38f95be9c"} 22:59:31 [22:59:31.155] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLeWdyS/2 with map size 1073741824 22:59:31 [22:59:31.159] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.181] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3cc1716ad1f68d40 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3cc1716ad1f68d40"} 22:59:31 [22:59:31.188] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2c007ba38f95be9c type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2c007ba38f95be9c"} 22:59:31 [22:59:31.199] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:423e6a82d26f1f5f epochNumber=1 {"provingJobId":"1:BASE_PARITY:423e6a82d26f1f5f"} 22:59:31 [22:59:31.206] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXxAU56/1 with map size 1073741824 22:59:31 [22:59:31.211] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.229] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:030540433dfef4c3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:030540433dfef4c3"} 22:59:31 [22:59:31.230] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXxAU56/2 with map size 1073741824 22:59:31 [22:59:31.235] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.265] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:423e6a82d26f1f5f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:423e6a82d26f1f5f"} 22:59:31 [22:59:31.265] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:423e6a82d26f1f5f type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:423e6a82d26f1f5f"} 22:59:31 [22:59:31.272] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:030540433dfef4c3 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:030540433dfef4c3"} 22:59:31 [22:59:31.283] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.284] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1703c7e9185c4a6a err=test error {"provingJoId":"1:BASE_PARITY:1703c7e9185c4a6a"} 22:59:31 [22:59:31.289] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.290] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:753e92ab91fd6531 not found {"provingJobId":"1:BASE_PARITY:753e92ab91fd6531"} 22:59:31 [22:59:31.294] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.295] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4691e45576f17aee epochNumber=1 {"provingJobId":"1:BASE_PARITY:4691e45576f17aee"} 22:59:31 [22:59:31.296] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testm0DdZT/1 with map size 1073741824 22:59:31 [22:59:31.298] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.324] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.325] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d5f5f378373ecbfd epochNumber=1 {"provingJobId":"1:BASE_PARITY:d5f5f378373ecbfd"} 22:59:31 [22:59:31.326] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3Y6b1r/1 with map size 1073741824 22:59:31 [22:59:31.328] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.452] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d5f5f378373ecbfd timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d5f5f378373ecbfd"} 22:59:31 [22:59:31.467] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.468] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cbf0787259c4b971 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cbf0787259c4b971"} 22:59:31 [22:59:31.469] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvg8JeC/1 with map size 1073741824 22:59:31 [22:59:31.472] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.593] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:cbf0787259c4b971 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:cbf0787259c4b971"} 22:59:31 [22:59:31.600] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:859684326e2b4960 epochNumber=10 {"provingJobId":"1:BASE_PARITY:859684326e2b4960"} 22:59:31 [22:59:31.601] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testvg8JeC/10 with map size 1073741824 22:59:31 [22:59:31.603] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.619] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:31 [22:59:31.620] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 22:59:31 [22:59:31.752] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.755] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20f6403c280e6109 epochNumber=1 {"provingJobId":"1:BASE_PARITY:20f6403c280e6109"} 22:59:31 [22:59:31.759] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8l7fnP/1 with map size 1073741824 22:59:31 [22:59:31.765] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:31 [22:59:31.957] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:20f6403c280e6109 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:20f6403c280e6109"} 22:59:31 [22:59:31.994] INFO: prover-client:proving-broker Proving Broker started 22:59:31 [22:59:31.995] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:887e4b07fac24238 epochNumber=1 {"provingJobId":"1:BASE_PARITY:887e4b07fac24238"} 22:59:31 [22:59:31.996] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGgID7H/1 with map size 1073741824 22:59:31 [22:59:31.999] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.020] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:887e4b07fac24238 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:887e4b07fac24238"} 22:59:32 [22:59:32.026] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.027] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5fb755114fca6557 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5fb755114fca6557"} 22:59:32 [22:59:32.029] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVUSuH5/1 with map size 1073741824 22:59:32 [22:59:32.031] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.063] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:5fb755114fca6557 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:5fb755114fca6557"} 22:59:32 [22:59:32.064] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5fb755114fca6557 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:5fb755114fca6557"} 22:59:32 [22:59:32.074] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.075] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cd6bca79d444d97d epochNumber=1 {"provingJobId":"1:BASE_PARITY:cd6bca79d444d97d"} 22:59:32 [22:59:32.075] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqcBlld/1 with map size 1073741824 22:59:32 [22:59:32.079] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.096] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cd6bca79d444d97d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:cd6bca79d444d97d"} 22:59:32 [22:59:32.103] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.104] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1b41edf313dbda18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1b41edf313dbda18"} 22:59:32 [22:59:32.105] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6CFHAo/1 with map size 1073741824 22:59:32 [22:59:32.108] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.124] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:1b41edf313dbda18 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:1b41edf313dbda18"} 22:59:32 [22:59:32.125] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:443a5a4cd95d9d54 epochNumber=3 {"provingJobId":"1:BASE_PARITY:443a5a4cd95d9d54"} 22:59:32 [22:59:32.126] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test6CFHAo/3 with map size 1073741824 22:59:32 [22:59:32.128] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.155] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:32 [22:59:32.156] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 22:59:32 [22:59:32.173] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1b41edf313dbda18 err=test error {"provingJoId":"1:BASE_PARITY:1b41edf313dbda18"} 22:59:32 [22:59:32.183] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfhp6Oj/1 with map size 1073741824 22:59:32 [22:59:32.186] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.205] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testfhp6Oj/2 with map size 1073741824 22:59:32 [22:59:32.210] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.227] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.229] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d5d5f02b58f3f6dd settled=false {"provingJobId":"1:BASE_PARITY:d5d5f02b58f3f6dd","status":"pending"} 22:59:32 [22:59:32.231] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f11a63cda7a1ff90 settled=false {"provingJobId":"1:BASE_PARITY:f11a63cda7a1ff90","status":"pending"} 22:59:32 [22:59:32.242] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmKLsvl/1 with map size 1073741824 22:59:32 [22:59:32.254] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.276] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testmKLsvl/2 with map size 1073741824 22:59:32 [22:59:32.278] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.309] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.312] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:13e08b72c0244743 settled=true {"provingJobId":"1:BASE_PARITY:13e08b72c0244743","status":"fulfilled"} 22:59:32 [22:59:32.314] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:839cf263431e1c7c settled=true {"provingJobId":"2:BASE_PARITY:839cf263431e1c7c","status":"fulfilled"} 22:59:32 [22:59:32.321] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzoZIKZ/1 with map size 1073741824 22:59:32 [22:59:32.323] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.359] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzoZIKZ/2 with map size 1073741824 22:59:32 [22:59:32.363] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.383] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.386] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:da7c6aa8e6614c6e settled=true {"provingJobId":"1:BASE_PARITY:da7c6aa8e6614c6e","status":"fulfilled"} 22:59:32 [22:59:32.388] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a702c9b25de07343 settled=false {"provingJobId":"1:BASE_PARITY:a702c9b25de07343","status":"pending"} 22:59:32 [22:59:32.394] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.395] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dd1ccbf589f5672e epochNumber=1 {"provingJobId":"1:BASE_PARITY:dd1ccbf589f5672e"} 22:59:32 [22:59:32.396] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testp1TT8o/1 with map size 1073741824 22:59:32 [22:59:32.398] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.423] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.424] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:32924ba50ea6da2b epochNumber=1 {"provingJobId":"1:BASE_PARITY:32924ba50ea6da2b"} 22:59:32 [22:59:32.425] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:32924ba50ea6da2b: Error: db error: Error: db error 22:59:32 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:32924ba50ea6da2b"} 22:59:32 [22:59:32.430] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.430] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2411a71cf275b5ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:2411a71cf275b5ba"} 22:59:32 [22:59:32.431] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testu98f1y/1 with map size 1073741824 22:59:32 [22:59:32.433] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.463] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2411a71cf275b5ba type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2411a71cf275b5ba"} 22:59:32 [22:59:32.464] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2411a71cf275b5ba type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2411a71cf275b5ba"} 22:59:32 [22:59:32.483] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.484] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:06b12f60a9f6a8d6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:06b12f60a9f6a8d6"} 22:59:32 [22:59:32.485] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwN5dZn/1 with map size 1073741824 22:59:32 [22:59:32.488] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.506] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:06b12f60a9f6a8d6 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:06b12f60a9f6a8d6"} 22:59:32 [22:59:32.507] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:06b12f60a9f6a8d6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:06b12f60a9f6a8d6"} 22:59:32 [22:59:32.508] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:06b12f60a9f6a8d6: Error: db error 22:59:32 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:06b12f60a9f6a8d6"} 22:59:32 [22:59:32.514] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.515] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:942ebdd99c4aa0e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:942ebdd99c4aa0e8"} 22:59:32 [22:59:32.516] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcY4DYG/1 with map size 1073741824 22:59:32 [22:59:32.522] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.549] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:942ebdd99c4aa0e8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:942ebdd99c4aa0e8"} 22:59:32 [22:59:32.551] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:942ebdd99c4aa0e8 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:942ebdd99c4aa0e8"} 22:59:32 [22:59:32.574] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.576] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2989f5253582a9ee epochNumber=1 {"provingJobId":"1:BASE_PARITY:2989f5253582a9ee"} 22:59:32 [22:59:32.577] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvKgFHZ/1 with map size 1073741824 22:59:32 [22:59:32.581] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.600] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2989f5253582a9ee type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2989f5253582a9ee"} 22:59:32 [22:59:32.600] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2989f5253582a9ee type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2989f5253582a9ee"} 22:59:32 [22:59:32.601] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:2989f5253582a9ee jobErr=test error: Error: db error 22:59:32 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:2989f5253582a9ee"} 22:59:32 [22:59:32.609] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.610] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4207cb04d7efd00c not found {"provingJobId":"1:BASE_PARITY:4207cb04d7efd00c"} 22:59:32 [22:59:32.616] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.617] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ae6d1eb56822ba10 err=test error {"provingJoId":"1:BASE_PARITY:ae6d1eb56822ba10"} 22:59:32 [22:59:32.621] INFO: prover-client:proving-broker Proving Broker started 22:59:32 [22:59:32.632] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9da372df80150462 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9da372df80150462"} 22:59:32 [22:59:32.634] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnkd6xZ/1 with map size 1073741824 22:59:32 [22:59:32.637] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.666] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9da372df80150462 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:9da372df80150462"} 22:59:32 [22:59:32.666] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9da372df80150462 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9da372df80150462"} 22:59:32 [22:59:32.683] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:60b95c2d9f0c2212 epochNumber=2 {"provingJobId":"2:BASE_PARITY:60b95c2d9f0c2212"} 22:59:32 [22:59:32.685] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testnkd6xZ/2 with map size 1073741824 22:59:32 [22:59:32.693] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.714] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:60b95c2d9f0c2212 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:60b95c2d9f0c2212"} 22:59:32 [22:59:32.715] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:60b95c2d9f0c2212 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:60b95c2d9f0c2212"} 22:59:32 [22:59:32.783] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:bddc3f8af6b63add epochNumber=3 {"provingJobId":"3:BASE_PARITY:bddc3f8af6b63add"} 22:59:32 [22:59:32.784] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testnkd6xZ/3 with map size 1073741824 22:59:32 [22:59:32.791] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.828] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:32 [22:59:32.829] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 22:59:32 [22:59:32.876] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:401ada2d0d111abf epochNumber=4 {"provingJobId":"4:BASE_PARITY:401ada2d0d111abf"} 22:59:32 [22:59:32.878] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testnkd6xZ/4 with map size 1073741824 22:59:32 [22:59:32.882] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:32 [22:59:32.910] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:32 [22:59:32.912] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 22:59:32 [22:59:32.968] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:7f4b75e02b2adb3e epochNumber=5 {"provingJobId":"5:BASE_PARITY:7f4b75e02b2adb3e"} 22:59:32 [22:59:32.969] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testnkd6xZ/5 with map size 1073741824 22:59:32 [22:59:32.972] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 22:59:33 [22:59:33.001] INFO: prover-client:proving-broker Cleaned up jobs=1 22:59:33 [22:59:33.002] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 22:59:33 PASS src/proving_broker/proving_broker.test.ts (12.797 s) 22:59:33 ProvingBroker 22:59:33 Producer API 22:59:33 ✓ refuses stale jobs (64 ms) 22:59:33 ✓ enqueues jobs (5 ms) 22:59:33 ✓ ignores duplicate jobs (4 ms) 22:59:33 ✓ reports correct status when enqueuing repeat jobs (5 ms) 22:59:33 ✓ throws an error in case of duplicate job IDs (30 ms) 22:59:33 ✓ returns not-found status for non-existing jobs (3 ms) 22:59:33 ✓ cancels jobs in queue (4 ms) 22:59:33 ✓ cancels jobs in-progress (3 ms) 22:59:33 ✓ returns job result if successful (3 ms) 22:59:33 ✓ returns job error if failed (2 ms) 22:59:33 ✓ correctly returns job status for concurrent writes (2 ms) 22:59:33 ✓ refuses stale jobs (47 ms) 22:59:33 ✓ enqueues jobs (35 ms) 22:59:33 ✓ ignores duplicate jobs (29 ms) 22:59:33 ✓ reports correct status when enqueuing repeat jobs (39 ms) 22:59:33 ✓ throws an error in case of duplicate job IDs (27 ms) 22:59:33 ✓ returns not-found status for non-existing jobs (4 ms) 22:59:33 ✓ cancels jobs in queue (38 ms) 22:59:33 ✓ cancels jobs in-progress (39 ms) 22:59:33 ✓ returns job result if successful (33 ms) 22:59:33 ✓ returns job error if failed (38 ms) 22:59:33 ✓ correctly returns job status for concurrent writes (44 ms) 22:59:33 Consumer API 22:59:33 ✓ returns undefined if no jobs are available (1 ms) 22:59:33 ✓ returns jobs in priority order (2 ms) 22:59:33 ✓ returns undefined if no jobs are available for the given allowList (9 ms) 22:59:33 ✓ returns a job if it is in the allowList (4 ms) 22:59:33 ✓ returns the most important job if it is in the allowList (3 ms) 22:59:33 ✓ returns any job if filter is empty (3 ms) 22:59:33 ✓ returns a new job when reporting job success (3 ms) 22:59:33 ✓ returns a new job when reporting permanent error (2 ms) 22:59:33 ✓ returns a new job when reporting retry-able error (3 ms) 22:59:33 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 22:59:33 ✓ returns a new job if job is already in progress elsewhere (1072 ms) 22:59:33 ✓ avoids sending the same job to a new agent after a restart (1010 ms) 22:59:33 ✓ avoids sending a completed job to a new agent after a restart (1009 ms) 22:59:33 ✓ tracks job result if in progress (5 ms) 22:59:33 ✓ tracks job result even if job is in queue (4 ms) 22:59:33 ✓ ignores reported job error if unknown job (2 ms) 22:59:33 ✓ ignores job result if unknown job (2 ms) 22:59:33 ✓ returns undefined if no jobs are available (4 ms) 22:59:33 ✓ returns jobs in priority order (48 ms) 22:59:33 ✓ returns undefined if no jobs are available for the given allowList (29 ms) 22:59:33 ✓ returns a job if it is in the allowList (62 ms) 22:59:33 ✓ returns the most important job if it is in the allowList (65 ms) 22:59:33 ✓ returns any job if filter is empty (67 ms) 22:59:33 ✓ returns a new job when reporting job success (37 ms) 22:59:33 ✓ returns a new job when reporting permanent error (45 ms) 22:59:33 ✓ returns a new job when reporting retry-able error (39 ms) 22:59:33 ✓ returns a new job when reporting progress if current one is cancelled (42 ms) 22:59:33 ✓ returns a new job if job is already in progress elsewhere (1141 ms) 22:59:33 ✓ avoids sending the same job to a new agent after a restart (1054 ms) 22:59:33 ✓ avoids sending a completed job to a new agent after a restart (1085 ms) 22:59:33 ✓ tracks job result if in progress (81 ms) 22:59:33 ✓ tracks job result even if job is in queue (84 ms) 22:59:33 ✓ ignores reported job error if unknown job (5 ms) 22:59:33 ✓ ignores job result if unknown job (5 ms) 22:59:33 Timeouts 22:59:33 ✓ tracks in progress jobs (3 ms) 22:59:33 ✓ re-enqueues jobs that time out (110 ms) 22:59:33 ✓ cancel stale jobs that time out (234 ms) 22:59:33 ✓ keeps the jobs in progress while it is alive (207 ms) 22:59:33 ✓ tracks in progress jobs (29 ms) 22:59:33 ✓ re-enqueues jobs that time out (138 ms) 22:59:33 ✓ cancel stale jobs that time out (288 ms) 22:59:33 ✓ keeps the jobs in progress while it is alive (244 ms) 22:59:33 Retries 22:59:33 ✓ retries jobs (5 ms) 22:59:33 ✓ retries up to a maximum number of times (6 ms) 22:59:33 ✓ passing retry=false does not retry (3 ms) 22:59:33 ✓ does not retry if job is stale (31 ms) 22:59:33 ✓ retries jobs (31 ms) 22:59:33 ✓ retries up to a maximum number of times (48 ms) 22:59:33 ✓ passing retry=false does not retry (30 ms) 22:59:33 ✓ does not retry if job is stale (74 ms) 22:59:33 Database management 22:59:33 ✓ re-enqueues proof requests on start (4 ms) 22:59:33 ✓ restores proof results on start (3 ms) 22:59:33 ✓ only re-enqueues unfinished jobs (3 ms) 22:59:33 ✓ saves job when enqueued (7 ms) 22:59:33 ✓ does not retain job if database fails to save (5 ms) 22:59:33 ✓ saves job result (4 ms) 22:59:33 ✓ saves result even if database fails to save (4 ms) 22:59:33 ✓ saves job error (3 ms) 22:59:33 ✓ saves job error even if database fails to save (3 ms) 22:59:33 ✓ does not save job result if job is unknown (2 ms) 22:59:33 ✓ does not save job error if job is unknown (2 ms) 22:59:33 ✓ cleans up old jobs periodically (266 ms) 22:59:33 ✓ re-enqueues proof requests on start (58 ms) 22:59:33 ✓ restores proof results on start (82 ms) 22:59:33 ✓ only re-enqueues unfinished jobs (73 ms) 22:59:33 ✓ saves job when enqueued (28 ms) 22:59:33 ✓ does not retain job if database fails to save (7 ms) 22:59:33 ✓ saves job result (49 ms) 22:59:33 ✓ saves result even if database fails to save (33 ms) 22:59:33 ✓ saves job error (59 ms) 22:59:33 ✓ saves job error even if database fails to save (32 ms) 22:59:33 ✓ does not save job result if job is unknown (7 ms) 22:59:33 ✓ does not save job error if job is unknown (6 ms) 22:59:33 ✓ cleans up old jobs periodically (432 ms) 22:59:33 22:59:33 Test Suites: 1 passed, 1 total 22:59:33 Tests: 96 passed, 96 total 22:59:33 Snapshots: 0 total 22:59:33 Time: 12.917 s 22:59:33 Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i. 22:59:33 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?