Parent Log:
http://ci.aztec-labs.com/6093ea1a8fb19ac6
Command: 1f25be2e8c7902e7 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/17ea18ade041a1bb437806e047c30b29da124094
Env: REF_NAME=gh-readonly-queue/next/pr-15007-4c695be4bff40a4a2ca2118c7e14c83a104e93aa CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Wed Jun 18 17:25:54 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15007_amd64_x4-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
17:25:59 [17:25:59.388]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.393]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31a33aa34585010c epochNumber=42
{"provingJobId":"1:BASE_PARITY:31a33aa34585010c"}
17:25:59 [17:25:59.396]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:959d68d7f5269d79 epochNumber=1
{"provingJobId":"1:BASE_PARITY:959d68d7f5269d79"}
17:25:59 [17:25:59.440]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.441]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:683672551d1000b3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:683672551d1000b3"}
17:25:59 [17:25:59.442]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:389a06a9a69804b3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:389a06a9a69804b3"}
17:25:59 [17:25:59.444]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.446]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:accda80ebfd5bd6e epochNumber=1
{"provingJobId":"1:BASE_PARITY:accda80ebfd5bd6e"}
17:25:59 [17:25:59.447]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:accda80ebfd5bd6e epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:accda80ebfd5bd6e"}
17:25:59 [17:25:59.449]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.450]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e019a3e0e17dfea epochNumber=1
{"provingJobId":"1:BASE_PARITY:9e019a3e0e17dfea"}
17:25:59 [17:25:59.452]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9e019a3e0e17dfea epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9e019a3e0e17dfea"}
17:25:59 [17:25:59.453]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9e019a3e0e17dfea type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9e019a3e0e17dfea"}
17:25:59 [17:25:59.454]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:9e019a3e0e17dfea epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:9e019a3e0e17dfea"}
17:25:59 [17:25:59.455]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.456]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e5883594eaa69c69 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e5883594eaa69c69"}
17:25:59 [17:25:59.481]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.482]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.483]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2e39b4c4b87ba3b1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2e39b4c4b87ba3b1"}
17:25:59 [17:25:59.483]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2e39b4c4b87ba3b1
{"provingJobId":"1:BASE_PARITY:2e39b4c4b87ba3b1"}
17:25:59 [17:25:59.484]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2e39b4c4b87ba3b1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2e39b4c4b87ba3b1"}
17:25:59 [17:25:59.484]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2e39b4c4b87ba3b1 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:2e39b4c4b87ba3b1"}
17:25:59 [17:25:59.486]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.486]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:713d5148c0b24a97 epochNumber=1
{"provingJobId":"1:BASE_PARITY:713d5148c0b24a97"}
17:25:59 [17:25:59.487]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:713d5148c0b24a97
{"provingJobId":"1:BASE_PARITY:713d5148c0b24a97"}
17:25:59 [17:25:59.487]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:713d5148c0b24a97 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:713d5148c0b24a97"}
17:25:59 [17:25:59.489]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.490]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ff320bc012c97844 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ff320bc012c97844"}
17:25:59 [17:25:59.490]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ff320bc012c97844 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ff320bc012c97844"}
17:25:59 [17:25:59.490]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ff320bc012c97844 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ff320bc012c97844"}
17:25:59 [17:25:59.492]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.493]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ae1273071e483d0e epochNumber=1
{"provingJobId":"1:BASE_PARITY:ae1273071e483d0e"}
17:25:59 [17:25:59.493]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ae1273071e483d0e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ae1273071e483d0e"}
17:25:59 [17:25:59.493]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ae1273071e483d0e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ae1273071e483d0e"}
17:25:59 [17:25:59.494]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.495]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3af510b5e296068 epochNumber=0
{"provingJobId":"1:BASE_PARITY:d3af510b5e296068"}
17:25:59 [17:25:59.495]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d3af510b5e296068 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d3af510b5e296068"}
17:25:59 [17:25:59.495]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1614ec90e3b15c0b epochNumber=0
{"provingJobId":"1:BASE_PARITY:1614ec90e3b15c0b"}
17:25:59 [17:25:59.496]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d3af510b5e296068 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d3af510b5e296068"}
17:25:59 [17:25:59.496]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:410959f3f6f0147e epochNumber=0
{"provingJobId":"1:BASE_PARITY:410959f3f6f0147e"}
17:25:59 [17:25:59.505]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.507]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.508]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ff40f9bc95a07baa epochNumber=2
{"provingJobId":"1:BASE_PARITY:ff40f9bc95a07baa"}
17:25:59 [17:25:59.508]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fbf937a5adfbabfe epochNumber=1
{"provingJobId":"1:BASE_PARITY:fbf937a5adfbabfe"}
17:25:59 [17:25:59.510]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.511]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa93ee4bea3fa2c7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa93ee4bea3fa2c7"}
17:25:59 [17:25:59.513]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.514]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0e12b1aba2d44ef4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0e12b1aba2d44ef4"}
17:25:59 [17:25:59.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8005ecc3ce16aa9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8005ecc3ce16aa9"}
17:25:59 [17:25:59.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c42dff95777f03b0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:c42dff95777f03b0"}
17:25:59 [17:25:59.516]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9f86025facdf58a2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9f86025facdf58a2"}
17:25:59 [17:25:59.517]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.518]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0ecf12f159176ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0ecf12f159176ab"}
17:25:59 [17:25:59.519]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:98555324b37feefc epochNumber=1
{"provingJobId":"1:BASE_PARITY:98555324b37feefc"}
17:25:59 [17:25:59.519]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d21c128cc512d9d epochNumber=2
{"provingJobId":"1:BASE_PARITY:3d21c128cc512d9d"}
17:25:59 [17:25:59.520]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ff93201cfcda1436 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ff93201cfcda1436"}
17:25:59 [17:25:59.521]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.522]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:310c58d4f9f42f26 epochNumber=1
{"provingJobId":"1:BASE_PARITY:310c58d4f9f42f26"}
17:25:59 [17:25:59.522]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c7d2c002263c970a epochNumber=1
{"provingJobId":"1:BASE_PARITY:c7d2c002263c970a"}
17:25:59 [17:25:59.523]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:32c0d7d6d07d77ff epochNumber=2
{"provingJobId":"1:BASE_PARITY:32c0d7d6d07d77ff"}
17:25:59 [17:25:59.523]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b1089b1ddbfee60a epochNumber=1
{"provingJobId":"1:BASE_PARITY:b1089b1ddbfee60a"}
17:25:59 [17:25:59.525]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.526]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:545841f310300fa9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:545841f310300fa9"}
17:25:59 [17:25:59.527]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:143f0311a064d68d epochNumber=1
{"provingJobId":"1:BASE_PARITY:143f0311a064d68d"}
17:25:59 [17:25:59.527]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:545841f310300fa9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:545841f310300fa9"}
17:25:59 [17:25:59.528]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.529]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5eef96772914c03f epochNumber=1
{"provingJobId":"1:BASE_PARITY:5eef96772914c03f"}
17:25:59 [17:25:59.529]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:75e8a85eb26e5ec2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:75e8a85eb26e5ec2"}
17:25:59 [17:25:59.530]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5eef96772914c03f type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:5eef96772914c03f"}
17:25:59 [17:25:59.531]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.531]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ab23aee585ccf91f epochNumber=1
{"provingJobId":"1:BASE_PARITY:ab23aee585ccf91f"}
17:25:59 [17:25:59.532]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8c39d6a0b0a6dc32 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8c39d6a0b0a6dc32"}
17:25:59 [17:25:59.532]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ab23aee585ccf91f type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:ab23aee585ccf91f"}
17:25:59 [17:25:59.533]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.533]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d92c2ecfc641e0ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:d92c2ecfc641e0ab"}
17:25:59 [17:25:59.534]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d92c2ecfc641e0ab
{"provingJobId":"1:BASE_PARITY:d92c2ecfc641e0ab"}
17:25:59 [17:25:59.534]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d92c2ecfc641e0ab type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:d92c2ecfc641e0ab"}
17:25:59 [17:25:59.534]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b3e738c9261f8bed epochNumber=1
{"provingJobId":"1:BASE_PARITY:b3e738c9261f8bed"}
17:25:59 [17:25:59.535]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d92c2ecfc641e0ab has already been completed
{"provingJobId":"1:BASE_PARITY:d92c2ecfc641e0ab"}
17:25:59 [17:25:59.536]
INFO:
prover-client:proving-broker Proving Broker started
17:25:59 [17:25:59.536]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d4a856ae26933a58 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d4a856ae26933a58"}
17:25:59 [17:25:59.536]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4c2333d7ae69fae epochNumber=2
{"provingJobId":"1:BASE_PARITY:a4c2333d7ae69fae"}
17:26:00 [17:26:00.590]
INFO:
prover-client:proving-broker Proving Broker started
17:26:00 [17:26:00.590]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d4a856ae26933a58 settled=false
{"provingJobId":"1:BASE_PARITY:d4a856ae26933a58","status":"pending"}
17:26:00 [17:26:00.591]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a4c2333d7ae69fae settled=false
{"provingJobId":"1:BASE_PARITY:a4c2333d7ae69fae","status":"pending"}
17:26:00 [17:26:00.591]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d4a856ae26933a58 type=BASE_PARITY startedAt=1750267559537 older agent has taken job
{"provingJobId":"1:BASE_PARITY:d4a856ae26933a58"}
17:26:00 [17:26:00.592]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d4a856ae26933a58 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:d4a856ae26933a58"}
17:26:00 [17:26:00.593]
INFO:
prover-client:proving-broker Proving Broker started
17:26:00 [17:26:00.593]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cbf75efa21e3a42e epochNumber=1
{"provingJobId":"1:BASE_PARITY:cbf75efa21e3a42e"}
17:26:00 [17:26:00.594]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:270769a987902690 epochNumber=2
{"provingJobId":"1:BASE_PARITY:270769a987902690"}
17:26:01 [17:26:01.594]
INFO:
prover-client:proving-broker Proving Broker started
17:26:01 [17:26:01.594]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cbf75efa21e3a42e settled=false
{"provingJobId":"1:BASE_PARITY:cbf75efa21e3a42e","status":"pending"}
17:26:01 [17:26:01.595]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:270769a987902690 settled=false
{"provingJobId":"1:BASE_PARITY:270769a987902690","status":"pending"}
17:26:01 [17:26:01.595]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cbf75efa21e3a42e type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:cbf75efa21e3a42e"}
17:26:01 [17:26:01.597]
INFO:
prover-client:proving-broker Proving Broker started
17:26:01 [17:26:01.598]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7d9fd79b1bf8d671 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7d9fd79b1bf8d671"}
17:26:01 [17:26:01.598]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac0f552250e077f2 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ac0f552250e077f2"}
17:26:02 [17:26:02.601]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.601]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7d9fd79b1bf8d671 settled=false
{"provingJobId":"1:BASE_PARITY:7d9fd79b1bf8d671","status":"pending"}
17:26:02 [17:26:02.602]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ac0f552250e077f2 settled=false
{"provingJobId":"1:BASE_PARITY:ac0f552250e077f2","status":"pending"}
17:26:02 [17:26:02.602]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7d9fd79b1bf8d671 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7d9fd79b1bf8d671"}
17:26:02 [17:26:02.603]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7d9fd79b1bf8d671 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7d9fd79b1bf8d671"}
17:26:02 [17:26:02.604]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.605]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d4627c5a23ea5c99 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d4627c5a23ea5c99"}
17:26:02 [17:26:02.605]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9be4db280d4c7040 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9be4db280d4c7040"}
17:26:02 [17:26:02.605]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d4627c5a23ea5c99 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d4627c5a23ea5c99"}
17:26:02 [17:26:02.606]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9be4db280d4c7040 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9be4db280d4c7040"}
17:26:02 [17:26:02.610]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.612]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:31cf0e80082f8f42 epochNumber=1
{"provingJobId":"1:BASE_PARITY:31cf0e80082f8f42"}
17:26:02 [17:26:02.612]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de2b36be558e32e6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:de2b36be558e32e6"}
17:26:02 [17:26:02.612]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:31cf0e80082f8f42 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:31cf0e80082f8f42"}
17:26:02 [17:26:02.613]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:31cf0e80082f8f42 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:31cf0e80082f8f42"}
17:26:02 [17:26:02.613]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:de2b36be558e32e6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:de2b36be558e32e6"}
17:26:02 [17:26:02.615]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.616]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ed44636020877a56 err=test error
{"provingJoId":"1:BASE_PARITY:ed44636020877a56"}
17:26:02 [17:26:02.617]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.618]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c5fffc0c28d80869 not found
{"provingJobId":"1:BASE_PARITY:c5fffc0c28d80869"}
17:26:02 [17:26:02.619]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.620]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2508c7cdb3d75705 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2508c7cdb3d75705"}
17:26:02 [17:26:02.622]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.622]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81a7ae3a549148a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:81a7ae3a549148a0"}
17:26:02 [17:26:02.747]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:81a7ae3a549148a0 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:81a7ae3a549148a0"}
17:26:02 [17:26:02.749]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.749]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc01fe7fdb088826 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc01fe7fdb088826"}
17:26:02 [17:26:02.851]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dc01fe7fdb088826 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:dc01fe7fdb088826"}
17:26:02 [17:26:02.857]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0beed8f139f75aaf epochNumber=10
{"provingJobId":"1:BASE_PARITY:0beed8f139f75aaf"}
17:26:02 [17:26:02.877]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:02 [17:26:02.984]
INFO:
prover-client:proving-broker Proving Broker started
17:26:02 [17:26:02.984]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb944b4c0569d29a epochNumber=1
{"provingJobId":"1:BASE_PARITY:bb944b4c0569d29a"}
17:26:03 [17:26:03.135]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bb944b4c0569d29a timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:bb944b4c0569d29a"}
17:26:03 [17:26:03.188]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.189]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:37f1c5aa74173850 epochNumber=1
{"provingJobId":"1:BASE_PARITY:37f1c5aa74173850"}
17:26:03 [17:26:03.190]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:37f1c5aa74173850 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:37f1c5aa74173850"}
17:26:03 [17:26:03.192]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.192]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c05e99f0429a6dd7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c05e99f0429a6dd7"}
17:26:03 [17:26:03.193]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c05e99f0429a6dd7 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c05e99f0429a6dd7"}
17:26:03 [17:26:03.194]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c05e99f0429a6dd7 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:c05e99f0429a6dd7"}
17:26:03 [17:26:03.195]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.196]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a8896908747d1135 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a8896908747d1135"}
17:26:03 [17:26:03.196]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a8896908747d1135 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a8896908747d1135"}
17:26:03 [17:26:03.197]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.198]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bff277780f21cb92 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bff277780f21cb92"}
17:26:03 [17:26:03.198]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bff277780f21cb92 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:bff277780f21cb92"}
17:26:03 [17:26:03.199]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f19409097aeeee5f epochNumber=3
{"provingJobId":"1:BASE_PARITY:f19409097aeeee5f"}
17:26:03 [17:26:03.223]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:03 [17:26:03.227]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:bff277780f21cb92 err=test error
{"provingJoId":"1:BASE_PARITY:bff277780f21cb92"}
17:26:03 [17:26:03.232]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.233]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3cbeab5ddc904b20 settled=false
{"provingJobId":"1:BASE_PARITY:3cbeab5ddc904b20","status":"pending"}
17:26:03 [17:26:03.233]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e8772de78d237710 settled=false
{"provingJobId":"1:BASE_PARITY:e8772de78d237710","status":"pending"}
17:26:03 [17:26:03.238]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.238]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:afc841b668f375b2 settled=true
{"provingJobId":"1:BASE_PARITY:afc841b668f375b2","status":"fulfilled"}
17:26:03 [17:26:03.238]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8e86cb61c40b052b settled=true
{"provingJobId":"2:BASE_PARITY:8e86cb61c40b052b","status":"fulfilled"}
17:26:03 [17:26:03.241]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.241]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ca592e33ff1720c5 settled=true
{"provingJobId":"1:BASE_PARITY:ca592e33ff1720c5","status":"fulfilled"}
17:26:03 [17:26:03.241]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3117d34cf2c4bc7f settled=false
{"provingJobId":"1:BASE_PARITY:3117d34cf2c4bc7f","status":"pending"}
17:26:03 [17:26:03.243]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.245]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:502ec5961fdbb563 epochNumber=1
{"provingJobId":"1:BASE_PARITY:502ec5961fdbb563"}
17:26:03 [17:26:03.246]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.247]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aacbc4d2048b0f2c epochNumber=1
{"provingJobId":"1:BASE_PARITY:aacbc4d2048b0f2c"}
17:26:03 [17:26:03.249]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:aacbc4d2048b0f2c: Error: db error: Error: db error
17:26:03 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:aacbc4d2048b0f2c"}
17:26:03 [17:26:03.252]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.253]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b64187884e7510bf epochNumber=1
{"provingJobId":"1:BASE_PARITY:b64187884e7510bf"}
17:26:03 [17:26:03.253]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b64187884e7510bf type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:b64187884e7510bf"}
17:26:03 [17:26:03.253]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b64187884e7510bf type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b64187884e7510bf"}
17:26:03 [17:26:03.254]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.255]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e61c54a14dbec3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:6e61c54a14dbec3e"}
17:26:03 [17:26:03.255]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6e61c54a14dbec3e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:6e61c54a14dbec3e"}
17:26:03 [17:26:03.256]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6e61c54a14dbec3e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6e61c54a14dbec3e"}
17:26:03 [17:26:03.256]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:6e61c54a14dbec3e: Error: db error
17:26:03 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:6e61c54a14dbec3e"}
17:26:03 [17:26:03.258]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.258]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:76d73fd77dbba086 epochNumber=1
{"provingJobId":"1:BASE_PARITY:76d73fd77dbba086"}
17:26:03 [17:26:03.259]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:76d73fd77dbba086 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:76d73fd77dbba086"}
17:26:03 [17:26:03.259]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:76d73fd77dbba086 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:76d73fd77dbba086"}
17:26:03 [17:26:03.260]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.261]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ca74c32ab126e2d epochNumber=1
{"provingJobId":"1:BASE_PARITY:7ca74c32ab126e2d"}
17:26:03 [17:26:03.261]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7ca74c32ab126e2d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7ca74c32ab126e2d"}
17:26:03 [17:26:03.261]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7ca74c32ab126e2d type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7ca74c32ab126e2d"}
17:26:03 [17:26:03.262]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:7ca74c32ab126e2d jobErr=test error: Error: db error
17:26:03 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:7ca74c32ab126e2d"}
17:26:03 [17:26:03.263]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.263]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1790aa81ce48f3d9 not found
{"provingJobId":"1:BASE_PARITY:1790aa81ce48f3d9"}
17:26:03 [17:26:03.264]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.265]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0299aaf296467649 err=test error
{"provingJoId":"1:BASE_PARITY:0299aaf296467649"}
17:26:03 [17:26:03.266]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.276]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d96a4329680c40db epochNumber=1
{"provingJobId":"1:BASE_PARITY:d96a4329680c40db"}
17:26:03 [17:26:03.277]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d96a4329680c40db type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d96a4329680c40db"}
17:26:03 [17:26:03.277]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d96a4329680c40db type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d96a4329680c40db"}
17:26:03 [17:26:03.288]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:811aa6fda4e9c4d9 epochNumber=2
{"provingJobId":"2:BASE_PARITY:811aa6fda4e9c4d9"}
17:26:03 [17:26:03.289]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:811aa6fda4e9c4d9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:811aa6fda4e9c4d9"}
17:26:03 [17:26:03.290]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:811aa6fda4e9c4d9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:811aa6fda4e9c4d9"}
17:26:03 [17:26:03.351]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:19deed463bebc23c epochNumber=3
{"provingJobId":"3:BASE_PARITY:19deed463bebc23c"}
17:26:03 [17:26:03.369]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:03 [17:26:03.413]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:3ae67ab334ad2d0f epochNumber=4
{"provingJobId":"4:BASE_PARITY:3ae67ab334ad2d0f"}
17:26:03 [17:26:03.421]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:03 [17:26:03.477]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:a70dfc50771a2ef5 epochNumber=5
{"provingJobId":"5:BASE_PARITY:a70dfc50771a2ef5"}
17:26:03 [17:26:03.498]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:03 [17:26:03.532]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.533]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b00accec8c335660 epochNumber=42
{"provingJobId":"1:BASE_PARITY:b00accec8c335660"}
17:26:03 [17:26:03.534]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testkXtnVs/42 with map size 1073741824
17:26:03 [17:26:03.537]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.566]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:7e2ae57be65af1d4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e2ae57be65af1d4"}
17:26:03 [17:26:03.572]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.572]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:15fe901ff86513f5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:15fe901ff86513f5"}
17:26:03 [17:26:03.573]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYzIM1b/1 with map size 1073741824
17:26:03 [17:26:03.575]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.595]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c73967c3edff759 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c73967c3edff759"}
17:26:03 [17:26:03.610]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.611]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:849d3fb9e7e0073f epochNumber=1
{"provingJobId":"1:BASE_PARITY:849d3fb9e7e0073f"}
17:26:03 [17:26:03.612]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD9w0Hm/1 with map size 1073741824
17:26:03 [17:26:03.621]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.644]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:849d3fb9e7e0073f epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:849d3fb9e7e0073f"}
17:26:03 [17:26:03.648]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0112b84400c4aaf8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0112b84400c4aaf8"}
17:26:03 [17:26:03.650]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testA8nObQ/1 with map size 1073741824
17:26:03 [17:26:03.654]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.674]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0112b84400c4aaf8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0112b84400c4aaf8"}
17:26:03 [17:26:03.675]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0112b84400c4aaf8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0112b84400c4aaf8"}
17:26:03 [17:26:03.681]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0112b84400c4aaf8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0112b84400c4aaf8"}
17:26:03 [17:26:03.685]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.685]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3430e7f3496e42a4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3430e7f3496e42a4"}
17:26:03 [17:26:03.686]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoVL18i/1 with map size 1073741824
17:26:03 [17:26:03.688]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.722]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.731]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.732]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7fc83cb838ba8105 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7fc83cb838ba8105"}
17:26:03 [17:26:03.733]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDQFQRR/1 with map size 1073741824
17:26:03 [17:26:03.736]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.756]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:7fc83cb838ba8105
{"provingJobId":"1:BASE_PARITY:7fc83cb838ba8105"}
17:26:03 [17:26:03.756]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7fc83cb838ba8105 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7fc83cb838ba8105"}
17:26:03 [17:26:03.757]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7fc83cb838ba8105 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:7fc83cb838ba8105"}
17:26:03 [17:26:03.767]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.767]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8d1c661f070b87b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8d1c661f070b87b0"}
17:26:03 [17:26:03.768]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCCy0SH/1 with map size 1073741824
17:26:03 [17:26:03.771]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.791]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:8d1c661f070b87b0
{"provingJobId":"1:BASE_PARITY:8d1c661f070b87b0"}
17:26:03 [17:26:03.791]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8d1c661f070b87b0 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:8d1c661f070b87b0"}
17:26:03 [17:26:03.802]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.803]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:948701c07a63eb82 epochNumber=1
{"provingJobId":"1:BASE_PARITY:948701c07a63eb82"}
17:26:03 [17:26:03.807]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRZIca1/1 with map size 1073741824
17:26:03 [17:26:03.816]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.837]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:948701c07a63eb82 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:948701c07a63eb82"}
17:26:03 [17:26:03.837]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:948701c07a63eb82 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:948701c07a63eb82"}
17:26:03 [17:26:03.845]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.846]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a663e642a91380d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a663e642a91380d3"}
17:26:03 [17:26:03.847]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSXfADv/1 with map size 1073741824
17:26:03 [17:26:03.851]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.868]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a663e642a91380d3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a663e642a91380d3"}
17:26:03 [17:26:03.868]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a663e642a91380d3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a663e642a91380d3"}
17:26:03 [17:26:03.876]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.877]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2a111e8ad9043ae epochNumber=0
{"provingJobId":"1:BASE_PARITY:f2a111e8ad9043ae"}
17:26:03 [17:26:03.879]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-test5pgcbo/0 with map size 1073741824
17:26:03 [17:26:03.880]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.900]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f2a111e8ad9043ae epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f2a111e8ad9043ae"}
17:26:03 [17:26:03.901]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1897b8fe148e5fb epochNumber=0
{"provingJobId":"1:BASE_PARITY:c1897b8fe148e5fb"}
17:26:03 [17:26:03.901]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f2a111e8ad9043ae epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f2a111e8ad9043ae"}
17:26:03 [17:26:03.902]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:37fad7ca041c4d50 epochNumber=0
{"provingJobId":"1:BASE_PARITY:37fad7ca041c4d50"}
17:26:03 [17:26:03.920]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.927]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.928]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:686783379940f6d7 epochNumber=2
{"provingJobId":"1:BASE_PARITY:686783379940f6d7"}
17:26:03 [17:26:03.929]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testG8vckF/2 with map size 1073741824
17:26:03 [17:26:03.932]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.951]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:abb39fd1da9e92be epochNumber=1
{"provingJobId":"1:BASE_PARITY:abb39fd1da9e92be"}
17:26:03 [17:26:03.952]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testG8vckF/1 with map size 1073741824
17:26:03 [17:26:03.954]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:03 [17:26:03.982]
INFO:
prover-client:proving-broker Proving Broker started
17:26:03 [17:26:03.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f74b53c8e1b4e218 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f74b53c8e1b4e218"}
17:26:03 [17:26:03.983]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testljP7LG/1 with map size 1073741824
17:26:03 [17:26:03.987]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.009]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.009]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c41824f89f566275 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c41824f89f566275"}
17:26:04 [17:26:04.011]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3NZkui/1 with map size 1073741824
17:26:04 [17:26:04.021]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.041]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:620676c9839609da epochNumber=1
{"provingJobId":"1:BASE_PARITY:620676c9839609da"}
17:26:04 [17:26:04.047]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:763680a12bc5e6ab epochNumber=2
{"provingJobId":"1:BASE_PARITY:763680a12bc5e6ab"}
17:26:04 [17:26:04.048]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3NZkui/2 with map size 1073741824
17:26:04 [17:26:04.051]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.070]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7dd813df07018cbb epochNumber=1
{"provingJobId":"1:BASE_PARITY:7dd813df07018cbb"}
17:26:04 [17:26:04.079]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.080]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7eabe3a4c0747953 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7eabe3a4c0747953"}
17:26:04 [17:26:04.081]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyw2zD4/1 with map size 1073741824
17:26:04 [17:26:04.084]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.100]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c167f4bf6ddd63f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c167f4bf6ddd63f8"}
17:26:04 [17:26:04.110]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:666c3a4bf79071a4 epochNumber=2
{"provingJobId":"1:BASE_PARITY:666c3a4bf79071a4"}
17:26:04 [17:26:04.112]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testyw2zD4/2 with map size 1073741824
17:26:04 [17:26:04.115]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.132]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e084b57f79ad89c1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e084b57f79ad89c1"}
17:26:04 [17:26:04.141]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.142]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe34d84aaf81f36c epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe34d84aaf81f36c"}
17:26:04 [17:26:04.142]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjSiIzc/1 with map size 1073741824
17:26:04 [17:26:04.144]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.165]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:df8fa1d829d9ee2e epochNumber=1
{"provingJobId":"1:BASE_PARITY:df8fa1d829d9ee2e"}
17:26:04 [17:26:04.175]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:62b857e83171a25e epochNumber=2
{"provingJobId":"1:BASE_PARITY:62b857e83171a25e"}
17:26:04 [17:26:04.176]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testjSiIzc/2 with map size 1073741824
17:26:04 [17:26:04.182]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.202]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b5f26c84f5676ae9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b5f26c84f5676ae9"}
17:26:04 [17:26:04.213]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.216]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c259a0c601e8d1cd epochNumber=1
{"provingJobId":"1:BASE_PARITY:c259a0c601e8d1cd"}
17:26:04 [17:26:04.218]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDpXGvI/1 with map size 1073741824
17:26:04 [17:26:04.220]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.243]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f334358d0fe4e518 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f334358d0fe4e518"}
17:26:04 [17:26:04.250]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c259a0c601e8d1cd type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c259a0c601e8d1cd"}
17:26:04 [17:26:04.262]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.263]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1a266cb3547f7c40 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1a266cb3547f7c40"}
17:26:04 [17:26:04.264]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaS7RGb/1 with map size 1073741824
17:26:04 [17:26:04.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.283]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:719683ec2ba4fb04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:719683ec2ba4fb04"}
17:26:04 [17:26:04.290]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1a266cb3547f7c40 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:1a266cb3547f7c40"}
17:26:04 [17:26:04.300]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.301]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0bcb7228a7678014 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0bcb7228a7678014"}
17:26:04 [17:26:04.302]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNePNKj/1 with map size 1073741824
17:26:04 [17:26:04.303]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.324]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e46c1e20610a5059 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e46c1e20610a5059"}
17:26:04 [17:26:04.332]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0bcb7228a7678014 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:0bcb7228a7678014"}
17:26:04 [17:26:04.337]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.338]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:88f02d18ae5c5b04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:88f02d18ae5c5b04"}
17:26:04 [17:26:04.339]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhkTtPa/1 with map size 1073741824
17:26:04 [17:26:04.341]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.360]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:88f02d18ae5c5b04
{"provingJobId":"1:BASE_PARITY:88f02d18ae5c5b04"}
17:26:04 [17:26:04.361]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:88f02d18ae5c5b04 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:88f02d18ae5c5b04"}
17:26:04 [17:26:04.368]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:10e4de1b0e31c0cd epochNumber=1
{"provingJobId":"1:BASE_PARITY:10e4de1b0e31c0cd"}
17:26:04 [17:26:04.376]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:88f02d18ae5c5b04 has already been completed
{"provingJobId":"1:BASE_PARITY:88f02d18ae5c5b04"}
17:26:04 [17:26:04.382]
INFO:
prover-client:proving-broker Proving Broker started
17:26:04 [17:26:04.383]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:068b2d1c1b0fae5a epochNumber=1
{"provingJobId":"1:BASE_PARITY:068b2d1c1b0fae5a"}
17:26:04 [17:26:04.385]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9jKXQx/1 with map size 1073741824
17:26:04 [17:26:04.388]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:04 [17:26:04.409]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:df3cb6f4cb9d1ec3 epochNumber=2
{"provingJobId":"1:BASE_PARITY:df3cb6f4cb9d1ec3"}
17:26:04 [17:26:04.410]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test9jKXQx/2 with map size 1073741824
17:26:04 [17:26:04.412]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:05 [17:26:05.485]
INFO:
prover-client:proving-broker Proving Broker started
17:26:05 [17:26:05.491]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:068b2d1c1b0fae5a settled=false
{"provingJobId":"1:BASE_PARITY:068b2d1c1b0fae5a","status":"pending"}
17:26:05 [17:26:05.493]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:df3cb6f4cb9d1ec3 settled=false
{"provingJobId":"1:BASE_PARITY:df3cb6f4cb9d1ec3","status":"pending"}
17:26:05 [17:26:05.494]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:068b2d1c1b0fae5a type=BASE_PARITY startedAt=1750267564433 older agent has taken job
{"provingJobId":"1:BASE_PARITY:068b2d1c1b0fae5a"}
17:26:05 [17:26:05.494]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:068b2d1c1b0fae5a type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:068b2d1c1b0fae5a"}
17:26:05 [17:26:05.497]
INFO:
prover-client:proving-broker Proving Broker started
17:26:05 [17:26:05.498]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a5b45f44e25e0bf epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a5b45f44e25e0bf"}
17:26:05 [17:26:05.499]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBa3cjN/1 with map size 1073741824
17:26:05 [17:26:05.502]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:05 [17:26:05.532]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36eba84ab58647c9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:36eba84ab58647c9"}
17:26:05 [17:26:05.533]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBa3cjN/2 with map size 1073741824
17:26:05 [17:26:05.539]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:06 [17:26:06.557]
INFO:
prover-client:proving-broker Proving Broker started
17:26:06 [17:26:06.560]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0a5b45f44e25e0bf settled=false
{"provingJobId":"1:BASE_PARITY:0a5b45f44e25e0bf","status":"pending"}
17:26:06 [17:26:06.562]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:36eba84ab58647c9 settled=false
{"provingJobId":"1:BASE_PARITY:36eba84ab58647c9","status":"pending"}
17:26:06 [17:26:06.563]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:0a5b45f44e25e0bf type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:0a5b45f44e25e0bf"}
17:26:06 [17:26:06.567]
INFO:
prover-client:proving-broker Proving Broker started
17:26:06 [17:26:06.568]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:517e10cc7a6e57fa epochNumber=1
{"provingJobId":"1:BASE_PARITY:517e10cc7a6e57fa"}
17:26:06 [17:26:06.569]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVbeqQl/1 with map size 1073741824
17:26:06 [17:26:06.570]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:06 [17:26:06.587]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:884f4517c3a0a12c epochNumber=2
{"provingJobId":"1:BASE_PARITY:884f4517c3a0a12c"}
17:26:06 [17:26:06.588]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVbeqQl/2 with map size 1073741824
17:26:06 [17:26:06.592]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.619]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.622]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:517e10cc7a6e57fa settled=false
{"provingJobId":"1:BASE_PARITY:517e10cc7a6e57fa","status":"pending"}
17:26:07 [17:26:07.625]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:884f4517c3a0a12c settled=false
{"provingJobId":"1:BASE_PARITY:884f4517c3a0a12c","status":"pending"}
17:26:07 [17:26:07.627]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:517e10cc7a6e57fa type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:517e10cc7a6e57fa"}
17:26:07 [17:26:07.627]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:517e10cc7a6e57fa type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:517e10cc7a6e57fa"}
17:26:07 [17:26:07.638]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.640]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b2adce1b115340a epochNumber=1
{"provingJobId":"1:BASE_PARITY:9b2adce1b115340a"}
17:26:07 [17:26:07.640]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZsTi3c/1 with map size 1073741824
17:26:07 [17:26:07.644]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.663]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5aade11f30ebd811 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5aade11f30ebd811"}
17:26:07 [17:26:07.665]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testZsTi3c/2 with map size 1073741824
17:26:07 [17:26:07.667]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.687]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9b2adce1b115340a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9b2adce1b115340a"}
17:26:07 [17:26:07.696]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5aade11f30ebd811 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:5aade11f30ebd811"}
17:26:07 [17:26:07.710]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.712]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dbf833e0903dd75d epochNumber=1
{"provingJobId":"1:BASE_PARITY:dbf833e0903dd75d"}
17:26:07 [17:26:07.714]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testefHlMG/1 with map size 1073741824
17:26:07 [17:26:07.722]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.743]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0012da91e38b277 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b0012da91e38b277"}
17:26:07 [17:26:07.745]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testefHlMG/2 with map size 1073741824
17:26:07 [17:26:07.750]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.773]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dbf833e0903dd75d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:dbf833e0903dd75d"}
17:26:07 [17:26:07.774]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:dbf833e0903dd75d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:dbf833e0903dd75d"}
17:26:07 [17:26:07.781]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b0012da91e38b277 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:b0012da91e38b277"}
17:26:07 [17:26:07.792]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.793]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:976802da93f60fa4 err=test error
{"provingJoId":"1:BASE_PARITY:976802da93f60fa4"}
17:26:07 [17:26:07.797]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.799]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:193a10e2191eb16b not found
{"provingJobId":"1:BASE_PARITY:193a10e2191eb16b"}
17:26:07 [17:26:07.803]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.804]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f57208b2ef06b866 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f57208b2ef06b866"}
17:26:07 [17:26:07.805]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0fIbFN/1 with map size 1073741824
17:26:07 [17:26:07.812]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.838]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.839]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d824fb6be4a29c70 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d824fb6be4a29c70"}
17:26:07 [17:26:07.840]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw2XD3Z/1 with map size 1073741824
17:26:07 [17:26:07.843]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:07 [17:26:07.968]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d824fb6be4a29c70 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:d824fb6be4a29c70"}
17:26:07 [17:26:07.978]
INFO:
prover-client:proving-broker Proving Broker started
17:26:07 [17:26:07.979]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b01610d6f8a7dd54 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b01610d6f8a7dd54"}
17:26:07 [17:26:07.980]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHVwkBI/1 with map size 1073741824
17:26:07 [17:26:07.985]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.135]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b01610d6f8a7dd54 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b01610d6f8a7dd54"}
17:26:08 [17:26:08.137]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d73c72f99cf287be epochNumber=10
{"provingJobId":"1:BASE_PARITY:d73c72f99cf287be"}
17:26:08 [17:26:08.137]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testHVwkBI/10 with map size 1073741824
17:26:08 [17:26:08.139]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.161]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:08 [17:26:08.161]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:26:08 [17:26:08.285]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.286]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc22a1391828eb35 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bc22a1391828eb35"}
17:26:08 [17:26:08.287]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKmnIcW/1 with map size 1073741824
17:26:08 [17:26:08.289]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.465]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bc22a1391828eb35 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:bc22a1391828eb35"}
17:26:08 [17:26:08.518]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.520]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0fda0bfb6159bb9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0fda0bfb6159bb9"}
17:26:08 [17:26:08.522]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKsk7rp/1 with map size 1073741824
17:26:08 [17:26:08.525]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.546]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d0fda0bfb6159bb9 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:d0fda0bfb6159bb9"}
17:26:08 [17:26:08.551]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.552]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:657e868b5f061a3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:657e868b5f061a3e"}
17:26:08 [17:26:08.554]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2qHk0m/1 with map size 1073741824
17:26:08 [17:26:08.557]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.575]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:657e868b5f061a3e type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:657e868b5f061a3e"}
17:26:08 [17:26:08.576]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:657e868b5f061a3e type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:657e868b5f061a3e"}
17:26:08 [17:26:08.586]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.587]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8ee7c62fec32fae2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8ee7c62fec32fae2"}
17:26:08 [17:26:08.587]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrxh4Nu/1 with map size 1073741824
17:26:08 [17:26:08.591]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.609]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8ee7c62fec32fae2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8ee7c62fec32fae2"}
17:26:08 [17:26:08.622]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.623]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b87bc77396bcbff epochNumber=1
{"provingJobId":"1:BASE_PARITY:8b87bc77396bcbff"}
17:26:08 [17:26:08.624]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4GjGh9/1 with map size 1073741824
17:26:08 [17:26:08.627]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.650]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8b87bc77396bcbff type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:8b87bc77396bcbff"}
17:26:08 [17:26:08.652]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:28be80a4e626093a epochNumber=3
{"provingJobId":"1:BASE_PARITY:28be80a4e626093a"}
17:26:08 [17:26:08.652]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-test4GjGh9/3 with map size 1073741824
17:26:08 [17:26:08.654]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.697]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:08 [17:26:08.698]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:26:08 [17:26:08.699]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8b87bc77396bcbff err=test error
{"provingJoId":"1:BASE_PARITY:8b87bc77396bcbff"}
17:26:08 [17:26:08.707]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUxrPZR/1 with map size 1073741824
17:26:08 [17:26:08.710]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.730]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUxrPZR/2 with map size 1073741824
17:26:08 [17:26:08.734]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.752]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.753]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5cb1156e979d15e3 settled=false
{"provingJobId":"1:BASE_PARITY:5cb1156e979d15e3","status":"pending"}
17:26:08 [17:26:08.755]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:5afca65e5bff9514 settled=false
{"provingJobId":"1:BASE_PARITY:5afca65e5bff9514","status":"pending"}
17:26:08 [17:26:08.760]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLw37jd/1 with map size 1073741824
17:26:08 [17:26:08.762]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.782]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLw37jd/2 with map size 1073741824
17:26:08 [17:26:08.785]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.817]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.819]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d1efefead0c2a5d8 settled=true
{"provingJobId":"1:BASE_PARITY:d1efefead0c2a5d8","status":"fulfilled"}
17:26:08 [17:26:08.821]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:13c7b59f1eabb6cd settled=true
{"provingJobId":"2:BASE_PARITY:13c7b59f1eabb6cd","status":"fulfilled"}
17:26:08 [17:26:08.826]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testOij9qp/1 with map size 1073741824
17:26:08 [17:26:08.828]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.863]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testOij9qp/2 with map size 1073741824
17:26:08 [17:26:08.867]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.886]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.889]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3c61a03a00625fb7 settled=true
{"provingJobId":"1:BASE_PARITY:3c61a03a00625fb7","status":"fulfilled"}
17:26:08 [17:26:08.891]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9b7a12edd4531488 settled=false
{"provingJobId":"1:BASE_PARITY:9b7a12edd4531488","status":"pending"}
17:26:08 [17:26:08.899]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.900]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:089732e4553ba5f4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:089732e4553ba5f4"}
17:26:08 [17:26:08.902]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYMTEUU/1 with map size 1073741824
17:26:08 [17:26:08.905]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.941]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:375063807d241106 epochNumber=1
{"provingJobId":"1:BASE_PARITY:375063807d241106"}
17:26:08 [17:26:08.943]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:375063807d241106: Error: db error: Error: db error
17:26:08 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:375063807d241106"}
17:26:08 [17:26:08.949]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.950]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ac2aeb786796a111 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ac2aeb786796a111"}
17:26:08 [17:26:08.951]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdhqDoG/1 with map size 1073741824
17:26:08 [17:26:08.958]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:08 [17:26:08.978]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ac2aeb786796a111 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ac2aeb786796a111"}
17:26:08 [17:26:08.978]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ac2aeb786796a111 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ac2aeb786796a111"}
17:26:08 [17:26:08.991]
INFO:
prover-client:proving-broker Proving Broker started
17:26:08 [17:26:08.994]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:964e5645f1c22a56 epochNumber=1
{"provingJobId":"1:BASE_PARITY:964e5645f1c22a56"}
17:26:08 [17:26:08.995]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testymRyS7/1 with map size 1073741824
17:26:08 [17:26:08.999]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.032]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:964e5645f1c22a56 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:964e5645f1c22a56"}
17:26:09 [17:26:09.033]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:964e5645f1c22a56 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:964e5645f1c22a56"}
17:26:09 [17:26:09.033]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:964e5645f1c22a56: Error: db error
17:26:09 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:964e5645f1c22a56"}
17:26:09 [17:26:09.037]
INFO:
prover-client:proving-broker Proving Broker started
17:26:09 [17:26:09.038]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f6468eea888f174 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f6468eea888f174"}
17:26:09 [17:26:09.039]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testu1z0nQ/1 with map size 1073741824
17:26:09 [17:26:09.042]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.064]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4f6468eea888f174 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4f6468eea888f174"}
17:26:09 [17:26:09.065]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4f6468eea888f174 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4f6468eea888f174"}
17:26:09 [17:26:09.077]
INFO:
prover-client:proving-broker Proving Broker started
17:26:09 [17:26:09.078]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:526c085a798c0107 epochNumber=1
{"provingJobId":"1:BASE_PARITY:526c085a798c0107"}
17:26:09 [17:26:09.079]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCsgB6S/1 with map size 1073741824
17:26:09 [17:26:09.083]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.140]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:526c085a798c0107 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:526c085a798c0107"}
17:26:09 [17:26:09.141]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:526c085a798c0107 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:526c085a798c0107"}
17:26:09 [17:26:09.142]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:526c085a798c0107 jobErr=test error: Error: db error
17:26:09 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:526c085a798c0107"}
17:26:09 [17:26:09.148]
INFO:
prover-client:proving-broker Proving Broker started
17:26:09 [17:26:09.148]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7d622c3f5dc93bdd not found
{"provingJobId":"1:BASE_PARITY:7d622c3f5dc93bdd"}
17:26:09 [17:26:09.152]
INFO:
prover-client:proving-broker Proving Broker started
17:26:09 [17:26:09.153]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:372f347d283a820f err=test error
{"provingJoId":"1:BASE_PARITY:372f347d283a820f"}
17:26:09 [17:26:09.157]
INFO:
prover-client:proving-broker Proving Broker started
17:26:09 [17:26:09.167]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a41315811d1490a4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a41315811d1490a4"}
17:26:09 [17:26:09.169]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhKankr/1 with map size 1073741824
17:26:09 [17:26:09.173]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.198]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a41315811d1490a4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a41315811d1490a4"}
17:26:09 [17:26:09.199]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a41315811d1490a4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a41315811d1490a4"}
17:26:09 [17:26:09.227]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:8fa9cc77fe2b0919 epochNumber=2
{"provingJobId":"2:BASE_PARITY:8fa9cc77fe2b0919"}
17:26:09 [17:26:09.230]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhKankr/2 with map size 1073741824
17:26:09 [17:26:09.239]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.266]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:8fa9cc77fe2b0919 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:8fa9cc77fe2b0919"}
17:26:09 [17:26:09.266]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:8fa9cc77fe2b0919 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:8fa9cc77fe2b0919"}
17:26:09 [17:26:09.336]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:e65594b69ded40d8 epochNumber=3
{"provingJobId":"3:BASE_PARITY:e65594b69ded40d8"}
17:26:09 [17:26:09.338]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testhKankr/3 with map size 1073741824
17:26:09 [17:26:09.346]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.394]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:09 [17:26:09.395]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
17:26:09 [17:26:09.430]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:6585dbee793ce84b epochNumber=4
{"provingJobId":"4:BASE_PARITY:6585dbee793ce84b"}
17:26:09 [17:26:09.432]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testhKankr/4 with map size 1073741824
17:26:09 [17:26:09.440]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.476]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:09 [17:26:09.476]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
17:26:09 [17:26:09.525]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:050f285955b3f11d epochNumber=5
{"provingJobId":"5:BASE_PARITY:050f285955b3f11d"}
17:26:09 [17:26:09.526]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testhKankr/5 with map size 1073741824
17:26:09 [17:26:09.529]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
17:26:09 [17:26:09.557]
INFO:
prover-client:proving-broker Cleaned up jobs=1
17:26:09 [17:26:09.558]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
17:26:09
PASS src/proving_broker/proving_broker.test.ts (
13.658 s)
17:26:09 ProvingBroker
17:26:09 Producer API
17:26:09
✓ refuses stale jobs (54 ms)
17:26:09
✓ enqueues jobs (4 ms)
17:26:09
✓ ignores duplicate jobs (5 ms)
17:26:09
✓ reports correct status when enqueuing repeat jobs (5 ms)
17:26:09
✓ throws an error in case of duplicate job IDs (25 ms)
17:26:09
✓ returns not-found status for non-existing jobs (2 ms)
17:26:09
✓ cancels jobs in queue (3 ms)
17:26:09
✓ cancels jobs in-progress (3 ms)
17:26:09
✓ returns job result if successful (3 ms)
17:26:09
✓ returns job error if failed (3 ms)
17:26:09
✓ correctly returns job status for concurrent writes (3 ms)
17:26:09
✓ refuses stale jobs (39 ms)
17:26:09
✓ enqueues jobs (34 ms)
17:26:09
✓ ignores duplicate jobs (42 ms)
17:26:09
✓ reports correct status when enqueuing repeat jobs (37 ms)
17:26:09
✓ throws an error in case of duplicate job IDs (34 ms)
17:26:09
✓ returns not-found status for non-existing jobs (5 ms)
17:26:09
✓ cancels jobs in queue (42 ms)
17:26:09
✓ cancels jobs in-progress (34 ms)
17:26:09
✓ returns job result if successful (45 ms)
17:26:09
✓ returns job error if failed (31 ms)
17:26:09
✓ correctly returns job status for concurrent writes (41 ms)
17:26:09 Consumer API
17:26:09
✓ returns undefined if no jobs are available (9 ms)
17:26:09
✓ returns jobs in priority order (2 ms)
17:26:09
✓ returns undefined if no jobs are available for the given allowList (2 ms)
17:26:09
✓ returns a job if it is in the allowList (4 ms)
17:26:09
✓ returns the most important job if it is in the allowList (4 ms)
17:26:09
✓ returns any job if filter is empty (4 ms)
17:26:09
✓ returns a new job when reporting job success (3 ms)
17:26:09
✓ returns a new job when reporting permanent error (2 ms)
17:26:09
✓ returns a new job when reporting retry-able error (2 ms)
17:26:09
✓ returns a new job when reporting progress if current one is cancelled (2 ms)
17:26:09
✓ returns a new job if job is already in progress elsewhere (1057 ms)
17:26:09
✓ avoids sending the same job to a new agent after a restart (1003 ms)
17:26:09
✓ avoids sending a completed job to a new agent after a restart (1007 ms)
17:26:09
✓ tracks job result if in progress (2 ms)
17:26:09
✓ tracks job result even if job is in queue (7 ms)
17:26:09
✓ ignores reported job error if unknown job (2 ms)
17:26:09
✓ ignores job result if unknown job (1 ms)
17:26:09
✓ returns undefined if no jobs are available (6 ms)
17:26:09
✓ returns jobs in priority order (57 ms)
17:26:09
✓ returns undefined if no jobs are available for the given allowList (25 ms)
17:26:09
✓ returns a job if it is in the allowList (71 ms)
17:26:09
✓ returns the most important job if it is in the allowList (62 ms)
17:26:09
✓ returns any job if filter is empty (70 ms)
17:26:09
✓ returns a new job when reporting job success (47 ms)
17:26:09
✓ returns a new job when reporting permanent error (39 ms)
17:26:09
✓ returns a new job when reporting retry-able error (37 ms)
17:26:09
✓ returns a new job when reporting progress if current one is cancelled (43 ms)
17:26:09
✓ returns a new job if job is already in progress elsewhere (1116 ms)
17:26:09
✓ avoids sending the same job to a new agent after a restart (1069 ms)
17:26:09
✓ avoids sending a completed job to a new agent after a restart (1069 ms)
17:26:09
✓ tracks job result if in progress (71 ms)
17:26:09
✓ tracks job result even if job is in queue (80 ms)
17:26:09
✓ ignores reported job error if unknown job (5 ms)
17:26:09
✓ ignores job result if unknown job (6 ms)
17:26:09 Timeouts
17:26:09
✓ tracks in progress jobs (2 ms)
17:26:09
✓ re-enqueues jobs that time out (127 ms)
17:26:09
✓ cancel stale jobs that time out (234 ms)
17:26:09
✓ keeps the jobs in progress while it is alive (204 ms)
17:26:09
✓ tracks in progress jobs (32 ms)
17:26:09
✓ re-enqueues jobs that time out (138 ms)
17:26:09
✓ cancel stale jobs that time out (309 ms)
17:26:09
✓ keeps the jobs in progress while it is alive (231 ms)
17:26:09 Retries
17:26:09
✓ retries jobs (3 ms)
17:26:09
✓ retries up to a maximum number of times (4 ms)
17:26:09
✓ passing retry=false does not retry (2 ms)
17:26:09
✓ does not retry if job is stale (31 ms)
17:26:09
✓ retries jobs (33 ms)
17:26:09
✓ retries up to a maximum number of times (34 ms)
17:26:09
✓ passing retry=false does not retry (35 ms)
17:26:09
✓ does not retry if job is stale (81 ms)
17:26:09 Database management
17:26:09
✓ re-enqueues proof requests on start (8 ms)
17:26:09
✓ restores proof results on start (3 ms)
17:26:09
✓ only re-enqueues unfinished jobs (2 ms)
17:26:09
✓ saves job when enqueued (3 ms)
17:26:09
✓ does not retain job if database fails to save (4 ms)
17:26:09
✓ saves job result (3 ms)
17:26:09
✓ saves result even if database fails to save (3 ms)
17:26:09
✓ saves job error (3 ms)
17:26:09
✓ saves job error even if database fails to save (2 ms)
17:26:09
✓ does not save job result if job is unknown (2 ms)
17:26:09
✓ does not save job error if job is unknown (1 ms)
17:26:09
✓ cleans up old jobs periodically (263 ms)
17:26:09
✓ re-enqueues proof requests on start (56 ms)
17:26:09
✓ restores proof results on start (65 ms)
17:26:09
✓ only re-enqueues unfinished jobs (70 ms)
17:26:09
✓ saves job when enqueued (43 ms)
17:26:09
✓ does not retain job if database fails to save (6 ms)
17:26:09
✓ saves job result (41 ms)
17:26:09
✓ saves result even if database fails to save (48 ms)
17:26:09
✓ saves job error (39 ms)
17:26:09
✓ saves job error even if database fails to save (69 ms)
17:26:09
✓ does not save job result if job is unknown (4 ms)
17:26:09
✓ does not save job error if job is unknown (4 ms)
17:26:09
✓ cleans up old jobs periodically (456 ms)
17:26:09
17:26:09
Test Suites: 1 passed, 1 total
17:26:09
Tests: 96 passed, 96 total
17:26:09
Snapshots: 0 total
17:26:09
Time: 13.778 s
17:26:09
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
17:26:09
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?