Parent Log:
http://ci.aztec-labs.com/3dbf639c7472924d
Command: 73379369dc8552ae yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/73e6175e802437ee93b26885b6aff2fc12640523
Env: REF_NAME=gh-readonly-queue/next/pr-15163-4608046e462faf9ff3fbee265ba34dbecb5d0b31 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 23 14:14:12 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15163_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
14:14:17 [14:14:17.130]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.137]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9664b935b923a072 epochNumber=42
{"provingJobId":"1:BASE_PARITY:9664b935b923a072"}
14:14:17 [14:14:17.140]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:0dff0f1a6538481a epochNumber=1
{"provingJobId":"1:BASE_PARITY:0dff0f1a6538481a"}
14:14:17 [14:14:17.166]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.167]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ec5bc069c889325 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7ec5bc069c889325"}
14:14:17 [14:14:17.168]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:064fb34c271ada7d epochNumber=1
{"provingJobId":"1:BASE_PARITY:064fb34c271ada7d"}
14:14:17 [14:14:17.169]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.170]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:02e6c4a14e3d4d91 epochNumber=1
{"provingJobId":"1:BASE_PARITY:02e6c4a14e3d4d91"}
14:14:17 [14:14:17.171]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:02e6c4a14e3d4d91 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:02e6c4a14e3d4d91"}
14:14:17 [14:14:17.172]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.173]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7557bf043b7355a8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7557bf043b7355a8"}
14:14:17 [14:14:17.174]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7557bf043b7355a8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:7557bf043b7355a8"}
14:14:17 [14:14:17.174]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7557bf043b7355a8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7557bf043b7355a8"}
14:14:17 [14:14:17.175]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7557bf043b7355a8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:7557bf043b7355a8"}
14:14:17 [14:14:17.176]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.177]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:309732847ecc639a epochNumber=1
{"provingJobId":"1:BASE_PARITY:309732847ecc639a"}
14:14:17 [14:14:17.218]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.219]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.220]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68bb8de2ee354792 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68bb8de2ee354792"}
14:14:17 [14:14:17.221]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:68bb8de2ee354792
{"provingJobId":"1:BASE_PARITY:68bb8de2ee354792"}
14:14:17 [14:14:17.221]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:68bb8de2ee354792 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:68bb8de2ee354792"}
14:14:17 [14:14:17.221]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:68bb8de2ee354792 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:68bb8de2ee354792"}
14:14:17 [14:14:17.223]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.223]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:633aeac882c74c72 epochNumber=1
{"provingJobId":"1:BASE_PARITY:633aeac882c74c72"}
14:14:17 [14:14:17.224]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:633aeac882c74c72
{"provingJobId":"1:BASE_PARITY:633aeac882c74c72"}
14:14:17 [14:14:17.224]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:633aeac882c74c72 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:633aeac882c74c72"}
14:14:17 [14:14:17.226]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.226]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f85183ca8ff5493 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2f85183ca8ff5493"}
14:14:17 [14:14:17.227]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2f85183ca8ff5493 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2f85183ca8ff5493"}
14:14:17 [14:14:17.227]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2f85183ca8ff5493 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2f85183ca8ff5493"}
14:14:17 [14:14:17.229]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.230]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7b89e50cf5136039 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7b89e50cf5136039"}
14:14:17 [14:14:17.231]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7b89e50cf5136039 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7b89e50cf5136039"}
14:14:17 [14:14:17.232]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7b89e50cf5136039 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7b89e50cf5136039"}
14:14:17 [14:14:17.234]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.235]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:94ea1d6303c42a0b epochNumber=0
{"provingJobId":"1:BASE_PARITY:94ea1d6303c42a0b"}
14:14:17 [14:14:17.235]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:94ea1d6303c42a0b epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:94ea1d6303c42a0b"}
14:14:17 [14:14:17.236]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:301b76a52132d67b epochNumber=0
{"provingJobId":"1:BASE_PARITY:301b76a52132d67b"}
14:14:17 [14:14:17.236]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:94ea1d6303c42a0b epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:94ea1d6303c42a0b"}
14:14:17 [14:14:17.236]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f7082a681f6948b epochNumber=0
{"provingJobId":"1:BASE_PARITY:7f7082a681f6948b"}
14:14:17 [14:14:17.238]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.239]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.240]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80e4ddaaabde996b epochNumber=2
{"provingJobId":"1:BASE_PARITY:80e4ddaaabde996b"}
14:14:17 [14:14:17.240]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1824da23b629cb66 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1824da23b629cb66"}
14:14:17 [14:14:17.241]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.242]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c4d538f8e94a184a epochNumber=1
{"provingJobId":"1:BASE_PARITY:c4d538f8e94a184a"}
14:14:17 [14:14:17.243]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1a5dc14aaf4cec63 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1a5dc14aaf4cec63"}
14:14:17 [14:14:17.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9ea20c1fc379542c epochNumber=1
{"provingJobId":"1:BASE_PARITY:9ea20c1fc379542c"}
14:14:17 [14:14:17.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01cdabc1289319e9 epochNumber=2
{"provingJobId":"1:BASE_PARITY:01cdabc1289319e9"}
14:14:17 [14:14:17.245]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:148b9b4c68ae08b7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:148b9b4c68ae08b7"}
14:14:17 [14:14:17.246]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:475d8f3be3101aa0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:475d8f3be3101aa0"}
14:14:17 [14:14:17.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f1fa85f4842af500 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f1fa85f4842af500"}
14:14:17 [14:14:17.247]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8e8e2b1250d85a95 epochNumber=2
{"provingJobId":"1:BASE_PARITY:8e8e2b1250d85a95"}
14:14:17 [14:14:17.247]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c5a66eeb43b7a8c epochNumber=1
{"provingJobId":"1:BASE_PARITY:7c5a66eeb43b7a8c"}
14:14:17 [14:14:17.248]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:108748a5e0fe8956 epochNumber=1
{"provingJobId":"1:BASE_PARITY:108748a5e0fe8956"}
14:14:17 [14:14:17.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a9943b878a2a2f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a9943b878a2a2f8"}
14:14:17 [14:14:17.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1cd1a0768af31e83 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1cd1a0768af31e83"}
14:14:17 [14:14:17.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0b73d7a13e818c0c epochNumber=1
{"provingJobId":"1:BASE_PARITY:0b73d7a13e818c0c"}
14:14:17 [14:14:17.251]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.251]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:39f09cd06eea56fe epochNumber=1
{"provingJobId":"1:BASE_PARITY:39f09cd06eea56fe"}
14:14:17 [14:14:17.252]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7aaf373f976db84 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d7aaf373f976db84"}
14:14:17 [14:14:17.252]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:39f09cd06eea56fe type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:39f09cd06eea56fe"}
14:14:17 [14:14:17.253]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1625c97e575114a1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1625c97e575114a1"}
14:14:17 [14:14:17.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5d50bba7ecee0edd epochNumber=1
{"provingJobId":"1:BASE_PARITY:5d50bba7ecee0edd"}
14:14:17 [14:14:17.255]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1625c97e575114a1 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:1625c97e575114a1"}
14:14:17 [14:14:17.256]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.256]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9b4ff49c2b5a8af3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9b4ff49c2b5a8af3"}
14:14:17 [14:14:17.257]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:40811a4cfc1ab940 epochNumber=1
{"provingJobId":"1:BASE_PARITY:40811a4cfc1ab940"}
14:14:17 [14:14:17.257]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9b4ff49c2b5a8af3 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:9b4ff49c2b5a8af3"}
14:14:17 [14:14:17.258]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.259]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:437309cc5fe5d021 epochNumber=1
{"provingJobId":"1:BASE_PARITY:437309cc5fe5d021"}
14:14:17 [14:14:17.259]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:437309cc5fe5d021
{"provingJobId":"1:BASE_PARITY:437309cc5fe5d021"}
14:14:17 [14:14:17.259]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:437309cc5fe5d021 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:437309cc5fe5d021"}
14:14:17 [14:14:17.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:570eb51a6c709548 epochNumber=1
{"provingJobId":"1:BASE_PARITY:570eb51a6c709548"}
14:14:17 [14:14:17.260]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:437309cc5fe5d021 has already been completed
{"provingJobId":"1:BASE_PARITY:437309cc5fe5d021"}
14:14:17 [14:14:17.261]
INFO:
prover-client:proving-broker Proving Broker started
14:14:17 [14:14:17.262]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eded6de6907517a2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:eded6de6907517a2"}
14:14:17 [14:14:17.262]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7e02aaed0118e44 epochNumber=2
{"provingJobId":"1:BASE_PARITY:b7e02aaed0118e44"}
14:14:18 [14:14:18.315]
INFO:
prover-client:proving-broker Proving Broker started
14:14:18 [14:14:18.316]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:eded6de6907517a2 settled=false
{"provingJobId":"1:BASE_PARITY:eded6de6907517a2","status":"pending"}
14:14:18 [14:14:18.316]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b7e02aaed0118e44 settled=false
{"provingJobId":"1:BASE_PARITY:b7e02aaed0118e44","status":"pending"}
14:14:18 [14:14:18.317]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eded6de6907517a2 type=BASE_PARITY startedAt=1750688057262 older agent has taken job
{"provingJobId":"1:BASE_PARITY:eded6de6907517a2"}
14:14:18 [14:14:18.318]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:eded6de6907517a2 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:eded6de6907517a2"}
14:14:18 [14:14:18.319]
INFO:
prover-client:proving-broker Proving Broker started
14:14:18 [14:14:18.319]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bdcb2241fcab29f8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bdcb2241fcab29f8"}
14:14:18 [14:14:18.320]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4e282ceb38c23afe epochNumber=2
{"provingJobId":"1:BASE_PARITY:4e282ceb38c23afe"}
14:14:19 [14:14:19.320]
INFO:
prover-client:proving-broker Proving Broker started
14:14:19 [14:14:19.321]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bdcb2241fcab29f8 settled=false
{"provingJobId":"1:BASE_PARITY:bdcb2241fcab29f8","status":"pending"}
14:14:19 [14:14:19.321]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4e282ceb38c23afe settled=false
{"provingJobId":"1:BASE_PARITY:4e282ceb38c23afe","status":"pending"}
14:14:19 [14:14:19.322]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:bdcb2241fcab29f8 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:bdcb2241fcab29f8"}
14:14:19 [14:14:19.323]
INFO:
prover-client:proving-broker Proving Broker started
14:14:19 [14:14:19.324]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c9879e2a5b793b11 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c9879e2a5b793b11"}
14:14:19 [14:14:19.324]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2941599633a61fd1 epochNumber=2
{"provingJobId":"1:BASE_PARITY:2941599633a61fd1"}
14:14:20 [14:14:20.325]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.326]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c9879e2a5b793b11 settled=false
{"provingJobId":"1:BASE_PARITY:c9879e2a5b793b11","status":"pending"}
14:14:20 [14:14:20.326]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2941599633a61fd1 settled=false
{"provingJobId":"1:BASE_PARITY:2941599633a61fd1","status":"pending"}
14:14:20 [14:14:20.327]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c9879e2a5b793b11 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c9879e2a5b793b11"}
14:14:20 [14:14:20.327]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c9879e2a5b793b11 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c9879e2a5b793b11"}
14:14:20 [14:14:20.329]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.329]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d2eb272b80e9b84 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d2eb272b80e9b84"}
14:14:20 [14:14:20.330]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a8ae6e98d36329f6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a8ae6e98d36329f6"}
14:14:20 [14:14:20.330]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2d2eb272b80e9b84 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2d2eb272b80e9b84"}
14:14:20 [14:14:20.331]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a8ae6e98d36329f6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a8ae6e98d36329f6"}
14:14:20 [14:14:20.332]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.332]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:82da97799ae8d03e epochNumber=1
{"provingJobId":"1:BASE_PARITY:82da97799ae8d03e"}
14:14:20 [14:14:20.333]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:79790d378f834612 epochNumber=2
{"provingJobId":"1:BASE_PARITY:79790d378f834612"}
14:14:20 [14:14:20.333]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:82da97799ae8d03e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:82da97799ae8d03e"}
14:14:20 [14:14:20.333]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:82da97799ae8d03e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:82da97799ae8d03e"}
14:14:20 [14:14:20.334]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:79790d378f834612 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:79790d378f834612"}
14:14:20 [14:14:20.335]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.335]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:a30ac7fa89c7d606 err=test error
{"provingJoId":"1:BASE_PARITY:a30ac7fa89c7d606"}
14:14:20 [14:14:20.336]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.337]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3cf23ec03295b7a0 not found
{"provingJobId":"1:BASE_PARITY:3cf23ec03295b7a0"}
14:14:20 [14:14:20.338]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.339]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:92bdb1591eccde67 epochNumber=1
{"provingJobId":"1:BASE_PARITY:92bdb1591eccde67"}
14:14:20 [14:14:20.340]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.340]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:65fc84634a729aa9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:65fc84634a729aa9"}
14:14:20 [14:14:20.441]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:65fc84634a729aa9 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:65fc84634a729aa9"}
14:14:20 [14:14:20.449]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.449]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:786c057567fdd30f epochNumber=1
{"provingJobId":"1:BASE_PARITY:786c057567fdd30f"}
14:14:20 [14:14:20.550]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:786c057567fdd30f timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:786c057567fdd30f"}
14:14:20 [14:14:20.557]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd1900a188f3a10b epochNumber=10
{"provingJobId":"1:BASE_PARITY:bd1900a188f3a10b"}
14:14:20 [14:14:20.576]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:20 [14:14:20.684]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.684]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:630c6843b13ef73d epochNumber=1
{"provingJobId":"1:BASE_PARITY:630c6843b13ef73d"}
14:14:20 [14:14:20.836]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:630c6843b13ef73d timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:630c6843b13ef73d"}
14:14:20 [14:14:20.888]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.890]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3bf698829b90c398 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3bf698829b90c398"}
14:14:20 [14:14:20.891]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3bf698829b90c398 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3bf698829b90c398"}
14:14:20 [14:14:20.894]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.895]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a852f0ea6aac19f0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a852f0ea6aac19f0"}
14:14:20 [14:14:20.896]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a852f0ea6aac19f0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a852f0ea6aac19f0"}
14:14:20 [14:14:20.898]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a852f0ea6aac19f0 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:a852f0ea6aac19f0"}
14:14:20 [14:14:20.899]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.900]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30c80f22adfb748c epochNumber=1
{"provingJobId":"1:BASE_PARITY:30c80f22adfb748c"}
14:14:20 [14:14:20.901]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:30c80f22adfb748c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:30c80f22adfb748c"}
14:14:20 [14:14:20.904]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.904]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:21c4ee1a50c8f91c epochNumber=1
{"provingJobId":"1:BASE_PARITY:21c4ee1a50c8f91c"}
14:14:20 [14:14:20.906]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:21c4ee1a50c8f91c type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:21c4ee1a50c8f91c"}
14:14:20 [14:14:20.907]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e04c0a1b2e8fa77 epochNumber=3
{"provingJobId":"1:BASE_PARITY:6e04c0a1b2e8fa77"}
14:14:20 [14:14:20.929]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:20 [14:14:20.932]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:21c4ee1a50c8f91c err=test error
{"provingJoId":"1:BASE_PARITY:21c4ee1a50c8f91c"}
14:14:20 [14:14:20.934]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.934]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9525950d68dd1f8b settled=false
{"provingJobId":"1:BASE_PARITY:9525950d68dd1f8b","status":"pending"}
14:14:20 [14:14:20.934]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d6a0b07fd3300f55 settled=false
{"provingJobId":"1:BASE_PARITY:d6a0b07fd3300f55","status":"pending"}
14:14:20 [14:14:20.937]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.938]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8c900f08ea7a7275 settled=true
{"provingJobId":"1:BASE_PARITY:8c900f08ea7a7275","status":"fulfilled"}
14:14:20 [14:14:20.938]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:e148a9137b907b6d settled=true
{"provingJobId":"2:BASE_PARITY:e148a9137b907b6d","status":"fulfilled"}
14:14:20 [14:14:20.939]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.939]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ad8b59d9bedbad17 settled=true
{"provingJobId":"1:BASE_PARITY:ad8b59d9bedbad17","status":"fulfilled"}
14:14:20 [14:14:20.939]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cf432ed7551f5ce9 settled=false
{"provingJobId":"1:BASE_PARITY:cf432ed7551f5ce9","status":"pending"}
14:14:20 [14:14:20.940]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.941]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:03cce608a4bb393e epochNumber=1
{"provingJobId":"1:BASE_PARITY:03cce608a4bb393e"}
14:14:20 [14:14:20.943]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.943]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f92aa3b00b50712 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2f92aa3b00b50712"}
14:14:20 [14:14:20.944]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:2f92aa3b00b50712: Error: db error: Error: db error
14:14:20 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:2f92aa3b00b50712"}
14:14:20 [14:14:20.946]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.946]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3df11a4f3a548986 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3df11a4f3a548986"}
14:14:20 [14:14:20.946]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3df11a4f3a548986 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3df11a4f3a548986"}
14:14:20 [14:14:20.947]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3df11a4f3a548986 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:3df11a4f3a548986"}
14:14:20 [14:14:20.948]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.949]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aa7e1e6565cde432 epochNumber=1
{"provingJobId":"1:BASE_PARITY:aa7e1e6565cde432"}
14:14:20 [14:14:20.949]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:aa7e1e6565cde432 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:aa7e1e6565cde432"}
14:14:20 [14:14:20.949]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:aa7e1e6565cde432 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:aa7e1e6565cde432"}
14:14:20 [14:14:20.950]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:aa7e1e6565cde432: Error: db error
14:14:20 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:aa7e1e6565cde432"}
14:14:20 [14:14:20.951]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.951]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:017ee769675457a5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:017ee769675457a5"}
14:14:20 [14:14:20.951]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:017ee769675457a5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:017ee769675457a5"}
14:14:20 [14:14:20.952]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:017ee769675457a5 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:017ee769675457a5"}
14:14:20 [14:14:20.953]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3abbe82a9ef783e1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3abbe82a9ef783e1"}
14:14:20 [14:14:20.953]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3abbe82a9ef783e1 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:3abbe82a9ef783e1"}
14:14:20 [14:14:20.953]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3abbe82a9ef783e1 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:3abbe82a9ef783e1"}
14:14:20 [14:14:20.954]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:3abbe82a9ef783e1 jobErr=test error: Error: db error
14:14:20 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:3abbe82a9ef783e1"}
14:14:20 [14:14:20.955]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.955]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:afb4c697f864ac7c not found
{"provingJobId":"1:BASE_PARITY:afb4c697f864ac7c"}
14:14:20 [14:14:20.956]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.956]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f5fdc436834c7854 err=test error
{"provingJoId":"1:BASE_PARITY:f5fdc436834c7854"}
14:14:20 [14:14:20.957]
INFO:
prover-client:proving-broker Proving Broker started
14:14:20 [14:14:20.969]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c94832fcce077cd4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c94832fcce077cd4"}
14:14:20 [14:14:20.969]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c94832fcce077cd4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c94832fcce077cd4"}
14:14:20 [14:14:20.970]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c94832fcce077cd4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c94832fcce077cd4"}
14:14:20 [14:14:20.980]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:2a26ec8f452814f2 epochNumber=2
{"provingJobId":"2:BASE_PARITY:2a26ec8f452814f2"}
14:14:20 [14:14:20.980]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:2a26ec8f452814f2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:2a26ec8f452814f2"}
14:14:20 [14:14:20.981]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:2a26ec8f452814f2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:2a26ec8f452814f2"}
14:14:21 [14:14:21.042]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:200655d905ce85f8 epochNumber=3
{"provingJobId":"3:BASE_PARITY:200655d905ce85f8"}
14:14:21 [14:14:21.060]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:21 [14:14:21.104]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:14862e5043cd192d epochNumber=4
{"provingJobId":"4:BASE_PARITY:14862e5043cd192d"}
14:14:21 [14:14:21.111]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:21 [14:14:21.167]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:57d1da48a054388e epochNumber=5
{"provingJobId":"5:BASE_PARITY:57d1da48a054388e"}
14:14:21 [14:14:21.188]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:21 [14:14:21.225]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.229]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8211994926b30d7e epochNumber=42
{"provingJobId":"1:BASE_PARITY:8211994926b30d7e"}
14:14:21 [14:14:21.231]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testxN1cE5/42 with map size 1073741824
14:14:21 [14:14:21.236]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.263]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:b0ff0fc95578c1b1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b0ff0fc95578c1b1"}
14:14:21 [14:14:21.269]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.269]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d201b0f749af95aa epochNumber=1
{"provingJobId":"1:BASE_PARITY:d201b0f749af95aa"}
14:14:21 [14:14:21.270]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7fUeeA/1 with map size 1073741824
14:14:21 [14:14:21.272]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.295]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5213b8397d879195 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5213b8397d879195"}
14:14:21 [14:14:21.310]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.311]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:292b7a856f6d28d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:292b7a856f6d28d9"}
14:14:21 [14:14:21.312]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4dvVNY/1 with map size 1073741824
14:14:21 [14:14:21.314]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.336]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:292b7a856f6d28d9 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:292b7a856f6d28d9"}
14:14:21 [14:14:21.341]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.342]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:448c6956fb85f604 epochNumber=1
{"provingJobId":"1:BASE_PARITY:448c6956fb85f604"}
14:14:21 [14:14:21.343]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsBfCWi/1 with map size 1073741824
14:14:21 [14:14:21.346]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.364]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:448c6956fb85f604 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:448c6956fb85f604"}
14:14:21 [14:14:21.365]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:448c6956fb85f604 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:448c6956fb85f604"}
14:14:21 [14:14:21.371]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:448c6956fb85f604 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:448c6956fb85f604"}
14:14:21 [14:14:21.375]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.377]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:89b6769d7b488027 epochNumber=1
{"provingJobId":"1:BASE_PARITY:89b6769d7b488027"}
14:14:21 [14:14:21.378]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIj5QT0/1 with map size 1073741824
14:14:21 [14:14:21.380]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.416]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.420]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.420]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:309462c5c725d271 epochNumber=1
{"provingJobId":"1:BASE_PARITY:309462c5c725d271"}
14:14:21 [14:14:21.422]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1nQcFB/1 with map size 1073741824
14:14:21 [14:14:21.424]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.441]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:309462c5c725d271
{"provingJobId":"1:BASE_PARITY:309462c5c725d271"}
14:14:21 [14:14:21.441]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:309462c5c725d271 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:309462c5c725d271"}
14:14:21 [14:14:21.442]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:309462c5c725d271 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:309462c5c725d271"}
14:14:21 [14:14:21.453]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.453]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:05d8baaa325bb9fd epochNumber=1
{"provingJobId":"1:BASE_PARITY:05d8baaa325bb9fd"}
14:14:21 [14:14:21.454]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4oH7kf/1 with map size 1073741824
14:14:21 [14:14:21.458]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.479]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:05d8baaa325bb9fd
{"provingJobId":"1:BASE_PARITY:05d8baaa325bb9fd"}
14:14:21 [14:14:21.479]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:05d8baaa325bb9fd type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:05d8baaa325bb9fd"}
14:14:21 [14:14:21.500]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.503]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8d084e9458951249 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8d084e9458951249"}
14:14:21 [14:14:21.509]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw4C5kF/1 with map size 1073741824
14:14:21 [14:14:21.512]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.538]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8d084e9458951249 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8d084e9458951249"}
14:14:21 [14:14:21.539]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8d084e9458951249 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8d084e9458951249"}
14:14:21 [14:14:21.547]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.548]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c147fda85f6f54f3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c147fda85f6f54f3"}
14:14:21 [14:14:21.548]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8jyKkS/1 with map size 1073741824
14:14:21 [14:14:21.552]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.571]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c147fda85f6f54f3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c147fda85f6f54f3"}
14:14:21 [14:14:21.571]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c147fda85f6f54f3 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c147fda85f6f54f3"}
14:14:21 [14:14:21.581]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.582]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:769d6b3e796f6eb1 epochNumber=0
{"provingJobId":"1:BASE_PARITY:769d6b3e796f6eb1"}
14:14:21 [14:14:21.583]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testvWnYY8/0 with map size 1073741824
14:14:21 [14:14:21.590]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.617]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:769d6b3e796f6eb1 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:769d6b3e796f6eb1"}
14:14:21 [14:14:21.618]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e5383f4543dca74 epochNumber=0
{"provingJobId":"1:BASE_PARITY:9e5383f4543dca74"}
14:14:21 [14:14:21.619]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:769d6b3e796f6eb1 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:769d6b3e796f6eb1"}
14:14:21 [14:14:21.620]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f03ff82c8b76f95 epochNumber=0
{"provingJobId":"1:BASE_PARITY:2f03ff82c8b76f95"}
14:14:21 [14:14:21.635]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.638]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.638]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ffa8d2cad02cc0ea epochNumber=2
{"provingJobId":"1:BASE_PARITY:ffa8d2cad02cc0ea"}
14:14:21 [14:14:21.639]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testFLA4Ju/2 with map size 1073741824
14:14:21 [14:14:21.643]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fef78b19dbc71087 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fef78b19dbc71087"}
14:14:21 [14:14:21.661]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFLA4Ju/1 with map size 1073741824
14:14:21 [14:14:21.664]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.683]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.684]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:418913ee7006fd8f epochNumber=1
{"provingJobId":"1:BASE_PARITY:418913ee7006fd8f"}
14:14:21 [14:14:21.689]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test4zpX7A/1 with map size 1073741824
14:14:21 [14:14:21.692]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.717]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.718]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:00d5d2e45fddf68c epochNumber=1
{"provingJobId":"1:BASE_PARITY:00d5d2e45fddf68c"}
14:14:21 [14:14:21.719]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBpqrFY/1 with map size 1073741824
14:14:21 [14:14:21.721]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.739]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eb162f71820ab91a epochNumber=1
{"provingJobId":"1:BASE_PARITY:eb162f71820ab91a"}
14:14:21 [14:14:21.746]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4ba43906555ec652 epochNumber=2
{"provingJobId":"1:BASE_PARITY:4ba43906555ec652"}
14:14:21 [14:14:21.747]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBpqrFY/2 with map size 1073741824
14:14:21 [14:14:21.749]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.765]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d7db81988f6ddcf epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d7db81988f6ddcf"}
14:14:21 [14:14:21.774]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.775]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36ac21df2860e160 epochNumber=1
{"provingJobId":"1:BASE_PARITY:36ac21df2860e160"}
14:14:21 [14:14:21.775]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8Nfh0b/1 with map size 1073741824
14:14:21 [14:14:21.777]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.801]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a52fb1125753796b epochNumber=1
{"provingJobId":"1:BASE_PARITY:a52fb1125753796b"}
14:14:21 [14:14:21.808]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d71af8290142eb08 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d71af8290142eb08"}
14:14:21 [14:14:21.809]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test8Nfh0b/2 with map size 1073741824
14:14:21 [14:14:21.812]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.828]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8dd94f787504447 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8dd94f787504447"}
14:14:21 [14:14:21.838]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.838]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e60e7e05d507ae4c epochNumber=1
{"provingJobId":"1:BASE_PARITY:e60e7e05d507ae4c"}
14:14:21 [14:14:21.839]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7NQnXp/1 with map size 1073741824
14:14:21 [14:14:21.841]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.856]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49952851104b7dd2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:49952851104b7dd2"}
14:14:21 [14:14:21.862]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:384ad5591bb02905 epochNumber=2
{"provingJobId":"1:BASE_PARITY:384ad5591bb02905"}
14:14:21 [14:14:21.862]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7NQnXp/2 with map size 1073741824
14:14:21 [14:14:21.864]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.880]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:57881ad990c78014 epochNumber=1
{"provingJobId":"1:BASE_PARITY:57881ad990c78014"}
14:14:21 [14:14:21.902]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.903]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:09cb6543aead790f epochNumber=1
{"provingJobId":"1:BASE_PARITY:09cb6543aead790f"}
14:14:21 [14:14:21.903]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzuTCm0/1 with map size 1073741824
14:14:21 [14:14:21.906]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.922]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd45f286b077897e epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd45f286b077897e"}
14:14:21 [14:14:21.928]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:09cb6543aead790f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:09cb6543aead790f"}
14:14:21 [14:14:21.936]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:abca0ccc6a6d7a5f epochNumber=1
{"provingJobId":"1:BASE_PARITY:abca0ccc6a6d7a5f"}
14:14:21 [14:14:21.937]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrXVLci/1 with map size 1073741824
14:14:21 [14:14:21.939]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bfee52002b71a954 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bfee52002b71a954"}
14:14:21 [14:14:21.961]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:abca0ccc6a6d7a5f type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:abca0ccc6a6d7a5f"}
14:14:21 [14:14:21.969]
INFO:
prover-client:proving-broker Proving Broker started
14:14:21 [14:14:21.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:548550c5af61eb50 epochNumber=1
{"provingJobId":"1:BASE_PARITY:548550c5af61eb50"}
14:14:21 [14:14:21.970]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkcavvY/1 with map size 1073741824
14:14:21 [14:14:21.972]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:21 [14:14:21.991]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5cba879c7513fee3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5cba879c7513fee3"}
14:14:21 [14:14:21.999]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:548550c5af61eb50 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:548550c5af61eb50"}
14:14:22 [14:14:22.004]
INFO:
prover-client:proving-broker Proving Broker started
14:14:22 [14:14:22.005]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:760546c0a7ee1a1b epochNumber=1
{"provingJobId":"1:BASE_PARITY:760546c0a7ee1a1b"}
14:14:22 [14:14:22.006]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHloca7/1 with map size 1073741824
14:14:22 [14:14:22.010]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:22 [14:14:22.032]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:760546c0a7ee1a1b
{"provingJobId":"1:BASE_PARITY:760546c0a7ee1a1b"}
14:14:22 [14:14:22.032]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:760546c0a7ee1a1b type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:760546c0a7ee1a1b"}
14:14:22 [14:14:22.038]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f564b51686d8133 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f564b51686d8133"}
14:14:22 [14:14:22.045]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:760546c0a7ee1a1b has already been completed
{"provingJobId":"1:BASE_PARITY:760546c0a7ee1a1b"}
14:14:22 [14:14:22.049]
INFO:
prover-client:proving-broker Proving Broker started
14:14:22 [14:14:22.050]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:75416979803d3c37 epochNumber=1
{"provingJobId":"1:BASE_PARITY:75416979803d3c37"}
14:14:22 [14:14:22.050]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testEwbErG/1 with map size 1073741824
14:14:22 [14:14:22.052]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:22 [14:14:22.068]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a951760b00f7e4ee epochNumber=2
{"provingJobId":"1:BASE_PARITY:a951760b00f7e4ee"}
14:14:22 [14:14:22.069]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testEwbErG/2 with map size 1073741824
14:14:22 [14:14:22.071]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:23 [14:14:23.140]
INFO:
prover-client:proving-broker Proving Broker started
14:14:23 [14:14:23.147]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:75416979803d3c37 settled=false
{"provingJobId":"1:BASE_PARITY:75416979803d3c37","status":"pending"}
14:14:23 [14:14:23.148]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a951760b00f7e4ee settled=false
{"provingJobId":"1:BASE_PARITY:a951760b00f7e4ee","status":"pending"}
14:14:23 [14:14:23.149]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:75416979803d3c37 type=BASE_PARITY startedAt=1750688062088 older agent has taken job
{"provingJobId":"1:BASE_PARITY:75416979803d3c37"}
14:14:23 [14:14:23.150]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:75416979803d3c37 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:75416979803d3c37"}
14:14:23 [14:14:23.155]
INFO:
prover-client:proving-broker Proving Broker started
14:14:23 [14:14:23.155]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce751d31639cb85b epochNumber=1
{"provingJobId":"1:BASE_PARITY:ce751d31639cb85b"}
14:14:23 [14:14:23.158]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwerskt/1 with map size 1073741824
14:14:23 [14:14:23.159]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:23 [14:14:23.175]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:186815a91f47ae0e epochNumber=2
{"provingJobId":"1:BASE_PARITY:186815a91f47ae0e"}
14:14:23 [14:14:23.176]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testwerskt/2 with map size 1073741824
14:14:23 [14:14:23.177]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:24 [14:14:24.194]
INFO:
prover-client:proving-broker Proving Broker started
14:14:24 [14:14:24.196]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ce751d31639cb85b settled=false
{"provingJobId":"1:BASE_PARITY:ce751d31639cb85b","status":"pending"}
14:14:24 [14:14:24.197]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:186815a91f47ae0e settled=false
{"provingJobId":"1:BASE_PARITY:186815a91f47ae0e","status":"pending"}
14:14:24 [14:14:24.198]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ce751d31639cb85b type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:ce751d31639cb85b"}
14:14:24 [14:14:24.202]
INFO:
prover-client:proving-broker Proving Broker started
14:14:24 [14:14:24.203]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d57d780ea43a2d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1d57d780ea43a2d3"}
14:14:24 [14:14:24.203]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsQoJai/1 with map size 1073741824
14:14:24 [14:14:24.206]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:24 [14:14:24.223]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aebfeb8764be9304 epochNumber=2
{"provingJobId":"1:BASE_PARITY:aebfeb8764be9304"}
14:14:24 [14:14:24.224]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testsQoJai/2 with map size 1073741824
14:14:24 [14:14:24.227]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.245]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.248]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1d57d780ea43a2d3 settled=false
{"provingJobId":"1:BASE_PARITY:1d57d780ea43a2d3","status":"pending"}
14:14:25 [14:14:25.250]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:aebfeb8764be9304 settled=false
{"provingJobId":"1:BASE_PARITY:aebfeb8764be9304","status":"pending"}
14:14:25 [14:14:25.251]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1d57d780ea43a2d3 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:1d57d780ea43a2d3"}
14:14:25 [14:14:25.251]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1d57d780ea43a2d3 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1d57d780ea43a2d3"}
14:14:25 [14:14:25.262]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.263]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7eb8b601d3f0d8d9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7eb8b601d3f0d8d9"}
14:14:25 [14:14:25.264]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testckC5Qm/1 with map size 1073741824
14:14:25 [14:14:25.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.282]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54ab06de513e296e epochNumber=2
{"provingJobId":"1:BASE_PARITY:54ab06de513e296e"}
14:14:25 [14:14:25.283]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testckC5Qm/2 with map size 1073741824
14:14:25 [14:14:25.288]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.306]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7eb8b601d3f0d8d9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7eb8b601d3f0d8d9"}
14:14:25 [14:14:25.312]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:54ab06de513e296e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:54ab06de513e296e"}
14:14:25 [14:14:25.321]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.321]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f2dd92329b796d8b epochNumber=1
{"provingJobId":"1:BASE_PARITY:f2dd92329b796d8b"}
14:14:25 [14:14:25.322]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrx1EbB/1 with map size 1073741824
14:14:25 [14:14:25.324]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.341]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce7f734e24ee3c60 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ce7f734e24ee3c60"}
14:14:25 [14:14:25.342]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrx1EbB/2 with map size 1073741824
14:14:25 [14:14:25.343]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.359]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f2dd92329b796d8b type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:f2dd92329b796d8b"}
14:14:25 [14:14:25.359]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f2dd92329b796d8b type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f2dd92329b796d8b"}
14:14:25 [14:14:25.365]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ce7f734e24ee3c60 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ce7f734e24ee3c60"}
14:14:25 [14:14:25.376]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.377]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:14b703f72982d142 err=test error
{"provingJoId":"1:BASE_PARITY:14b703f72982d142"}
14:14:25 [14:14:25.381]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.382]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d063509650fb7376 not found
{"provingJobId":"1:BASE_PARITY:d063509650fb7376"}
14:14:25 [14:14:25.395]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.396]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:65fbf78765ed8c2c epochNumber=1
{"provingJobId":"1:BASE_PARITY:65fbf78765ed8c2c"}
14:14:25 [14:14:25.398]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testClCu7b/1 with map size 1073741824
14:14:25 [14:14:25.405]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.428]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.429]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:804ac1a472bf38e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:804ac1a472bf38e6"}
14:14:25 [14:14:25.430]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgKViZb/1 with map size 1073741824
14:14:25 [14:14:25.432]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.555]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:804ac1a472bf38e6 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:804ac1a472bf38e6"}
14:14:25 [14:14:25.565]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.566]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f67b03ac89009425 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f67b03ac89009425"}
14:14:25 [14:14:25.567]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuSREsM/1 with map size 1073741824
14:14:25 [14:14:25.569]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.695]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f67b03ac89009425 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:f67b03ac89009425"}
14:14:25 [14:14:25.703]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:559834c3697a2da0 epochNumber=10
{"provingJobId":"1:BASE_PARITY:559834c3697a2da0"}
14:14:25 [14:14:25.708]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testuSREsM/10 with map size 1073741824
14:14:25 [14:14:25.712]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:25 [14:14:25.747]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:25 [14:14:25.747]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:14:25 [14:14:25.867]
INFO:
prover-client:proving-broker Proving Broker started
14:14:25 [14:14:25.868]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e6d016b6fd08a48a epochNumber=1
{"provingJobId":"1:BASE_PARITY:e6d016b6fd08a48a"}
14:14:25 [14:14:25.869]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdBpMAx/1 with map size 1073741824
14:14:25 [14:14:25.870]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.048]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e6d016b6fd08a48a timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:e6d016b6fd08a48a"}
14:14:26 [14:14:26.110]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.111]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a4e667aaef6d1f31 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a4e667aaef6d1f31"}
14:14:26 [14:14:26.112]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAtOwQC/1 with map size 1073741824
14:14:26 [14:14:26.115]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.132]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a4e667aaef6d1f31 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a4e667aaef6d1f31"}
14:14:26 [14:14:26.136]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.136]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f7e5c165f957c8a4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f7e5c165f957c8a4"}
14:14:26 [14:14:26.137]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTNRWrk/1 with map size 1073741824
14:14:26 [14:14:26.139]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.155]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f7e5c165f957c8a4 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:f7e5c165f957c8a4"}
14:14:26 [14:14:26.156]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f7e5c165f957c8a4 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:f7e5c165f957c8a4"}
14:14:26 [14:14:26.164]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.165]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81058721c804225f epochNumber=1
{"provingJobId":"1:BASE_PARITY:81058721c804225f"}
14:14:26 [14:14:26.165]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnA3w4l/1 with map size 1073741824
14:14:26 [14:14:26.167]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.185]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:81058721c804225f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:81058721c804225f"}
14:14:26 [14:14:26.203]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.204]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c31c9695337933c5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c31c9695337933c5"}
14:14:26 [14:14:26.205]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuglLNX/1 with map size 1073741824
14:14:26 [14:14:26.208]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.224]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c31c9695337933c5 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c31c9695337933c5"}
14:14:26 [14:14:26.225]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7453ac22ed779a34 epochNumber=3
{"provingJobId":"1:BASE_PARITY:7453ac22ed779a34"}
14:14:26 [14:14:26.226]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testuglLNX/3 with map size 1073741824
14:14:26 [14:14:26.228]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.253]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:26 [14:14:26.254]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:14:26 [14:14:26.273]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c31c9695337933c5 err=test error
{"provingJoId":"1:BASE_PARITY:c31c9695337933c5"}
14:14:26 [14:14:26.277]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testD8xuEu/1 with map size 1073741824
14:14:26 [14:14:26.280]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.302]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testD8xuEu/2 with map size 1073741824
14:14:26 [14:14:26.305]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.323]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.325]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:08c676a3370026a9 settled=false
{"provingJobId":"1:BASE_PARITY:08c676a3370026a9","status":"pending"}
14:14:26 [14:14:26.327]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:584496903a00f19a settled=false
{"provingJobId":"1:BASE_PARITY:584496903a00f19a","status":"pending"}
14:14:26 [14:14:26.335]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVTw0Y2/1 with map size 1073741824
14:14:26 [14:14:26.337]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.354]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVTw0Y2/2 with map size 1073741824
14:14:26 [14:14:26.356]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.384]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.387]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9a6f3f91028a3f42 settled=true
{"provingJobId":"1:BASE_PARITY:9a6f3f91028a3f42","status":"fulfilled"}
14:14:26 [14:14:26.390]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:4b95caa49a07ded5 settled=true
{"provingJobId":"2:BASE_PARITY:4b95caa49a07ded5","status":"fulfilled"}
14:14:26 [14:14:26.405]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFaJc0Z/1 with map size 1073741824
14:14:26 [14:14:26.408]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.434]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testFaJc0Z/2 with map size 1073741824
14:14:26 [14:14:26.438]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.456]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.458]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a646b15ce7a09abf settled=true
{"provingJobId":"1:BASE_PARITY:a646b15ce7a09abf","status":"fulfilled"}
14:14:26 [14:14:26.460]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9ebce6f6b8bdaf32 settled=false
{"provingJobId":"1:BASE_PARITY:9ebce6f6b8bdaf32","status":"pending"}
14:14:26 [14:14:26.467]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.468]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa8128f52353dfa9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa8128f52353dfa9"}
14:14:26 [14:14:26.469]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5f2MmX/1 with map size 1073741824
14:14:26 [14:14:26.471]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.498]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.499]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3d2803a981609878 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3d2803a981609878"}
14:14:26 [14:14:26.500]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:3d2803a981609878: Error: db error: Error: db error
14:14:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:3d2803a981609878"}
14:14:26 [14:14:26.506]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.506]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db350c6d36917dfe epochNumber=1
{"provingJobId":"1:BASE_PARITY:db350c6d36917dfe"}
14:14:26 [14:14:26.508]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBQgOmf/1 with map size 1073741824
14:14:26 [14:14:26.510]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.527]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:db350c6d36917dfe type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:db350c6d36917dfe"}
14:14:26 [14:14:26.528]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:db350c6d36917dfe type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:db350c6d36917dfe"}
14:14:26 [14:14:26.540]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.541]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71eceec6e66cca77 epochNumber=1
{"provingJobId":"1:BASE_PARITY:71eceec6e66cca77"}
14:14:26 [14:14:26.542]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXXqcNA/1 with map size 1073741824
14:14:26 [14:14:26.544]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.562]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:71eceec6e66cca77 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:71eceec6e66cca77"}
14:14:26 [14:14:26.562]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:71eceec6e66cca77 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:71eceec6e66cca77"}
14:14:26 [14:14:26.563]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:71eceec6e66cca77: Error: db error
14:14:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:71eceec6e66cca77"}
14:14:26 [14:14:26.573]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.575]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:057c50d44748def2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:057c50d44748def2"}
14:14:26 [14:14:26.576]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDi8NK7/1 with map size 1073741824
14:14:26 [14:14:26.579]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.601]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:057c50d44748def2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:057c50d44748def2"}
14:14:26 [14:14:26.602]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:057c50d44748def2 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:057c50d44748def2"}
14:14:26 [14:14:26.613]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.614]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7afe916abe8411f9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7afe916abe8411f9"}
14:14:26 [14:14:26.614]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYKosYD/1 with map size 1073741824
14:14:26 [14:14:26.617]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.634]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7afe916abe8411f9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7afe916abe8411f9"}
14:14:26 [14:14:26.634]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7afe916abe8411f9 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7afe916abe8411f9"}
14:14:26 [14:14:26.635]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:7afe916abe8411f9 jobErr=test error: Error: db error
14:14:26 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:7afe916abe8411f9"}
14:14:26 [14:14:26.640]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.640]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:6b66a8554ea262b4 not found
{"provingJobId":"1:BASE_PARITY:6b66a8554ea262b4"}
14:14:26 [14:14:26.643]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.644]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e80d5e81a7889b54 err=test error
{"provingJoId":"1:BASE_PARITY:e80d5e81a7889b54"}
14:14:26 [14:14:26.649]
INFO:
prover-client:proving-broker Proving Broker started
14:14:26 [14:14:26.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e83309ed7250b55c epochNumber=1
{"provingJobId":"1:BASE_PARITY:e83309ed7250b55c"}
14:14:26 [14:14:26.661]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqJNH65/1 with map size 1073741824
14:14:26 [14:14:26.664]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.683]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e83309ed7250b55c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e83309ed7250b55c"}
14:14:26 [14:14:26.683]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e83309ed7250b55c type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e83309ed7250b55c"}
14:14:26 [14:14:26.703]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:5f4731d9b4c518c2 epochNumber=2
{"provingJobId":"2:BASE_PARITY:5f4731d9b4c518c2"}
14:14:26 [14:14:26.705]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testqJNH65/2 with map size 1073741824
14:14:26 [14:14:26.708]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.726]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:5f4731d9b4c518c2 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:5f4731d9b4c518c2"}
14:14:26 [14:14:26.727]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:5f4731d9b4c518c2 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:5f4731d9b4c518c2"}
14:14:26 [14:14:26.792]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:b2cee4d8678e2e50 epochNumber=3
{"provingJobId":"3:BASE_PARITY:b2cee4d8678e2e50"}
14:14:26 [14:14:26.793]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testqJNH65/3 with map size 1073741824
14:14:26 [14:14:26.798]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.831]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:26 [14:14:26.832]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
14:14:26 [14:14:26.885]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:f18fe53f34c7c508 epochNumber=4
{"provingJobId":"4:BASE_PARITY:f18fe53f34c7c508"}
14:14:26 [14:14:26.888]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testqJNH65/4 with map size 1073741824
14:14:26 [14:14:26.899]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:26 [14:14:26.936]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:26 [14:14:26.937]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
14:14:26 [14:14:26.981]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:a0bbe2029457f768 epochNumber=5
{"provingJobId":"5:BASE_PARITY:a0bbe2029457f768"}
14:14:26 [14:14:26.983]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testqJNH65/5 with map size 1073741824
14:14:26 [14:14:26.991]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
14:14:27 [14:14:27.015]
INFO:
prover-client:proving-broker Cleaned up jobs=1
14:14:27 [14:14:27.016]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
14:14:27
PASS src/proving_broker/proving_broker.test.ts (
13.304 s)
14:14:27 ProvingBroker
14:14:27 Producer API
14:14:27
✓ refuses stale jobs (38 ms)
14:14:27
✓ enqueues jobs (4 ms)
14:14:27
✓ ignores duplicate jobs (3 ms)
14:14:27
✓ reports correct status when enqueuing repeat jobs (4 ms)
14:14:27
✓ throws an error in case of duplicate job IDs (41 ms)
14:14:27
✓ returns not-found status for non-existing jobs (1 ms)
14:14:27
✓ cancels jobs in queue (3 ms)
14:14:27
✓ cancels jobs in-progress (3 ms)
14:14:27
✓ returns job result if successful (3 ms)
14:14:27
✓ returns job error if failed (5 ms)
14:14:27
✓ correctly returns job status for concurrent writes (4 ms)
14:14:27
✓ refuses stale jobs (46 ms)
14:14:27
✓ enqueues jobs (39 ms)
14:14:27
✓ ignores duplicate jobs (33 ms)
14:14:27
✓ reports correct status when enqueuing repeat jobs (34 ms)
14:14:27
✓ throws an error in case of duplicate job IDs (40 ms)
14:14:27
✓ returns not-found status for non-existing jobs (5 ms)
14:14:27
✓ cancels jobs in queue (33 ms)
14:14:27
✓ cancels jobs in-progress (41 ms)
14:14:27
✓ returns job result if successful (52 ms)
14:14:27
✓ returns job error if failed (33 ms)
14:14:27
✓ correctly returns job status for concurrent writes (52 ms)
14:14:27 Consumer API
14:14:27
✓ returns undefined if no jobs are available (2 ms)
14:14:27
✓ returns jobs in priority order (2 ms)
14:14:27
✓ returns undefined if no jobs are available for the given allowList (2 ms)
14:14:27
✓ returns a job if it is in the allowList (2 ms)
14:14:27
✓ returns the most important job if it is in the allowList (3 ms)
14:14:27
✓ returns any job if filter is empty (2 ms)
14:14:27
✓ returns a new job when reporting job success (3 ms)
14:14:27
✓ returns a new job when reporting permanent error (2 ms)
14:14:27
✓ returns a new job when reporting retry-able error (3 ms)
14:14:27
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
14:14:27
✓ returns a new job if job is already in progress elsewhere (1057 ms)
14:14:27
✓ avoids sending the same job to a new agent after a restart (1004 ms)
14:14:27
✓ avoids sending a completed job to a new agent after a restart (1005 ms)
14:14:27
✓ tracks job result if in progress (3 ms)
14:14:27
✓ tracks job result even if job is in queue (2 ms)
14:14:27
✓ ignores reported job error if unknown job (2 ms)
14:14:27
✓ ignores job result if unknown job (1 ms)
14:14:27
✓ returns undefined if no jobs are available (3 ms)
14:14:27
✓ returns jobs in priority order (46 ms)
14:14:27
✓ returns undefined if no jobs are available for the given allowList (33 ms)
14:14:27
✓ returns a job if it is in the allowList (57 ms)
14:14:27
✓ returns the most important job if it is in the allowList (62 ms)
14:14:27
✓ returns any job if filter is empty (63 ms)
14:14:27
✓ returns a new job when reporting job success (34 ms)
14:14:27
✓ returns a new job when reporting permanent error (32 ms)
14:14:27
✓ returns a new job when reporting retry-able error (33 ms)
14:14:27
✓ returns a new job when reporting progress if current one is cancelled (45 ms)
14:14:27
✓ returns a new job if job is already in progress elsewhere (1105 ms)
14:14:27
✓ avoids sending the same job to a new agent after a restart (1048 ms)
14:14:27
✓ avoids sending a completed job to a new agent after a restart (1058 ms)
14:14:27
✓ tracks job result if in progress (59 ms)
14:14:27
✓ tracks job result even if job is in queue (54 ms)
14:14:27
✓ ignores reported job error if unknown job (4 ms)
14:14:27
✓ ignores job result if unknown job (4 ms)
14:14:27 Timeouts
14:14:27
✓ tracks in progress jobs (1 ms)
14:14:27
✓ re-enqueues jobs that time out (108 ms)
14:14:27
✓ cancel stale jobs that time out (235 ms)
14:14:27
✓ keeps the jobs in progress while it is alive (204 ms)
14:14:27
✓ tracks in progress jobs (42 ms)
14:14:27
✓ re-enqueues jobs that time out (136 ms)
14:14:27
✓ cancel stale jobs that time out (298 ms)
14:14:27
✓ keeps the jobs in progress while it is alive (242 ms)
14:14:27 Retries
14:14:27
✓ retries jobs (7 ms)
14:14:27
✓ retries up to a maximum number of times (4 ms)
14:14:27
✓ passing retry=false does not retry (3 ms)
14:14:27
✓ does not retry if job is stale (31 ms)
14:14:27
✓ retries jobs (29 ms)
14:14:27
✓ retries up to a maximum number of times (28 ms)
14:14:27
✓ passing retry=false does not retry (33 ms)
14:14:27
✓ does not retry if job is stale (78 ms)
14:14:27 Database management
14:14:27
✓ re-enqueues proof requests on start (3 ms)
14:14:27
✓ restores proof results on start (1 ms)
14:14:27
✓ only re-enqueues unfinished jobs (2 ms)
14:14:27
✓ saves job when enqueued (2 ms)
14:14:27
✓ does not retain job if database fails to save (2 ms)
14:14:27
✓ saves job result (3 ms)
14:14:27
✓ saves result even if database fails to save (2 ms)
14:14:27
✓ saves job error (2 ms)
14:14:27
✓ saves job error even if database fails to save (2 ms)
14:14:27
✓ does not save job result if job is unknown (2 ms)
14:14:27
✓ does not save job error if job is unknown (1 ms)
14:14:27
✓ cleans up old jobs periodically (263 ms)
14:14:27
✓ re-enqueues proof requests on start (57 ms)
14:14:27
✓ restores proof results on start (65 ms)
14:14:27
✓ only re-enqueues unfinished jobs (66 ms)
14:14:27
✓ saves job when enqueued (30 ms)
14:14:27
✓ does not retain job if database fails to save (6 ms)
14:14:27
✓ saves job result (35 ms)
14:14:27
✓ saves result even if database fails to save (29 ms)
14:14:27
✓ saves job error (44 ms)
14:14:27
✓ saves job error even if database fails to save (26 ms)
14:14:27
✓ does not save job result if job is unknown (3 ms)
14:14:27
✓ does not save job error if job is unknown (3 ms)
14:14:27
✓ cleans up old jobs periodically (418 ms)
14:14:27
14:14:27
Test Suites: 1 passed, 1 total
14:14:27
Tests: 96 passed, 96 total
14:14:27
Snapshots: 0 total
14:14:27
Time: 13.411 s
14:14:27
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
14:14:27
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?