Parent Log:
http://ci.aztec-labs.com/9b7174caf3ef97da
Command: 816a8a0e28c14627 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/7d223783d91db15002a09abc1b52d1455eb3e3da
Env: REF_NAME=gh-readonly-queue/next/pr-14161-97080f178c0a3e244c64b3c17bf255bed63fc3cf CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Mon Jun 16 10:40:26 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-14161_amd64_x3-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
10:40:30 [10:40:30.161]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.169]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:792cb1188b549fd7 epochNumber=42
{"provingJobId":"1:BASE_PARITY:792cb1188b549fd7"}
10:40:30 [10:40:30.173]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:05fbcb903be75461 epochNumber=1
{"provingJobId":"1:BASE_PARITY:05fbcb903be75461"}
10:40:30 [10:40:30.217]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.218]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b0dd4f0facf0ccce epochNumber=1
{"provingJobId":"1:BASE_PARITY:b0dd4f0facf0ccce"}
10:40:30 [10:40:30.219]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4da8872bbd390406 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4da8872bbd390406"}
10:40:30 [10:40:30.221]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.222]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b1e14fca25dfa34b epochNumber=1
{"provingJobId":"1:BASE_PARITY:b1e14fca25dfa34b"}
10:40:30 [10:40:30.224]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:b1e14fca25dfa34b epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:b1e14fca25dfa34b"}
10:40:30 [10:40:30.226]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.227]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:de6bfba5e78dde58 epochNumber=1
{"provingJobId":"1:BASE_PARITY:de6bfba5e78dde58"}
10:40:30 [10:40:30.228]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:de6bfba5e78dde58 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:de6bfba5e78dde58"}
10:40:30 [10:40:30.229]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:de6bfba5e78dde58 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:de6bfba5e78dde58"}
10:40:30 [10:40:30.230]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:de6bfba5e78dde58 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:de6bfba5e78dde58"}
10:40:30 [10:40:30.231]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.231]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7143062a8ae25110 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7143062a8ae25110"}
10:40:30 [10:40:30.253]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.255]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.256]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:87630cf87bc3bb3c epochNumber=1
{"provingJobId":"1:BASE_PARITY:87630cf87bc3bb3c"}
10:40:30 [10:40:30.257]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:87630cf87bc3bb3c
{"provingJobId":"1:BASE_PARITY:87630cf87bc3bb3c"}
10:40:30 [10:40:30.257]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:87630cf87bc3bb3c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:87630cf87bc3bb3c"}
10:40:30 [10:40:30.258]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:87630cf87bc3bb3c type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:87630cf87bc3bb3c"}
10:40:30 [10:40:30.259]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:078b061ac458b8e7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:078b061ac458b8e7"}
10:40:30 [10:40:30.261]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:078b061ac458b8e7
{"provingJobId":"1:BASE_PARITY:078b061ac458b8e7"}
10:40:30 [10:40:30.261]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:078b061ac458b8e7 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:078b061ac458b8e7"}
10:40:30 [10:40:30.263]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.263]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c44265027c54846 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4c44265027c54846"}
10:40:30 [10:40:30.264]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4c44265027c54846 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4c44265027c54846"}
10:40:30 [10:40:30.264]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4c44265027c54846 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4c44265027c54846"}
10:40:30 [10:40:30.266]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.266]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:73022c5240c13da6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:73022c5240c13da6"}
10:40:30 [10:40:30.267]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:73022c5240c13da6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:73022c5240c13da6"}
10:40:30 [10:40:30.267]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:73022c5240c13da6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:73022c5240c13da6"}
10:40:30 [10:40:30.268]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.269]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e2633cf894d4d970 epochNumber=0
{"provingJobId":"1:BASE_PARITY:e2633cf894d4d970"}
10:40:30 [10:40:30.269]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e2633cf894d4d970 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e2633cf894d4d970"}
10:40:30 [10:40:30.270]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:49711d44ce23a8eb epochNumber=0
{"provingJobId":"1:BASE_PARITY:49711d44ce23a8eb"}
10:40:30 [10:40:30.270]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:e2633cf894d4d970 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:e2633cf894d4d970"}
10:40:30 [10:40:30.271]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4b66d2e8531f3bac epochNumber=0
{"provingJobId":"1:BASE_PARITY:4b66d2e8531f3bac"}
10:40:30 [10:40:30.273]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.275]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.276]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3008fa20d895e688 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3008fa20d895e688"}
10:40:30 [10:40:30.277]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7bd74e52c8e62d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7bd74e52c8e62d3"}
10:40:30 [10:40:30.278]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.279]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f27fa3a1761c7552 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f27fa3a1761c7552"}
10:40:30 [10:40:30.280]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.281]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6d09c15c119b7955 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6d09c15c119b7955"}
10:40:30 [10:40:30.282]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:24712e69c28ee035 epochNumber=1
{"provingJobId":"1:BASE_PARITY:24712e69c28ee035"}
10:40:30 [10:40:30.282]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc1b4a30ae754afa epochNumber=2
{"provingJobId":"1:BASE_PARITY:fc1b4a30ae754afa"}
10:40:30 [10:40:30.282]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d3eeae9fd266554b epochNumber=1
{"provingJobId":"1:BASE_PARITY:d3eeae9fd266554b"}
10:40:30 [10:40:30.284]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.284]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b89b0ced2b5ad098 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b89b0ced2b5ad098"}
10:40:30 [10:40:30.285]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:abff7b4e36485bdc epochNumber=1
{"provingJobId":"1:BASE_PARITY:abff7b4e36485bdc"}
10:40:30 [10:40:30.285]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:24d2d3b92720ba35 epochNumber=2
{"provingJobId":"1:BASE_PARITY:24d2d3b92720ba35"}
10:40:30 [10:40:30.286]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9de9d4cba4e637e0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9de9d4cba4e637e0"}
10:40:30 [10:40:30.287]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.287]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf022f20a5a84351 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bf022f20a5a84351"}
10:40:30 [10:40:30.288]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dbd06fe720561c42 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dbd06fe720561c42"}
10:40:30 [10:40:30.288]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ce7998bbf50a8dc6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ce7998bbf50a8dc6"}
10:40:30 [10:40:30.288]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c3e1a1f7f3f5ac1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2c3e1a1f7f3f5ac1"}
10:40:30 [10:40:30.291]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.291]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:44465383e1c536bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:44465383e1c536bc"}
10:40:30 [10:40:30.293]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38918de794b5e715 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38918de794b5e715"}
10:40:30 [10:40:30.293]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:44465383e1c536bc type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:44465383e1c536bc"}
10:40:30 [10:40:30.294]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.295]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e642f8a5d2f82eca epochNumber=1
{"provingJobId":"1:BASE_PARITY:e642f8a5d2f82eca"}
10:40:30 [10:40:30.296]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2175ad551c4b3965 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2175ad551c4b3965"}
10:40:30 [10:40:30.296]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e642f8a5d2f82eca type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:e642f8a5d2f82eca"}
10:40:30 [10:40:30.298]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.299]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9db99913a2267b3c epochNumber=1
{"provingJobId":"1:BASE_PARITY:9db99913a2267b3c"}
10:40:30 [10:40:30.299]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:442d1c69bd8e3f9c epochNumber=1
{"provingJobId":"1:BASE_PARITY:442d1c69bd8e3f9c"}
10:40:30 [10:40:30.300]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9db99913a2267b3c type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:9db99913a2267b3c"}
10:40:30 [10:40:30.301]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.302]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c0f03f70e7764fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:4c0f03f70e7764fc"}
10:40:30 [10:40:30.312]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:4c0f03f70e7764fc
{"provingJobId":"1:BASE_PARITY:4c0f03f70e7764fc"}
10:40:30 [10:40:30.312]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4c0f03f70e7764fc type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:4c0f03f70e7764fc"}
10:40:30 [10:40:30.313]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9db599d89c5ac888 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9db599d89c5ac888"}
10:40:30 [10:40:30.314]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4c0f03f70e7764fc has already been completed
{"provingJobId":"1:BASE_PARITY:4c0f03f70e7764fc"}
10:40:30 [10:40:30.315]
INFO:
prover-client:proving-broker Proving Broker started
10:40:30 [10:40:30.315]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a05e444068212fc7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a05e444068212fc7"}
10:40:30 [10:40:30.316]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d75e70c4fa680c00 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d75e70c4fa680c00"}
10:40:31 [10:40:31.368]
INFO:
prover-client:proving-broker Proving Broker started
10:40:31 [10:40:31.369]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a05e444068212fc7 settled=false
{"provingJobId":"1:BASE_PARITY:a05e444068212fc7","status":"pending"}
10:40:31 [10:40:31.369]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d75e70c4fa680c00 settled=false
{"provingJobId":"1:BASE_PARITY:d75e70c4fa680c00","status":"pending"}
10:40:31 [10:40:31.370]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a05e444068212fc7 type=BASE_PARITY startedAt=1750070430316 older agent has taken job
{"provingJobId":"1:BASE_PARITY:a05e444068212fc7"}
10:40:31 [10:40:31.370]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a05e444068212fc7 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:a05e444068212fc7"}
10:40:31 [10:40:31.371]
INFO:
prover-client:proving-broker Proving Broker started
10:40:31 [10:40:31.372]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e39c8cdf1d09c339 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e39c8cdf1d09c339"}
10:40:31 [10:40:31.372]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d26b25682fdee8ac epochNumber=2
{"provingJobId":"1:BASE_PARITY:d26b25682fdee8ac"}
10:40:32 [10:40:32.375]
INFO:
prover-client:proving-broker Proving Broker started
10:40:32 [10:40:32.376]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e39c8cdf1d09c339 settled=false
{"provingJobId":"1:BASE_PARITY:e39c8cdf1d09c339","status":"pending"}
10:40:32 [10:40:32.376]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d26b25682fdee8ac settled=false
{"provingJobId":"1:BASE_PARITY:d26b25682fdee8ac","status":"pending"}
10:40:32 [10:40:32.377]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e39c8cdf1d09c339 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:e39c8cdf1d09c339"}
10:40:32 [10:40:32.384]
INFO:
prover-client:proving-broker Proving Broker started
10:40:32 [10:40:32.385]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dbed99a2fc55c851 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dbed99a2fc55c851"}
10:40:32 [10:40:32.386]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1a79d678544e4563 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1a79d678544e4563"}
10:40:33 [10:40:33.388]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.389]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:dbed99a2fc55c851 settled=false
{"provingJobId":"1:BASE_PARITY:dbed99a2fc55c851","status":"pending"}
10:40:33 [10:40:33.389]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1a79d678544e4563 settled=false
{"provingJobId":"1:BASE_PARITY:1a79d678544e4563","status":"pending"}
10:40:33 [10:40:33.390]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:dbed99a2fc55c851 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:dbed99a2fc55c851"}
10:40:33 [10:40:33.391]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:dbed99a2fc55c851 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:dbed99a2fc55c851"}
10:40:33 [10:40:33.392]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.393]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5e96b3387b69b1e epochNumber=1
{"provingJobId":"1:BASE_PARITY:c5e96b3387b69b1e"}
10:40:33 [10:40:33.393]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0883685c8747d740 epochNumber=2
{"provingJobId":"1:BASE_PARITY:0883685c8747d740"}
10:40:33 [10:40:33.394]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c5e96b3387b69b1e type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c5e96b3387b69b1e"}
10:40:33 [10:40:33.395]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0883685c8747d740 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0883685c8747d740"}
10:40:33 [10:40:33.397]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.398]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e79868ba66cf5e91 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e79868ba66cf5e91"}
10:40:33 [10:40:33.399]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a3db2db4eea35319 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a3db2db4eea35319"}
10:40:33 [10:40:33.399]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e79868ba66cf5e91 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e79868ba66cf5e91"}
10:40:33 [10:40:33.399]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e79868ba66cf5e91 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e79868ba66cf5e91"}
10:40:33 [10:40:33.400]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3db2db4eea35319 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a3db2db4eea35319"}
10:40:33 [10:40:33.401]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.402]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:dc632f92da0c26e9 err=test error
{"provingJoId":"1:BASE_PARITY:dc632f92da0c26e9"}
10:40:33 [10:40:33.403]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.403]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ec3897b86fafc90f not found
{"provingJobId":"1:BASE_PARITY:ec3897b86fafc90f"}
10:40:33 [10:40:33.404]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.405]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e1f43dd1dedfa13 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9e1f43dd1dedfa13"}
10:40:33 [10:40:33.406]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.407]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8ba0ff52298bd088 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8ba0ff52298bd088"}
10:40:33 [10:40:33.508]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8ba0ff52298bd088 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:8ba0ff52298bd088"}
10:40:33 [10:40:33.514]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.515]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:82e1839cfbf6c52b epochNumber=1
{"provingJobId":"1:BASE_PARITY:82e1839cfbf6c52b"}
10:40:33 [10:40:33.617]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:82e1839cfbf6c52b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:82e1839cfbf6c52b"}
10:40:33 [10:40:33.622]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b90899b33f5b491 epochNumber=10
{"provingJobId":"1:BASE_PARITY:8b90899b33f5b491"}
10:40:33 [10:40:33.643]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:33 [10:40:33.748]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.749]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a258e5d7d6adffd5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a258e5d7d6adffd5"}
10:40:33 [10:40:33.924]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a258e5d7d6adffd5 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:a258e5d7d6adffd5"}
10:40:33 [10:40:33.953]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1db8e6c19a426388 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1db8e6c19a426388"}
10:40:33 [10:40:33.954]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:1db8e6c19a426388 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:1db8e6c19a426388"}
10:40:33 [10:40:33.955]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.956]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a9be019f703720b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a9be019f703720b0"}
10:40:33 [10:40:33.956]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:a9be019f703720b0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:a9be019f703720b0"}
10:40:33 [10:40:33.957]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a9be019f703720b0 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:a9be019f703720b0"}
10:40:33 [10:40:33.957]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.958]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a911848f20f082f epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a911848f20f082f"}
10:40:33 [10:40:33.958]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0a911848f20f082f type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:0a911848f20f082f"}
10:40:33 [10:40:33.959]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.959]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:815403e444b9a3e3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:815403e444b9a3e3"}
10:40:33 [10:40:33.960]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:815403e444b9a3e3 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:815403e444b9a3e3"}
10:40:33 [10:40:33.960]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f81b7341807400f epochNumber=3
{"provingJobId":"1:BASE_PARITY:4f81b7341807400f"}
10:40:33 [10:40:33.984]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:33 [10:40:33.986]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:815403e444b9a3e3 err=test error
{"provingJoId":"1:BASE_PARITY:815403e444b9a3e3"}
10:40:33 [10:40:33.989]
INFO:
prover-client:proving-broker Proving Broker started
10:40:33 [10:40:33.989]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b85c8710853be58a settled=false
{"provingJobId":"1:BASE_PARITY:b85c8710853be58a","status":"pending"}
10:40:33 [10:40:33.990]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6131a181fed00d0c settled=false
{"provingJobId":"1:BASE_PARITY:6131a181fed00d0c","status":"pending"}
10:40:34 [10:40:34.003]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.004]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:24247ca02aa52359 settled=true
{"provingJobId":"1:BASE_PARITY:24247ca02aa52359","status":"fulfilled"}
10:40:34 [10:40:34.005]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:878d0119b7ca1976 settled=true
{"provingJobId":"2:BASE_PARITY:878d0119b7ca1976","status":"fulfilled"}
10:40:34 [10:40:34.006]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.006]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:462ab6ca241d9081 settled=true
{"provingJobId":"1:BASE_PARITY:462ab6ca241d9081","status":"fulfilled"}
10:40:34 [10:40:34.006]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d6312819d1b07003 settled=false
{"provingJobId":"1:BASE_PARITY:d6312819d1b07003","status":"pending"}
10:40:34 [10:40:34.008]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.010]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3025eeb0729e3c3f epochNumber=1
{"provingJobId":"1:BASE_PARITY:3025eeb0729e3c3f"}
10:40:34 [10:40:34.012]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.012]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:08720f75992d759e epochNumber=1
{"provingJobId":"1:BASE_PARITY:08720f75992d759e"}
10:40:34 [10:40:34.013]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:08720f75992d759e: Error: db error: Error: db error
10:40:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:08720f75992d759e"}
10:40:34 [10:40:34.014]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:181aa4474add9eb4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:181aa4474add9eb4"}
10:40:34 [10:40:34.015]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:181aa4474add9eb4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:181aa4474add9eb4"}
10:40:34 [10:40:34.015]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:181aa4474add9eb4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:181aa4474add9eb4"}
10:40:34 [10:40:34.017]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.017]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7e717d5c5c1a432f epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e717d5c5c1a432f"}
10:40:34 [10:40:34.017]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7e717d5c5c1a432f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7e717d5c5c1a432f"}
10:40:34 [10:40:34.018]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7e717d5c5c1a432f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7e717d5c5c1a432f"}
10:40:34 [10:40:34.018]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:7e717d5c5c1a432f: Error: db error
10:40:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:7e717d5c5c1a432f"}
10:40:34 [10:40:34.019]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.020]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:39f5a96552aa35ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:39f5a96552aa35ab"}
10:40:34 [10:40:34.020]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:39f5a96552aa35ab type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:39f5a96552aa35ab"}
10:40:34 [10:40:34.020]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:39f5a96552aa35ab type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:39f5a96552aa35ab"}
10:40:34 [10:40:34.021]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd1dc9c4aa1a010d epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd1dc9c4aa1a010d"}
10:40:34 [10:40:34.022]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fd1dc9c4aa1a010d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fd1dc9c4aa1a010d"}
10:40:34 [10:40:34.022]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fd1dc9c4aa1a010d type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fd1dc9c4aa1a010d"}
10:40:34 [10:40:34.023]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:fd1dc9c4aa1a010d jobErr=test error: Error: db error
10:40:34 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:fd1dc9c4aa1a010d"}
10:40:34 [10:40:34.024]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.025]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7bf1b232c99a6be1 not found
{"provingJobId":"1:BASE_PARITY:7bf1b232c99a6be1"}
10:40:34 [10:40:34.026]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.027]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:3d9166a8e803fc2d err=test error
{"provingJoId":"1:BASE_PARITY:3d9166a8e803fc2d"}
10:40:34 [10:40:34.028]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.039]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7566f5d81fc61a84 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7566f5d81fc61a84"}
10:40:34 [10:40:34.039]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7566f5d81fc61a84 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:7566f5d81fc61a84"}
10:40:34 [10:40:34.040]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7566f5d81fc61a84 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7566f5d81fc61a84"}
10:40:34 [10:40:34.049]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:d9619f1f3ee3b5b0 epochNumber=2
{"provingJobId":"2:BASE_PARITY:d9619f1f3ee3b5b0"}
10:40:34 [10:40:34.050]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:d9619f1f3ee3b5b0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:d9619f1f3ee3b5b0"}
10:40:34 [10:40:34.050]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:d9619f1f3ee3b5b0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:d9619f1f3ee3b5b0"}
10:40:34 [10:40:34.111]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:8e85cd6fa5d838fa epochNumber=3
{"provingJobId":"3:BASE_PARITY:8e85cd6fa5d838fa"}
10:40:34 [10:40:34.130]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:34 [10:40:34.173]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:2c4dab0c566690a5 epochNumber=4
{"provingJobId":"4:BASE_PARITY:2c4dab0c566690a5"}
10:40:34 [10:40:34.182]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:34 [10:40:34.235]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:301bbd6f0d2dd9ba epochNumber=5
{"provingJobId":"5:BASE_PARITY:301bbd6f0d2dd9ba"}
10:40:34 [10:40:34.258]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:34 [10:40:34.290]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.291]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1d70961da7b4ee4b epochNumber=42
{"provingJobId":"1:BASE_PARITY:1d70961da7b4ee4b"}
10:40:34 [10:40:34.292]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testa7ebwK/42 with map size 1073741824
10:40:34 [10:40:34.296]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.320]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:d2f64bc710eeaded epochNumber=1
{"provingJobId":"1:BASE_PARITY:d2f64bc710eeaded"}
10:40:34 [10:40:34.324]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.324]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:58ac36104a52be59 epochNumber=1
{"provingJobId":"1:BASE_PARITY:58ac36104a52be59"}
10:40:34 [10:40:34.325]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXVQUn1/1 with map size 1073741824
10:40:34 [10:40:34.326]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.346]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:494cd101184d0113 epochNumber=1
{"provingJobId":"1:BASE_PARITY:494cd101184d0113"}
10:40:34 [10:40:34.355]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.355]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dc0b961091c1042a epochNumber=1
{"provingJobId":"1:BASE_PARITY:dc0b961091c1042a"}
10:40:34 [10:40:34.356]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-tests7U61e/1 with map size 1073741824
10:40:34 [10:40:34.358]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.385]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:dc0b961091c1042a epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:dc0b961091c1042a"}
10:40:34 [10:40:34.388]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.389]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c8e18789d6b3d64 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6c8e18789d6b3d64"}
10:40:34 [10:40:34.389]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmQs9ms/1 with map size 1073741824
10:40:34 [10:40:34.392]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.404]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6c8e18789d6b3d64 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6c8e18789d6b3d64"}
10:40:34 [10:40:34.405]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6c8e18789d6b3d64 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6c8e18789d6b3d64"}
10:40:34 [10:40:34.408]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6c8e18789d6b3d64 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:6c8e18789d6b3d64"}
10:40:34 [10:40:34.412]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.413]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9c5c3805872ca7ea epochNumber=1
{"provingJobId":"1:BASE_PARITY:9c5c3805872ca7ea"}
10:40:34 [10:40:34.413]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbos5DO/1 with map size 1073741824
10:40:34 [10:40:34.415]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.434]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.436]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.437]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d9e0c6474d0fa776 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d9e0c6474d0fa776"}
10:40:34 [10:40:34.438]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMI6q2e/1 with map size 1073741824
10:40:34 [10:40:34.440]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.456]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d9e0c6474d0fa776
{"provingJobId":"1:BASE_PARITY:d9e0c6474d0fa776"}
10:40:34 [10:40:34.457]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d9e0c6474d0fa776 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d9e0c6474d0fa776"}
10:40:34 [10:40:34.457]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d9e0c6474d0fa776 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:d9e0c6474d0fa776"}
10:40:34 [10:40:34.470]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.471]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:617c504d0ff6373c epochNumber=1
{"provingJobId":"1:BASE_PARITY:617c504d0ff6373c"}
10:40:34 [10:40:34.472]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testNyfUGn/1 with map size 1073741824
10:40:34 [10:40:34.474]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.492]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:617c504d0ff6373c
{"provingJobId":"1:BASE_PARITY:617c504d0ff6373c"}
10:40:34 [10:40:34.493]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:617c504d0ff6373c type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:617c504d0ff6373c"}
10:40:34 [10:40:34.503]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.504]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a62473267f5771d8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a62473267f5771d8"}
10:40:34 [10:40:34.505]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVDSEe1/1 with map size 1073741824
10:40:34 [10:40:34.508]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.524]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a62473267f5771d8 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a62473267f5771d8"}
10:40:34 [10:40:34.524]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a62473267f5771d8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a62473267f5771d8"}
10:40:34 [10:40:34.532]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.533]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:16d53d94422faff7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:16d53d94422faff7"}
10:40:34 [10:40:34.534]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testbprGdU/1 with map size 1073741824
10:40:34 [10:40:34.536]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.551]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:16d53d94422faff7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:16d53d94422faff7"}
10:40:34 [10:40:34.551]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:16d53d94422faff7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:16d53d94422faff7"}
10:40:34 [10:40:34.559]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.560]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5a3d43ca8ed42ed epochNumber=0
{"provingJobId":"1:BASE_PARITY:c5a3d43ca8ed42ed"}
10:40:34 [10:40:34.560]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testF0KjH1/0 with map size 1073741824
10:40:34 [10:40:34.562]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.582]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c5a3d43ca8ed42ed epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c5a3d43ca8ed42ed"}
10:40:34 [10:40:34.583]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c4a49e0338f4252 epochNumber=0
{"provingJobId":"1:BASE_PARITY:4c4a49e0338f4252"}
10:40:34 [10:40:34.583]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:c5a3d43ca8ed42ed epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:c5a3d43ca8ed42ed"}
10:40:34 [10:40:34.584]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:279ba6c2781ee344 epochNumber=0
{"provingJobId":"1:BASE_PARITY:279ba6c2781ee344"}
10:40:34 [10:40:34.598]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.600]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.601]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cc861f47b55099fc epochNumber=2
{"provingJobId":"1:BASE_PARITY:cc861f47b55099fc"}
10:40:34 [10:40:34.602]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testFC6Xyd/2 with map size 1073741824
10:40:34 [10:40:34.604]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.619]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e71bbd8a1b2206e0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e71bbd8a1b2206e0"}
10:40:34 [10:40:34.620]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFC6Xyd/1 with map size 1073741824
10:40:34 [10:40:34.622]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.644]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.644]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3f670a29f13bd4f1 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3f670a29f13bd4f1"}
10:40:34 [10:40:34.645]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testItmd8j/1 with map size 1073741824
10:40:34 [10:40:34.647]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.668]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.669]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:081a013a7493a79b epochNumber=1
{"provingJobId":"1:BASE_PARITY:081a013a7493a79b"}
10:40:34 [10:40:34.669]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoJFsmt/1 with map size 1073741824
10:40:34 [10:40:34.672]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.689]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:417d5f3e1b5f1bea epochNumber=1
{"provingJobId":"1:BASE_PARITY:417d5f3e1b5f1bea"}
10:40:34 [10:40:34.694]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c2c1ba2a4140cdde epochNumber=2
{"provingJobId":"1:BASE_PARITY:c2c1ba2a4140cdde"}
10:40:34 [10:40:34.695]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testoJFsmt/2 with map size 1073741824
10:40:34 [10:40:34.697]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.716]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6d94f19215753bdb epochNumber=1
{"provingJobId":"1:BASE_PARITY:6d94f19215753bdb"}
10:40:34 [10:40:34.727]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.728]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf6ea37c18ad125d epochNumber=1
{"provingJobId":"1:BASE_PARITY:bf6ea37c18ad125d"}
10:40:34 [10:40:34.728]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaIqEV5/1 with map size 1073741824
10:40:34 [10:40:34.730]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.749]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2627b1f48308dd7b epochNumber=1
{"provingJobId":"1:BASE_PARITY:2627b1f48308dd7b"}
10:40:34 [10:40:34.758]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6e3961af5f12e469 epochNumber=2
{"provingJobId":"1:BASE_PARITY:6e3961af5f12e469"}
10:40:34 [10:40:34.758]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testaIqEV5/2 with map size 1073741824
10:40:34 [10:40:34.760]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.779]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d6a6f91f56ac06cc epochNumber=1
{"provingJobId":"1:BASE_PARITY:d6a6f91f56ac06cc"}
10:40:34 [10:40:34.789]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.789]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3f4b4774a2312a5a epochNumber=1
{"provingJobId":"1:BASE_PARITY:3f4b4774a2312a5a"}
10:40:34 [10:40:34.790]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhLt96F/1 with map size 1073741824
10:40:34 [10:40:34.792]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.811]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e19e95363783fdc2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e19e95363783fdc2"}
10:40:34 [10:40:34.820]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:78955d81d06f064f epochNumber=2
{"provingJobId":"1:BASE_PARITY:78955d81d06f064f"}
10:40:34 [10:40:34.821]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhLt96F/2 with map size 1073741824
10:40:34 [10:40:34.824]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.836]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51affbae52deff83 epochNumber=1
{"provingJobId":"1:BASE_PARITY:51affbae52deff83"}
10:40:34 [10:40:34.846]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.847]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7a2a3205dc5f9bda epochNumber=1
{"provingJobId":"1:BASE_PARITY:7a2a3205dc5f9bda"}
10:40:34 [10:40:34.848]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqLr5En/1 with map size 1073741824
10:40:34 [10:40:34.850]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.867]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:531d1385bc72fb0b epochNumber=1
{"provingJobId":"1:BASE_PARITY:531d1385bc72fb0b"}
10:40:34 [10:40:34.873]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7a2a3205dc5f9bda type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7a2a3205dc5f9bda"}
10:40:34 [10:40:34.886]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.886]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:827cc977c49774ed epochNumber=1
{"provingJobId":"1:BASE_PARITY:827cc977c49774ed"}
10:40:34 [10:40:34.887]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYRghfh/1 with map size 1073741824
10:40:34 [10:40:34.891]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.911]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:28d5365ba5e7cd05 epochNumber=1
{"provingJobId":"1:BASE_PARITY:28d5365ba5e7cd05"}
10:40:34 [10:40:34.918]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:827cc977c49774ed type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:827cc977c49774ed"}
10:40:34 [10:40:34.928]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.929]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d2fba453ee4ae7bc epochNumber=1
{"provingJobId":"1:BASE_PARITY:d2fba453ee4ae7bc"}
10:40:34 [10:40:34.930]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7Zksqf/1 with map size 1073741824
10:40:34 [10:40:34.935]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.954]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:edf1fedc90fdb412 epochNumber=1
{"provingJobId":"1:BASE_PARITY:edf1fedc90fdb412"}
10:40:34 [10:40:34.961]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d2fba453ee4ae7bc type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:d2fba453ee4ae7bc"}
10:40:34 [10:40:34.965]
INFO:
prover-client:proving-broker Proving Broker started
10:40:34 [10:40:34.966]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80481023482157ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:80481023482157ab"}
10:40:34 [10:40:34.967]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqrZhfo/1 with map size 1073741824
10:40:34 [10:40:34.969]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:34 [10:40:34.989]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:80481023482157ab
{"provingJobId":"1:BASE_PARITY:80481023482157ab"}
10:40:34 [10:40:34.989]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:80481023482157ab type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:80481023482157ab"}
10:40:34 [10:40:34.996]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6ccca0d665e25e3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:6ccca0d665e25e3e"}
10:40:35 [10:40:35.002]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:80481023482157ab has already been completed
{"provingJobId":"1:BASE_PARITY:80481023482157ab"}
10:40:35 [10:40:35.006]
INFO:
prover-client:proving-broker Proving Broker started
10:40:35 [10:40:35.007]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a89fb8273bc572a3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a89fb8273bc572a3"}
10:40:35 [10:40:35.008]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcbR8ZN/1 with map size 1073741824
10:40:35 [10:40:35.010]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:35 [10:40:35.027]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:43b38de6079ca64f epochNumber=2
{"provingJobId":"1:BASE_PARITY:43b38de6079ca64f"}
10:40:35 [10:40:35.028]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testcbR8ZN/2 with map size 1073741824
10:40:35 [10:40:35.029]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:36 [10:40:36.097]
INFO:
prover-client:proving-broker Proving Broker started
10:40:36 [10:40:36.104]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a89fb8273bc572a3 settled=false
{"provingJobId":"1:BASE_PARITY:a89fb8273bc572a3","status":"pending"}
10:40:36 [10:40:36.106]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:43b38de6079ca64f settled=false
{"provingJobId":"1:BASE_PARITY:43b38de6079ca64f","status":"pending"}
10:40:36 [10:40:36.107]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a89fb8273bc572a3 type=BASE_PARITY startedAt=1750070435045 older agent has taken job
{"provingJobId":"1:BASE_PARITY:a89fb8273bc572a3"}
10:40:36 [10:40:36.107]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a89fb8273bc572a3 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:a89fb8273bc572a3"}
10:40:36 [10:40:36.111]
INFO:
prover-client:proving-broker Proving Broker started
10:40:36 [10:40:36.111]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b8e807bd8b5eac09 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b8e807bd8b5eac09"}
10:40:36 [10:40:36.112]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testC2MaFC/1 with map size 1073741824
10:40:36 [10:40:36.114]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:36 [10:40:36.135]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:110ba64a58222d35 epochNumber=2
{"provingJobId":"1:BASE_PARITY:110ba64a58222d35"}
10:40:36 [10:40:36.136]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testC2MaFC/2 with map size 1073741824
10:40:36 [10:40:36.138]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:37 [10:40:37.156]
INFO:
prover-client:proving-broker Proving Broker started
10:40:37 [10:40:37.157]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b8e807bd8b5eac09 settled=false
{"provingJobId":"1:BASE_PARITY:b8e807bd8b5eac09","status":"pending"}
10:40:37 [10:40:37.159]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:110ba64a58222d35 settled=false
{"provingJobId":"1:BASE_PARITY:110ba64a58222d35","status":"pending"}
10:40:37 [10:40:37.159]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b8e807bd8b5eac09 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:b8e807bd8b5eac09"}
10:40:37 [10:40:37.162]
INFO:
prover-client:proving-broker Proving Broker started
10:40:37 [10:40:37.163]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ee7f7b548f292a06 epochNumber=1
{"provingJobId":"1:BASE_PARITY:ee7f7b548f292a06"}
10:40:37 [10:40:37.164]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPqBJWC/1 with map size 1073741824
10:40:37 [10:40:37.165]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:37 [10:40:37.181]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:491f87b82e4f6c28 epochNumber=2
{"provingJobId":"1:BASE_PARITY:491f87b82e4f6c28"}
10:40:37 [10:40:37.182]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testPqBJWC/2 with map size 1073741824
10:40:37 [10:40:37.184]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.202]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.205]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ee7f7b548f292a06 settled=false
{"provingJobId":"1:BASE_PARITY:ee7f7b548f292a06","status":"pending"}
10:40:38 [10:40:38.207]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:491f87b82e4f6c28 settled=false
{"provingJobId":"1:BASE_PARITY:491f87b82e4f6c28","status":"pending"}
10:40:38 [10:40:38.208]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ee7f7b548f292a06 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:ee7f7b548f292a06"}
10:40:38 [10:40:38.209]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ee7f7b548f292a06 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:ee7f7b548f292a06"}
10:40:38 [10:40:38.222]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.223]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:60b028502d3a3ab4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:60b028502d3a3ab4"}
10:40:38 [10:40:38.224]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testvGTwww/1 with map size 1073741824
10:40:38 [10:40:38.227]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e9a204a9343b214 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9e9a204a9343b214"}
10:40:38 [10:40:38.246]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testvGTwww/2 with map size 1073741824
10:40:38 [10:40:38.248]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.268]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:60b028502d3a3ab4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:60b028502d3a3ab4"}
10:40:38 [10:40:38.274]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9e9a204a9343b214 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9e9a204a9343b214"}
10:40:38 [10:40:38.284]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.284]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e819ef137802070a epochNumber=1
{"provingJobId":"1:BASE_PARITY:e819ef137802070a"}
10:40:38 [10:40:38.286]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb70H96/1 with map size 1073741824
10:40:38 [10:40:38.288]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.303]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91d97b512a864981 epochNumber=2
{"provingJobId":"1:BASE_PARITY:91d97b512a864981"}
10:40:38 [10:40:38.304]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testb70H96/2 with map size 1073741824
10:40:38 [10:40:38.306]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.322]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:e819ef137802070a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:e819ef137802070a"}
10:40:38 [10:40:38.322]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e819ef137802070a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:e819ef137802070a"}
10:40:38 [10:40:38.329]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:91d97b512a864981 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:91d97b512a864981"}
10:40:38 [10:40:38.338]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.340]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:33e79dcde984061a err=test error
{"provingJoId":"1:BASE_PARITY:33e79dcde984061a"}
10:40:38 [10:40:38.344]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.345]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d26176992cfe87c6 not found
{"provingJobId":"1:BASE_PARITY:d26176992cfe87c6"}
10:40:38 [10:40:38.348]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.348]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f1ffb8faaaade6a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f1ffb8faaaade6a0"}
10:40:38 [10:40:38.349]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9k60MS/1 with map size 1073741824
10:40:38 [10:40:38.352]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.373]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.374]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:173f66111dee6a84 epochNumber=1
{"provingJobId":"1:BASE_PARITY:173f66111dee6a84"}
10:40:38 [10:40:38.375]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlWac9I/1 with map size 1073741824
10:40:38 [10:40:38.376]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.501]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:173f66111dee6a84 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:173f66111dee6a84"}
10:40:38 [10:40:38.508]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.509]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d3e5584a341c277 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d3e5584a341c277"}
10:40:38 [10:40:38.510]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testu8pC3K/1 with map size 1073741824
10:40:38 [10:40:38.511]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.653]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2d3e5584a341c277 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:2d3e5584a341c277"}
10:40:38 [10:40:38.659]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8b143833cb2bccb6 epochNumber=10
{"provingJobId":"1:BASE_PARITY:8b143833cb2bccb6"}
10:40:38 [10:40:38.660]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testu8pC3K/10 with map size 1073741824
10:40:38 [10:40:38.662]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:38 [10:40:38.705]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:38 [10:40:38.706]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:40:38 [10:40:38.820]
INFO:
prover-client:proving-broker Proving Broker started
10:40:38 [10:40:38.820]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:10786c382664458c epochNumber=1
{"provingJobId":"1:BASE_PARITY:10786c382664458c"}
10:40:38 [10:40:38.821]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2TbKcR/1 with map size 1073741824
10:40:38 [10:40:38.824]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.003]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:10786c382664458c timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:10786c382664458c"}
10:40:39 [10:40:39.050]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.051]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b126a550b852bf89 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b126a550b852bf89"}
10:40:39 [10:40:39.052]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3okH83/1 with map size 1073741824
10:40:39 [10:40:39.055]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.089]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b126a550b852bf89 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:b126a550b852bf89"}
10:40:39 [10:40:39.095]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.096]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5c66376d1ec8d6ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:5c66376d1ec8d6ab"}
10:40:39 [10:40:39.098]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testldD4et/1 with map size 1073741824
10:40:39 [10:40:39.101]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.121]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:5c66376d1ec8d6ab type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:5c66376d1ec8d6ab"}
10:40:39 [10:40:39.122]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5c66376d1ec8d6ab type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:5c66376d1ec8d6ab"}
10:40:39 [10:40:39.133]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.134]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:376dcccacf7664de epochNumber=1
{"provingJobId":"1:BASE_PARITY:376dcccacf7664de"}
10:40:39 [10:40:39.137]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVGiYWF/1 with map size 1073741824
10:40:39 [10:40:39.141]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.159]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:376dcccacf7664de type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:376dcccacf7664de"}
10:40:39 [10:40:39.183]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.186]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e37e68c964126da4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e37e68c964126da4"}
10:40:39 [10:40:39.187]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testZW4nqw/1 with map size 1073741824
10:40:39 [10:40:39.191]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.213]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e37e68c964126da4 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:e37e68c964126da4"}
10:40:39 [10:40:39.224]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eff4cfa836d14c1b epochNumber=3
{"provingJobId":"1:BASE_PARITY:eff4cfa836d14c1b"}
10:40:39 [10:40:39.226]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testZW4nqw/3 with map size 1073741824
10:40:39 [10:40:39.230]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.267]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:39 [10:40:39.270]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:40:39 [10:40:39.277]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:e37e68c964126da4 err=test error
{"provingJoId":"1:BASE_PARITY:e37e68c964126da4"}
10:40:39 [10:40:39.292]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLVhWYQ/1 with map size 1073741824
10:40:39 [10:40:39.295]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.316]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLVhWYQ/2 with map size 1073741824
10:40:39 [10:40:39.320]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.341]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.343]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e3a44937f6e96aa5 settled=false
{"provingJobId":"1:BASE_PARITY:e3a44937f6e96aa5","status":"pending"}
10:40:39 [10:40:39.344]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2c43d0efb68de3b6 settled=false
{"provingJobId":"1:BASE_PARITY:2c43d0efb68de3b6","status":"pending"}
10:40:39 [10:40:39.352]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7BFurx/1 with map size 1073741824
10:40:39 [10:40:39.355]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.387]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test7BFurx/2 with map size 1073741824
10:40:39 [10:40:39.390]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.425]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.429]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:47770c2ede88d667 settled=true
{"provingJobId":"1:BASE_PARITY:47770c2ede88d667","status":"fulfilled"}
10:40:39 [10:40:39.431]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:945918157f8dad8b settled=true
{"provingJobId":"2:BASE_PARITY:945918157f8dad8b","status":"fulfilled"}
10:40:39 [10:40:39.443]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLD9iHD/1 with map size 1073741824
10:40:39 [10:40:39.447]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.480]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLD9iHD/2 with map size 1073741824
10:40:39 [10:40:39.484]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.504]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.506]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:86d74e5859de7f38 settled=true
{"provingJobId":"1:BASE_PARITY:86d74e5859de7f38","status":"fulfilled"}
10:40:39 [10:40:39.508]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d960e9820e39d35e settled=false
{"provingJobId":"1:BASE_PARITY:d960e9820e39d35e","status":"pending"}
10:40:39 [10:40:39.513]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.514]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd9fbc96cbb2f600 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd9fbc96cbb2f600"}
10:40:39 [10:40:39.515]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQsYBfE/1 with map size 1073741824
10:40:39 [10:40:39.518]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.543]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.544]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b28756b5ca4aacfb epochNumber=1
{"provingJobId":"1:BASE_PARITY:b28756b5ca4aacfb"}
10:40:39 [10:40:39.545]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:b28756b5ca4aacfb: Error: db error: Error: db error
10:40:39 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:b28756b5ca4aacfb"}
10:40:39 [10:40:39.549]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.549]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:80e0ce3f6b707e41 epochNumber=1
{"provingJobId":"1:BASE_PARITY:80e0ce3f6b707e41"}
10:40:39 [10:40:39.552]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8uf3AE/1 with map size 1073741824
10:40:39 [10:40:39.556]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.582]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:80e0ce3f6b707e41 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:80e0ce3f6b707e41"}
10:40:39 [10:40:39.583]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:80e0ce3f6b707e41 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:80e0ce3f6b707e41"}
10:40:39 [10:40:39.594]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.595]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5cd590e8c122dde7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5cd590e8c122dde7"}
10:40:39 [10:40:39.595]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWzjBHY/1 with map size 1073741824
10:40:39 [10:40:39.597]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.617]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5cd590e8c122dde7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5cd590e8c122dde7"}
10:40:39 [10:40:39.618]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5cd590e8c122dde7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5cd590e8c122dde7"}
10:40:39 [10:40:39.619]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:5cd590e8c122dde7: Error: db error
10:40:39 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:5cd590e8c122dde7"}
10:40:39 [10:40:39.624]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.625]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9cfbeb92168531d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9cfbeb92168531d5"}
10:40:39 [10:40:39.626]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testyk2fLb/1 with map size 1073741824
10:40:39 [10:40:39.630]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.647]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9cfbeb92168531d5 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9cfbeb92168531d5"}
10:40:39 [10:40:39.647]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9cfbeb92168531d5 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9cfbeb92168531d5"}
10:40:39 [10:40:39.656]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.657]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:142c3829b7a3d7fb epochNumber=1
{"provingJobId":"1:BASE_PARITY:142c3829b7a3d7fb"}
10:40:39 [10:40:39.658]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testYHUO0B/1 with map size 1073741824
10:40:39 [10:40:39.660]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.683]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:142c3829b7a3d7fb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:142c3829b7a3d7fb"}
10:40:39 [10:40:39.683]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:142c3829b7a3d7fb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:142c3829b7a3d7fb"}
10:40:39 [10:40:39.685]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:142c3829b7a3d7fb jobErr=test error: Error: db error
10:40:39 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:142c3829b7a3d7fb"}
10:40:39 [10:40:39.691]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.692]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:725f2aba72239b99 not found
{"provingJobId":"1:BASE_PARITY:725f2aba72239b99"}
10:40:39 [10:40:39.698]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.699]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:24c805ee5fb064f0 err=test error
{"provingJoId":"1:BASE_PARITY:24c805ee5fb064f0"}
10:40:39 [10:40:39.706]
INFO:
prover-client:proving-broker Proving Broker started
10:40:39 [10:40:39.718]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb17d7ef9138c28f epochNumber=1
{"provingJobId":"1:BASE_PARITY:cb17d7ef9138c28f"}
10:40:39 [10:40:39.720]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTu4SOI/1 with map size 1073741824
10:40:39 [10:40:39.724]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.749]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cb17d7ef9138c28f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cb17d7ef9138c28f"}
10:40:39 [10:40:39.750]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:cb17d7ef9138c28f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:cb17d7ef9138c28f"}
10:40:39 [10:40:39.768]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:24e1996268e4fddd epochNumber=2
{"provingJobId":"2:BASE_PARITY:24e1996268e4fddd"}
10:40:39 [10:40:39.770]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTu4SOI/2 with map size 1073741824
10:40:39 [10:40:39.775]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.800]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:24e1996268e4fddd type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:24e1996268e4fddd"}
10:40:39 [10:40:39.801]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:24e1996268e4fddd type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:24e1996268e4fddd"}
10:40:39 [10:40:39.869]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:b5c9a963637e0da3 epochNumber=3
{"provingJobId":"3:BASE_PARITY:b5c9a963637e0da3"}
10:40:39 [10:40:39.870]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testTu4SOI/3 with map size 1073741824
10:40:39 [10:40:39.876]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:39 [10:40:39.911]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:39 [10:40:39.911]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
10:40:39 [10:40:39.960]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:bf668b86725d5873 epochNumber=4
{"provingJobId":"4:BASE_PARITY:bf668b86725d5873"}
10:40:39 [10:40:39.965]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testTu4SOI/4 with map size 1073741824
10:40:39 [10:40:39.972]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:40 [10:40:40.017]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:40 [10:40:40.017]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
10:40:40 [10:40:40.053]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:f11479bc87482855 epochNumber=5
{"provingJobId":"5:BASE_PARITY:f11479bc87482855"}
10:40:40 [10:40:40.054]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testTu4SOI/5 with map size 1073741824
10:40:40 [10:40:40.056]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
10:40:40 [10:40:40.096]
INFO:
prover-client:proving-broker Cleaned up jobs=1
10:40:40 [10:40:40.097]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
10:40:40
PASS src/proving_broker/proving_broker.test.ts (
12.676 s)
10:40:40 ProvingBroker
10:40:40 Producer API
10:40:40
✓ refuses stale jobs (52 ms)
10:40:40
✓ enqueues jobs (4 ms)
10:40:40
✓ ignores duplicate jobs (4 ms)
10:40:40
✓ reports correct status when enqueuing repeat jobs (4 ms)
10:40:40
✓ throws an error in case of duplicate job IDs (21 ms)
10:40:40
✓ returns not-found status for non-existing jobs (1 ms)
10:40:40
✓ cancels jobs in queue (3 ms)
10:40:40
✓ cancels jobs in-progress (3 ms)
10:40:40
✓ returns job result if successful (3 ms)
10:40:40
✓ returns job error if failed (3 ms)
10:40:40
✓ correctly returns job status for concurrent writes (4 ms)
10:40:40
✓ refuses stale jobs (35 ms)
10:40:40
✓ enqueues jobs (30 ms)
10:40:40
✓ ignores duplicate jobs (33 ms)
10:40:40
✓ reports correct status when enqueuing repeat jobs (23 ms)
10:40:40
✓ throws an error in case of duplicate job IDs (22 ms)
10:40:40
✓ returns not-found status for non-existing jobs (3 ms)
10:40:40
✓ cancels jobs in queue (32 ms)
10:40:40
✓ cancels jobs in-progress (34 ms)
10:40:40
✓ returns job result if successful (29 ms)
10:40:40
✓ returns job error if failed (26 ms)
10:40:40
✓ correctly returns job status for concurrent writes (38 ms)
10:40:40 Consumer API
10:40:40
✓ returns undefined if no jobs are available (3 ms)
10:40:40
✓ returns jobs in priority order (3 ms)
10:40:40
✓ returns undefined if no jobs are available for the given allowList (2 ms)
10:40:40
✓ returns a job if it is in the allowList (3 ms)
10:40:40
✓ returns the most important job if it is in the allowList (3 ms)
10:40:40
✓ returns any job if filter is empty (2 ms)
10:40:40
✓ returns a new job when reporting job success (5 ms)
10:40:40
✓ returns a new job when reporting permanent error (3 ms)
10:40:40
✓ returns a new job when reporting retry-able error (3 ms)
10:40:40
✓ returns a new job when reporting progress if current one is cancelled (13 ms)
10:40:40
✓ returns a new job if job is already in progress elsewhere (1057 ms)
10:40:40
✓ avoids sending the same job to a new agent after a restart (1012 ms)
10:40:40
✓ avoids sending a completed job to a new agent after a restart (1009 ms)
10:40:40
✓ tracks job result if in progress (4 ms)
10:40:40
✓ tracks job result even if job is in queue (3 ms)
10:40:40
✓ ignores reported job error if unknown job (1 ms)
10:40:40
✓ ignores job result if unknown job (2 ms)
10:40:40
✓ returns undefined if no jobs are available (3 ms)
10:40:40
✓ returns jobs in priority order (40 ms)
10:40:40
✓ returns undefined if no jobs are available for the given allowList (26 ms)
10:40:40
✓ returns a job if it is in the allowList (58 ms)
10:40:40
✓ returns the most important job if it is in the allowList (62 ms)
10:40:40
✓ returns any job if filter is empty (55 ms)
10:40:40
✓ returns a new job when reporting job success (40 ms)
10:40:40
✓ returns a new job when reporting permanent error (42 ms)
10:40:40
✓ returns a new job when reporting retry-able error (36 ms)
10:40:40
✓ returns a new job when reporting progress if current one is cancelled (40 ms)
10:40:40
✓ returns a new job if job is already in progress elsewhere (1104 ms)
10:40:40
✓ avoids sending the same job to a new agent after a restart (1052 ms)
10:40:40
✓ avoids sending a completed job to a new agent after a restart (1056 ms)
10:40:40
✓ tracks job result if in progress (64 ms)
10:40:40
✓ tracks job result even if job is in queue (55 ms)
10:40:40
✓ ignores reported job error if unknown job (4 ms)
10:40:40
✓ ignores job result if unknown job (5 ms)
10:40:40 Timeouts
10:40:40
✓ tracks in progress jobs (2 ms)
10:40:40
✓ re-enqueues jobs that time out (108 ms)
10:40:40
✓ cancel stale jobs that time out (234 ms)
10:40:40
✓ keeps the jobs in progress while it is alive (204 ms)
10:40:40
✓ tracks in progress jobs (24 ms)
10:40:40
✓ re-enqueues jobs that time out (135 ms)
10:40:40
✓ cancel stale jobs that time out (311 ms)
10:40:40
✓ keeps the jobs in progress while it is alive (230 ms)
10:40:40 Retries
10:40:40
✓ retries jobs (2 ms)
10:40:40
✓ retries up to a maximum number of times (2 ms)
10:40:40
✓ passing retry=false does not retry (2 ms)
10:40:40
✓ does not retry if job is stale (29 ms)
10:40:40
✓ retries jobs (44 ms)
10:40:40
✓ retries up to a maximum number of times (38 ms)
10:40:40
✓ passing retry=false does not retry (42 ms)
10:40:40
✓ does not retry if job is stale (112 ms)
10:40:40 Database management
10:40:40
✓ re-enqueues proof requests on start (12 ms)
10:40:40
✓ restores proof results on start (4 ms)
10:40:40
✓ only re-enqueues unfinished jobs (2 ms)
10:40:40
✓ saves job when enqueued (4 ms)
10:40:40
✓ does not retain job if database fails to save (3 ms)
10:40:40
✓ saves job result (2 ms)
10:40:40
✓ saves result even if database fails to save (3 ms)
10:40:40
✓ saves job error (2 ms)
10:40:40
✓ saves job error even if database fails to save (3 ms)
10:40:40
✓ does not save job result if job is unknown (2 ms)
10:40:40
✓ does not save job error if job is unknown (1 ms)
10:40:40
✓ cleans up old jobs periodically (259 ms)
10:40:40
✓ re-enqueues proof requests on start (60 ms)
10:40:40
✓ restores proof results on start (88 ms)
10:40:40
✓ only re-enqueues unfinished jobs (74 ms)
10:40:40
✓ saves job when enqueued (28 ms)
10:40:40
✓ does not retain job if database fails to save (7 ms)
10:40:40
✓ saves job result (45 ms)
10:40:40
✓ saves result even if database fails to save (29 ms)
10:40:40
✓ saves job error (33 ms)
10:40:40
✓ saves job error even if database fails to save (33 ms)
10:40:40
✓ does not save job result if job is unknown (7 ms)
10:40:40
✓ does not save job error if job is unknown (6 ms)
10:40:40
✓ cleans up old jobs periodically (432 ms)
10:40:40
10:40:40
Test Suites: 1 passed, 1 total
10:40:40
Tests: 96 passed, 96 total
10:40:40
Snapshots: 0 total
10:40:40
Time: 12.763 s
10:40:40
Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i
.
10:40:40
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?