Parent Log:
http://ci.aztec-labs.com/1e5f68eab57c502e
Command: 3f1d1d87da1584d1 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/aac426c9418162aeee8162ed45014b9393d605b8
Env: REF_NAME=gh-readonly-queue/next/pr-15051-92d685c3a802eb31897b6f13d28b9ee428c73744 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Fri Jun 20 06:35:47 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15051_amd64_x3-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
06:35:52 [06:35:52.123]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.129]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:17ca2991730e9c82 epochNumber=42
{"provingJobId":"1:BASE_PARITY:17ca2991730e9c82"}
06:35:52 [06:35:52.133]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:d7ea0ac164764b9e epochNumber=1
{"provingJobId":"1:BASE_PARITY:d7ea0ac164764b9e"}
06:35:52 [06:35:52.171]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.172]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68248f9be7d79b18 epochNumber=1
{"provingJobId":"1:BASE_PARITY:68248f9be7d79b18"}
06:35:52 [06:35:52.173]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:71e528c8bea2b9d5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:71e528c8bea2b9d5"}
06:35:52 [06:35:52.175]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.176]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5da82e1a885168ca epochNumber=1
{"provingJobId":"1:BASE_PARITY:5da82e1a885168ca"}
06:35:52 [06:35:52.177]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:5da82e1a885168ca epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:5da82e1a885168ca"}
06:35:52 [06:35:52.179]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.180]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54654add253caa28 epochNumber=1
{"provingJobId":"1:BASE_PARITY:54654add253caa28"}
06:35:52 [06:35:52.181]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:54654add253caa28 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:54654add253caa28"}
06:35:52 [06:35:52.182]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:54654add253caa28 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:54654add253caa28"}
06:35:52 [06:35:52.183]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:54654add253caa28 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:54654add253caa28"}
06:35:52 [06:35:52.184]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.185]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:959a8d83d6cf9b14 epochNumber=1
{"provingJobId":"1:BASE_PARITY:959a8d83d6cf9b14"}
06:35:52 [06:35:52.218]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.220]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.221]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cbf19c1293f90b53 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cbf19c1293f90b53"}
06:35:52 [06:35:52.221]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cbf19c1293f90b53
{"provingJobId":"1:BASE_PARITY:cbf19c1293f90b53"}
06:35:52 [06:35:52.222]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cbf19c1293f90b53 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cbf19c1293f90b53"}
06:35:52 [06:35:52.222]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cbf19c1293f90b53 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:cbf19c1293f90b53"}
06:35:52 [06:35:52.224]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.225]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e0216d6baccbc8e5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e0216d6baccbc8e5"}
06:35:52 [06:35:52.225]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e0216d6baccbc8e5
{"provingJobId":"1:BASE_PARITY:e0216d6baccbc8e5"}
06:35:52 [06:35:52.226]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e0216d6baccbc8e5 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:e0216d6baccbc8e5"}
06:35:52 [06:35:52.227]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.228]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c4196fd4de25c05d epochNumber=1
{"provingJobId":"1:BASE_PARITY:c4196fd4de25c05d"}
06:35:52 [06:35:52.228]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c4196fd4de25c05d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c4196fd4de25c05d"}
06:35:52 [06:35:52.229]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c4196fd4de25c05d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:c4196fd4de25c05d"}
06:35:52 [06:35:52.230]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.231]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:53b269439e2540a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:53b269439e2540a0"}
06:35:52 [06:35:52.231]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:53b269439e2540a0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:53b269439e2540a0"}
06:35:52 [06:35:52.231]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:53b269439e2540a0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:53b269439e2540a0"}
06:35:52 [06:35:52.233]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.233]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:eeeddcaf03e2ad59 epochNumber=0
{"provingJobId":"1:BASE_PARITY:eeeddcaf03e2ad59"}
06:35:52 [06:35:52.233]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:eeeddcaf03e2ad59 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:eeeddcaf03e2ad59"}
06:35:52 [06:35:52.234]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c04936bf8b21f766 epochNumber=0
{"provingJobId":"1:BASE_PARITY:c04936bf8b21f766"}
06:35:52 [06:35:52.234]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:eeeddcaf03e2ad59 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:eeeddcaf03e2ad59"}
06:35:52 [06:35:52.234]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1887289e1f05fb30 epochNumber=0
{"provingJobId":"1:BASE_PARITY:1887289e1f05fb30"}
06:35:52 [06:35:52.235]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.237]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.237]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9190dee98cf8b31d epochNumber=2
{"provingJobId":"1:BASE_PARITY:9190dee98cf8b31d"}
06:35:52 [06:35:52.238]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b706f4367079711f epochNumber=1
{"provingJobId":"1:BASE_PARITY:b706f4367079711f"}
06:35:52 [06:35:52.239]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.239]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f0689867d355c872 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f0689867d355c872"}
06:35:52 [06:35:52.240]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.241]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7f7050d431ffaf2a epochNumber=1
{"provingJobId":"1:BASE_PARITY:7f7050d431ffaf2a"}
06:35:52 [06:35:52.241]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:369249c7f26ef8a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:369249c7f26ef8a0"}
06:35:52 [06:35:52.242]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:42f68ec33190ca3a epochNumber=2
{"provingJobId":"1:BASE_PARITY:42f68ec33190ca3a"}
06:35:52 [06:35:52.242]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9667c50f8ec83412 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9667c50f8ec83412"}
06:35:52 [06:35:52.243]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.243]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91583b30460d3833 epochNumber=1
{"provingJobId":"1:BASE_PARITY:91583b30460d3833"}
06:35:52 [06:35:52.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b002ee4d90323cd3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b002ee4d90323cd3"}
06:35:52 [06:35:52.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5117601d1c5ee6d6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:5117601d1c5ee6d6"}
06:35:52 [06:35:52.244]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f48c3dc1a48b297a epochNumber=1
{"provingJobId":"1:BASE_PARITY:f48c3dc1a48b297a"}
06:35:52 [06:35:52.246]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e731b70faf6a264a epochNumber=1
{"provingJobId":"1:BASE_PARITY:e731b70faf6a264a"}
06:35:52 [06:35:52.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc9aa90ba7c2c2be epochNumber=1
{"provingJobId":"1:BASE_PARITY:bc9aa90ba7c2c2be"}
06:35:52 [06:35:52.247]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7304d23828b2230c epochNumber=2
{"provingJobId":"1:BASE_PARITY:7304d23828b2230c"}
06:35:52 [06:35:52.247]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7688e98a3990190 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7688e98a3990190"}
06:35:52 [06:35:52.248]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c67783cab732d40 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7c67783cab732d40"}
06:35:52 [06:35:52.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bd407b9fd40205b2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:bd407b9fd40205b2"}
06:35:52 [06:35:52.250]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7c67783cab732d40 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:7c67783cab732d40"}
06:35:52 [06:35:52.251]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.251]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d200ed0f045474c2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d200ed0f045474c2"}
06:35:52 [06:35:52.252]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:42377150049b0d86 epochNumber=1
{"provingJobId":"1:BASE_PARITY:42377150049b0d86"}
06:35:52 [06:35:52.252]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d200ed0f045474c2 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:d200ed0f045474c2"}
06:35:52 [06:35:52.254]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.254]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c5e1518a0ef61da5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c5e1518a0ef61da5"}
06:35:52 [06:35:52.255]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:97b68efa84483210 epochNumber=1
{"provingJobId":"1:BASE_PARITY:97b68efa84483210"}
06:35:52 [06:35:52.255]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c5e1518a0ef61da5 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:c5e1518a0ef61da5"}
06:35:52 [06:35:52.256]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.257]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:61f885282d4bc6b4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:61f885282d4bc6b4"}
06:35:52 [06:35:52.257]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:61f885282d4bc6b4
{"provingJobId":"1:BASE_PARITY:61f885282d4bc6b4"}
06:35:52 [06:35:52.258]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:61f885282d4bc6b4 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:61f885282d4bc6b4"}
06:35:52 [06:35:52.258]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:272dabe62a635098 epochNumber=1
{"provingJobId":"1:BASE_PARITY:272dabe62a635098"}
06:35:52 [06:35:52.258]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:61f885282d4bc6b4 has already been completed
{"provingJobId":"1:BASE_PARITY:61f885282d4bc6b4"}
06:35:52 [06:35:52.260]
INFO:
prover-client:proving-broker Proving Broker started
06:35:52 [06:35:52.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:05d38bd40b451cf2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:05d38bd40b451cf2"}
06:35:52 [06:35:52.260]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:29df392a1e15828e epochNumber=2
{"provingJobId":"1:BASE_PARITY:29df392a1e15828e"}
06:35:53 [06:35:53.313]
INFO:
prover-client:proving-broker Proving Broker started
06:35:53 [06:35:53.314]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:05d38bd40b451cf2 settled=false
{"provingJobId":"1:BASE_PARITY:05d38bd40b451cf2","status":"pending"}
06:35:53 [06:35:53.314]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:29df392a1e15828e settled=false
{"provingJobId":"1:BASE_PARITY:29df392a1e15828e","status":"pending"}
06:35:53 [06:35:53.315]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:05d38bd40b451cf2 type=BASE_PARITY startedAt=1750401352261 older agent has taken job
{"provingJobId":"1:BASE_PARITY:05d38bd40b451cf2"}
06:35:53 [06:35:53.315]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:05d38bd40b451cf2 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:05d38bd40b451cf2"}
06:35:53 [06:35:53.317]
INFO:
prover-client:proving-broker Proving Broker started
06:35:53 [06:35:53.318]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:81a304a2602861ad epochNumber=1
{"provingJobId":"1:BASE_PARITY:81a304a2602861ad"}
06:35:53 [06:35:53.318]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ec3ca06636e85d78 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ec3ca06636e85d78"}
06:35:54 [06:35:54.318]
INFO:
prover-client:proving-broker Proving Broker started
06:35:54 [06:35:54.319]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:81a304a2602861ad settled=false
{"provingJobId":"1:BASE_PARITY:81a304a2602861ad","status":"pending"}
06:35:54 [06:35:54.319]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ec3ca06636e85d78 settled=false
{"provingJobId":"1:BASE_PARITY:ec3ca06636e85d78","status":"pending"}
06:35:54 [06:35:54.319]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:81a304a2602861ad type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:81a304a2602861ad"}
06:35:54 [06:35:54.321]
INFO:
prover-client:proving-broker Proving Broker started
06:35:54 [06:35:54.322]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d024747f3d4e0a04 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d024747f3d4e0a04"}
06:35:54 [06:35:54.322]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68a46029ae2a5356 epochNumber=2
{"provingJobId":"1:BASE_PARITY:68a46029ae2a5356"}
06:35:55 [06:35:55.322]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.323]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d024747f3d4e0a04 settled=false
{"provingJobId":"1:BASE_PARITY:d024747f3d4e0a04","status":"pending"}
06:35:55 [06:35:55.323]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:68a46029ae2a5356 settled=false
{"provingJobId":"1:BASE_PARITY:68a46029ae2a5356","status":"pending"}
06:35:55 [06:35:55.324]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d024747f3d4e0a04 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d024747f3d4e0a04"}
06:35:55 [06:35:55.324]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d024747f3d4e0a04 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d024747f3d4e0a04"}
06:35:55 [06:35:55.326]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.326]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b05cf3226472569a epochNumber=1
{"provingJobId":"1:BASE_PARITY:b05cf3226472569a"}
06:35:55 [06:35:55.327]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:23dd908bb0423480 epochNumber=2
{"provingJobId":"1:BASE_PARITY:23dd908bb0423480"}
06:35:55 [06:35:55.327]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:b05cf3226472569a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:b05cf3226472569a"}
06:35:55 [06:35:55.328]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:23dd908bb0423480 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:23dd908bb0423480"}
06:35:55 [06:35:55.329]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.329]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:356938fc795425e4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:356938fc795425e4"}
06:35:55 [06:35:55.329]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6f8d1ef02c7fcfee epochNumber=2
{"provingJobId":"1:BASE_PARITY:6f8d1ef02c7fcfee"}
06:35:55 [06:35:55.330]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:356938fc795425e4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:356938fc795425e4"}
06:35:55 [06:35:55.330]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:356938fc795425e4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:356938fc795425e4"}
06:35:55 [06:35:55.330]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6f8d1ef02c7fcfee type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6f8d1ef02c7fcfee"}
06:35:55 [06:35:55.331]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.332]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c1a0e34d0bcd01fb err=test error
{"provingJoId":"1:BASE_PARITY:c1a0e34d0bcd01fb"}
06:35:55 [06:35:55.333]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.333]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7be46891edda21b5 not found
{"provingJobId":"1:BASE_PARITY:7be46891edda21b5"}
06:35:55 [06:35:55.335]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.335]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5d7b9be88aa4935d epochNumber=1
{"provingJobId":"1:BASE_PARITY:5d7b9be88aa4935d"}
06:35:55 [06:35:55.336]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.336]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b59af010fdbdfdf6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b59af010fdbdfdf6"}
06:35:55 [06:35:55.437]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:b59af010fdbdfdf6 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:b59af010fdbdfdf6"}
06:35:55 [06:35:55.445]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:011368fe70517d0e epochNumber=1
{"provingJobId":"1:BASE_PARITY:011368fe70517d0e"}
06:35:55 [06:35:55.547]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:011368fe70517d0e timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:011368fe70517d0e"}
06:35:55 [06:35:55.554]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bc5232e93d536d9e epochNumber=10
{"provingJobId":"1:BASE_PARITY:bc5232e93d536d9e"}
06:35:55 [06:35:55.572]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:35:55 [06:35:55.681]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.682]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5436b322ce39291d epochNumber=1
{"provingJobId":"1:BASE_PARITY:5436b322ce39291d"}
06:35:55 [06:35:55.835]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5436b322ce39291d timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:5436b322ce39291d"}
06:35:55 [06:35:55.886]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.887]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6d97c3741b024fc3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6d97c3741b024fc3"}
06:35:55 [06:35:55.888]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6d97c3741b024fc3 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:6d97c3741b024fc3"}
06:35:55 [06:35:55.889]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.889]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6a445afd92b50f8e epochNumber=1
{"provingJobId":"1:BASE_PARITY:6a445afd92b50f8e"}
06:35:55 [06:35:55.890]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6a445afd92b50f8e type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:6a445afd92b50f8e"}
06:35:55 [06:35:55.890]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6a445afd92b50f8e type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:6a445afd92b50f8e"}
06:35:55 [06:35:55.891]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.891]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e94432b4d406e7bb epochNumber=1
{"provingJobId":"1:BASE_PARITY:e94432b4d406e7bb"}
06:35:55 [06:35:55.892]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e94432b4d406e7bb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:e94432b4d406e7bb"}
06:35:55 [06:35:55.893]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.894]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8e72aaf45c9786a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8e72aaf45c9786a0"}
06:35:55 [06:35:55.894]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8e72aaf45c9786a0 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:8e72aaf45c9786a0"}
06:35:55 [06:35:55.895]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4c6af3ce3feb35d9 epochNumber=3
{"provingJobId":"1:BASE_PARITY:4c6af3ce3feb35d9"}
06:35:55 [06:35:55.919]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:35:55 [06:35:55.920]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8e72aaf45c9786a0 err=test error
{"provingJoId":"1:BASE_PARITY:8e72aaf45c9786a0"}
06:35:55 [06:35:55.923]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.923]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7ba181fc5d0e5f0f settled=false
{"provingJobId":"1:BASE_PARITY:7ba181fc5d0e5f0f","status":"pending"}
06:35:55 [06:35:55.923]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c5a151e1c6779644 settled=false
{"provingJobId":"1:BASE_PARITY:c5a151e1c6779644","status":"pending"}
06:35:55 [06:35:55.926]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.927]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:58ada7bbbf45471b settled=true
{"provingJobId":"1:BASE_PARITY:58ada7bbbf45471b","status":"fulfilled"}
06:35:55 [06:35:55.927]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:0ec4c960a16a2009 settled=true
{"provingJobId":"2:BASE_PARITY:0ec4c960a16a2009","status":"fulfilled"}
06:35:55 [06:35:55.928]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.929]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d07850604ec5c8e8 settled=true
{"provingJobId":"1:BASE_PARITY:d07850604ec5c8e8","status":"fulfilled"}
06:35:55 [06:35:55.929]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:29a389b890ed28b7 settled=false
{"provingJobId":"1:BASE_PARITY:29a389b890ed28b7","status":"pending"}
06:35:55 [06:35:55.930]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.932]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:44ad710d88906e71 epochNumber=1
{"provingJobId":"1:BASE_PARITY:44ad710d88906e71"}
06:35:55 [06:35:55.934]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:813ccd83aeab9d56 epochNumber=1
{"provingJobId":"1:BASE_PARITY:813ccd83aeab9d56"}
06:35:55 [06:35:55.935]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:813ccd83aeab9d56: Error: db error: Error: db error
06:35:55 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:813ccd83aeab9d56"}
06:35:55 [06:35:55.937]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.938]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db2910500aa6877d epochNumber=1
{"provingJobId":"1:BASE_PARITY:db2910500aa6877d"}
06:35:55 [06:35:55.938]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:db2910500aa6877d type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:db2910500aa6877d"}
06:35:55 [06:35:55.938]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:db2910500aa6877d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:db2910500aa6877d"}
06:35:55 [06:35:55.940]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:07ac36fb76ec9d43 epochNumber=1
{"provingJobId":"1:BASE_PARITY:07ac36fb76ec9d43"}
06:35:55 [06:35:55.941]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:07ac36fb76ec9d43 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:07ac36fb76ec9d43"}
06:35:55 [06:35:55.941]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:07ac36fb76ec9d43 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:07ac36fb76ec9d43"}
06:35:55 [06:35:55.942]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:07ac36fb76ec9d43: Error: db error
06:35:55 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:07ac36fb76ec9d43"}
06:35:55 [06:35:55.943]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.944]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2d606c7d6c905f3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:2d606c7d6c905f3e"}
06:35:55 [06:35:55.944]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2d606c7d6c905f3e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2d606c7d6c905f3e"}
06:35:55 [06:35:55.945]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2d606c7d6c905f3e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:2d606c7d6c905f3e"}
06:35:55 [06:35:55.946]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.947]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:69a3300c872ff37e epochNumber=1
{"provingJobId":"1:BASE_PARITY:69a3300c872ff37e"}
06:35:55 [06:35:55.947]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:69a3300c872ff37e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:69a3300c872ff37e"}
06:35:55 [06:35:55.947]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:69a3300c872ff37e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:69a3300c872ff37e"}
06:35:55 [06:35:55.948]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:69a3300c872ff37e jobErr=test error: Error: db error
06:35:55 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:69a3300c872ff37e"}
06:35:55 [06:35:55.949]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.950]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c30317531aba1028 not found
{"provingJobId":"1:BASE_PARITY:c30317531aba1028"}
06:35:55 [06:35:55.951]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.952]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:929ceebd173bb3e7 err=test error
{"provingJoId":"1:BASE_PARITY:929ceebd173bb3e7"}
06:35:55 [06:35:55.953]
INFO:
prover-client:proving-broker Proving Broker started
06:35:55 [06:35:55.964]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4231415db7a208e4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4231415db7a208e4"}
06:35:55 [06:35:55.964]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4231415db7a208e4 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4231415db7a208e4"}
06:35:55 [06:35:55.965]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4231415db7a208e4 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4231415db7a208e4"}
06:35:55 [06:35:55.976]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:0bf529fa010862ea epochNumber=2
{"provingJobId":"2:BASE_PARITY:0bf529fa010862ea"}
06:35:55 [06:35:55.976]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:0bf529fa010862ea type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:0bf529fa010862ea"}
06:35:55 [06:35:55.977]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:0bf529fa010862ea type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:0bf529fa010862ea"}
06:35:56 [06:35:56.038]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:4ebe25862bf55ece epochNumber=3
{"provingJobId":"3:BASE_PARITY:4ebe25862bf55ece"}
06:35:56 [06:35:56.055]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:35:56 [06:35:56.099]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:c91adc7cba2e674b epochNumber=4
{"provingJobId":"4:BASE_PARITY:c91adc7cba2e674b"}
06:35:56 [06:35:56.107]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:35:56 [06:35:56.161]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:6de5e7eb0ae27df0 epochNumber=5
{"provingJobId":"5:BASE_PARITY:6de5e7eb0ae27df0"}
06:35:56 [06:35:56.184]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:35:56 [06:35:56.215]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.216]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:91b56e6d6d87c675 epochNumber=42
{"provingJobId":"1:BASE_PARITY:91b56e6d6d87c675"}
06:35:56 [06:35:56.218]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testXdHdee/42 with map size 1073741824
06:35:56 [06:35:56.222]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.249]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:09105a783ce79e89 epochNumber=1
{"provingJobId":"1:BASE_PARITY:09105a783ce79e89"}
06:35:56 [06:35:56.255]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.255]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3327c882723dc642 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3327c882723dc642"}
06:35:56 [06:35:56.256]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnuRcOQ/1 with map size 1073741824
06:35:56 [06:35:56.258]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.277]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:059bee850a2e8a18 epochNumber=1
{"provingJobId":"1:BASE_PARITY:059bee850a2e8a18"}
06:35:56 [06:35:56.287]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.288]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d8689f71832d51b8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d8689f71832d51b8"}
06:35:56 [06:35:56.289]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw1D6EJ/1 with map size 1073741824
06:35:56 [06:35:56.291]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.310]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d8689f71832d51b8 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d8689f71832d51b8"}
06:35:56 [06:35:56.314]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.314]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4211ae64a554dcc0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4211ae64a554dcc0"}
06:35:56 [06:35:56.315]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8J4SYp/1 with map size 1073741824
06:35:56 [06:35:56.317]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.337]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4211ae64a554dcc0 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:4211ae64a554dcc0"}
06:35:56 [06:35:56.338]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4211ae64a554dcc0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4211ae64a554dcc0"}
06:35:56 [06:35:56.345]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:4211ae64a554dcc0 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:4211ae64a554dcc0"}
06:35:56 [06:35:56.348]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.349]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:922f35a8d108c13e epochNumber=1
{"provingJobId":"1:BASE_PARITY:922f35a8d108c13e"}
06:35:56 [06:35:56.350]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTP4dIL/1 with map size 1073741824
06:35:56 [06:35:56.352]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.374]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.377]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.378]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cbc394634e7d0353 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cbc394634e7d0353"}
06:35:56 [06:35:56.378]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmTvXFf/1 with map size 1073741824
06:35:56 [06:35:56.380]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.398]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:cbc394634e7d0353
{"provingJobId":"1:BASE_PARITY:cbc394634e7d0353"}
06:35:56 [06:35:56.398]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:cbc394634e7d0353 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:cbc394634e7d0353"}
06:35:56 [06:35:56.398]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cbc394634e7d0353 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:cbc394634e7d0353"}
06:35:56 [06:35:56.406]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.407]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5d98756466134254 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5d98756466134254"}
06:35:56 [06:35:56.408]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testwC2sj8/1 with map size 1073741824
06:35:56 [06:35:56.410]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.427]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5d98756466134254
{"provingJobId":"1:BASE_PARITY:5d98756466134254"}
06:35:56 [06:35:56.427]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5d98756466134254 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:5d98756466134254"}
06:35:56 [06:35:56.437]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.438]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01d39ba9ae54eca0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:01d39ba9ae54eca0"}
06:35:56 [06:35:56.439]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPyPdyl/1 with map size 1073741824
06:35:56 [06:35:56.441]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.458]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:01d39ba9ae54eca0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:01d39ba9ae54eca0"}
06:35:56 [06:35:56.458]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:01d39ba9ae54eca0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:01d39ba9ae54eca0"}
06:35:56 [06:35:56.468]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.468]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d5517ea6aa24e5b0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d5517ea6aa24e5b0"}
06:35:56 [06:35:56.469]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgkwmxj/1 with map size 1073741824
06:35:56 [06:35:56.473]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.492]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d5517ea6aa24e5b0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d5517ea6aa24e5b0"}
06:35:56 [06:35:56.492]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d5517ea6aa24e5b0 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:d5517ea6aa24e5b0"}
06:35:56 [06:35:56.502]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.503]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aad4f99cb8dbd023 epochNumber=0
{"provingJobId":"1:BASE_PARITY:aad4f99cb8dbd023"}
06:35:56 [06:35:56.504]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testlYSQPH/0 with map size 1073741824
06:35:56 [06:35:56.507]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.524]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:aad4f99cb8dbd023 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:aad4f99cb8dbd023"}
06:35:56 [06:35:56.525]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:607cb5b6ff0b4624 epochNumber=0
{"provingJobId":"1:BASE_PARITY:607cb5b6ff0b4624"}
06:35:56 [06:35:56.526]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:aad4f99cb8dbd023 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:aad4f99cb8dbd023"}
06:35:56 [06:35:56.526]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:85d6b0a48784431a epochNumber=0
{"provingJobId":"1:BASE_PARITY:85d6b0a48784431a"}
06:35:56 [06:35:56.541]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.547]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.547]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3e53d641d9575644 epochNumber=2
{"provingJobId":"1:BASE_PARITY:3e53d641d9575644"}
06:35:56 [06:35:56.548]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpbktQN/2 with map size 1073741824
06:35:56 [06:35:56.549]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.578]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c2ccd0b5bb92a42 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7c2ccd0b5bb92a42"}
06:35:56 [06:35:56.579]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpbktQN/1 with map size 1073741824
06:35:56 [06:35:56.580]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.604]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.605]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:160c4bf48362f583 epochNumber=1
{"provingJobId":"1:BASE_PARITY:160c4bf48362f583"}
06:35:56 [06:35:56.606]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw60FAV/1 with map size 1073741824
06:35:56 [06:35:56.608]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.629]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.630]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4edfd9e55345bb28 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4edfd9e55345bb28"}
06:35:56 [06:35:56.632]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testECFOMp/1 with map size 1073741824
06:35:56 [06:35:56.634]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.653]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e4c522f1a213d6d0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e4c522f1a213d6d0"}
06:35:56 [06:35:56.661]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:502522ec40a97fa0 epochNumber=2
{"provingJobId":"1:BASE_PARITY:502522ec40a97fa0"}
06:35:56 [06:35:56.662]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testECFOMp/2 with map size 1073741824
06:35:56 [06:35:56.664]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.682]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb650d40e3086203 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cb650d40e3086203"}
06:35:56 [06:35:56.691]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.692]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6fa97bf8043492a9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6fa97bf8043492a9"}
06:35:56 [06:35:56.692]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5QPakE/1 with map size 1073741824
06:35:56 [06:35:56.694]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.712]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7cb4c58b66be3105 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7cb4c58b66be3105"}
06:35:56 [06:35:56.720]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cf41ac54233346a6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:cf41ac54233346a6"}
06:35:56 [06:35:56.721]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5QPakE/2 with map size 1073741824
06:35:56 [06:35:56.722]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.739]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c690e8388f37a3a epochNumber=1
{"provingJobId":"1:BASE_PARITY:1c690e8388f37a3a"}
06:35:56 [06:35:56.750]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.751]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:989c33d6bcab5b8f epochNumber=1
{"provingJobId":"1:BASE_PARITY:989c33d6bcab5b8f"}
06:35:56 [06:35:56.752]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnWHUX4/1 with map size 1073741824
06:35:56 [06:35:56.754]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.772]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:14d13af871b2692f epochNumber=1
{"provingJobId":"1:BASE_PARITY:14d13af871b2692f"}
06:35:56 [06:35:56.778]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:14046d9bf02ac46c epochNumber=2
{"provingJobId":"1:BASE_PARITY:14046d9bf02ac46c"}
06:35:56 [06:35:56.779]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testnWHUX4/2 with map size 1073741824
06:35:56 [06:35:56.781]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.798]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7be45353cb65eac3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7be45353cb65eac3"}
06:35:56 [06:35:56.808]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.809]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6debb83d0b38302d epochNumber=1
{"provingJobId":"1:BASE_PARITY:6debb83d0b38302d"}
06:35:56 [06:35:56.809]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb04os0/1 with map size 1073741824
06:35:56 [06:35:56.811]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.829]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e90f571b1291d176 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e90f571b1291d176"}
06:35:56 [06:35:56.835]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6debb83d0b38302d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:6debb83d0b38302d"}
06:35:56 [06:35:56.846]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.847]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4443ea3b9ca68e8c epochNumber=1
{"provingJobId":"1:BASE_PARITY:4443ea3b9ca68e8c"}
06:35:56 [06:35:56.847]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzJZCeG/1 with map size 1073741824
06:35:56 [06:35:56.849]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.865]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c3a1788c42469e38 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c3a1788c42469e38"}
06:35:56 [06:35:56.873]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4443ea3b9ca68e8c type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:4443ea3b9ca68e8c"}
06:35:56 [06:35:56.889]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.890]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fa20c4c1a5d40124 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fa20c4c1a5d40124"}
06:35:56 [06:35:56.891]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testGT24Q2/1 with map size 1073741824
06:35:56 [06:35:56.893]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.917]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a91b1202edd96a6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0a91b1202edd96a6"}
06:35:56 [06:35:56.924]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:fa20c4c1a5d40124 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:fa20c4c1a5d40124"}
06:35:56 [06:35:56.936]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.937]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5fbe122d1eaf6736 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5fbe122d1eaf6736"}
06:35:56 [06:35:56.938]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testqfQ9bi/1 with map size 1073741824
06:35:56 [06:35:56.944]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:56 [06:35:56.963]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:5fbe122d1eaf6736
{"provingJobId":"1:BASE_PARITY:5fbe122d1eaf6736"}
06:35:56 [06:35:56.964]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:5fbe122d1eaf6736 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:5fbe122d1eaf6736"}
06:35:56 [06:35:56.970]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fc7e03358e5a3e28 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fc7e03358e5a3e28"}
06:35:56 [06:35:56.977]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5fbe122d1eaf6736 has already been completed
{"provingJobId":"1:BASE_PARITY:5fbe122d1eaf6736"}
06:35:56 [06:35:56.981]
INFO:
prover-client:proving-broker Proving Broker started
06:35:56 [06:35:56.982]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a2da369cfbd10dc7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a2da369cfbd10dc7"}
06:35:56 [06:35:56.983]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuuk7gS/1 with map size 1073741824
06:35:56 [06:35:56.985]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:57 [06:35:57.002]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d5befda5fe5ef091 epochNumber=2
{"provingJobId":"1:BASE_PARITY:d5befda5fe5ef091"}
06:35:57 [06:35:57.003]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testuuk7gS/2 with map size 1073741824
06:35:57 [06:35:57.005]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:58 [06:35:58.075]
INFO:
prover-client:proving-broker Proving Broker started
06:35:58 [06:35:58.082]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a2da369cfbd10dc7 settled=false
{"provingJobId":"1:BASE_PARITY:a2da369cfbd10dc7","status":"pending"}
06:35:58 [06:35:58.084]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d5befda5fe5ef091 settled=false
{"provingJobId":"1:BASE_PARITY:d5befda5fe5ef091","status":"pending"}
06:35:58 [06:35:58.085]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a2da369cfbd10dc7 type=BASE_PARITY startedAt=1750401357023 older agent has taken job
{"provingJobId":"1:BASE_PARITY:a2da369cfbd10dc7"}
06:35:58 [06:35:58.085]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a2da369cfbd10dc7 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:a2da369cfbd10dc7"}
06:35:58 [06:35:58.089]
INFO:
prover-client:proving-broker Proving Broker started
06:35:58 [06:35:58.090]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7ddc2a66cecf8457 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7ddc2a66cecf8457"}
06:35:58 [06:35:58.090]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLqkXuk/1 with map size 1073741824
06:35:58 [06:35:58.092]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:58 [06:35:58.110]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:546e967bbe34e8ea epochNumber=2
{"provingJobId":"1:BASE_PARITY:546e967bbe34e8ea"}
06:35:58 [06:35:58.111]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLqkXuk/2 with map size 1073741824
06:35:58 [06:35:58.113]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:59 [06:35:59.132]
INFO:
prover-client:proving-broker Proving Broker started
06:35:59 [06:35:59.134]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7ddc2a66cecf8457 settled=false
{"provingJobId":"1:BASE_PARITY:7ddc2a66cecf8457","status":"pending"}
06:35:59 [06:35:59.136]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:546e967bbe34e8ea settled=false
{"provingJobId":"1:BASE_PARITY:546e967bbe34e8ea","status":"pending"}
06:35:59 [06:35:59.136]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7ddc2a66cecf8457 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:7ddc2a66cecf8457"}
06:35:59 [06:35:59.139]
INFO:
prover-client:proving-broker Proving Broker started
06:35:59 [06:35:59.140]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55de7af1c36dd408 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55de7af1c36dd408"}
06:35:59 [06:35:59.141]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLz3oOh/1 with map size 1073741824
06:35:59 [06:35:59.142]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:35:59 [06:35:59.161]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db819013d89344bf epochNumber=2
{"provingJobId":"1:BASE_PARITY:db819013d89344bf"}
06:35:59 [06:35:59.162]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testLz3oOh/2 with map size 1073741824
06:35:59 [06:35:59.164]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.180]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.182]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:55de7af1c36dd408 settled=false
{"provingJobId":"1:BASE_PARITY:55de7af1c36dd408","status":"pending"}
06:36:00 [06:36:00.184]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:db819013d89344bf settled=false
{"provingJobId":"1:BASE_PARITY:db819013d89344bf","status":"pending"}
06:36:00 [06:36:00.185]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:55de7af1c36dd408 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:55de7af1c36dd408"}
06:36:00 [06:36:00.185]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:55de7af1c36dd408 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:55de7af1c36dd408"}
06:36:00 [06:36:00.193]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.194]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1bf46e49f94e23f0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1bf46e49f94e23f0"}
06:36:00 [06:36:00.195]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2gfyzx/1 with map size 1073741824
06:36:00 [06:36:00.196]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.212]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7c732c7e97056651 epochNumber=2
{"provingJobId":"1:BASE_PARITY:7c732c7e97056651"}
06:36:00 [06:36:00.213]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test2gfyzx/2 with map size 1073741824
06:36:00 [06:36:00.215]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.232]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1bf46e49f94e23f0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1bf46e49f94e23f0"}
06:36:00 [06:36:00.238]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:7c732c7e97056651 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:7c732c7e97056651"}
06:36:00 [06:36:00.248]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.249]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5dfa401df203e4a0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5dfa401df203e4a0"}
06:36:00 [06:36:00.249]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrMRcye/1 with map size 1073741824
06:36:00 [06:36:00.251]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.267]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ef67a666273fd397 epochNumber=2
{"provingJobId":"1:BASE_PARITY:ef67a666273fd397"}
06:36:00 [06:36:00.268]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrMRcye/2 with map size 1073741824
06:36:00 [06:36:00.271]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.288]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5dfa401df203e4a0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:5dfa401df203e4a0"}
06:36:00 [06:36:00.289]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5dfa401df203e4a0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:5dfa401df203e4a0"}
06:36:00 [06:36:00.295]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ef67a666273fd397 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:ef67a666273fd397"}
06:36:00 [06:36:00.305]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.306]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8c8471f784141590 err=test error
{"provingJoId":"1:BASE_PARITY:8c8471f784141590"}
06:36:00 [06:36:00.311]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.312]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:7a3bd769b1a1337a not found
{"provingJobId":"1:BASE_PARITY:7a3bd769b1a1337a"}
06:36:00 [06:36:00.316]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.317]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fd44eba9bc7cda34 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fd44eba9bc7cda34"}
06:36:00 [06:36:00.317]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQJLblw/1 with map size 1073741824
06:36:00 [06:36:00.319]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.338]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.339]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:64644a4888915b5b epochNumber=1
{"provingJobId":"1:BASE_PARITY:64644a4888915b5b"}
06:36:00 [06:36:00.340]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMXPuVr/1 with map size 1073741824
06:36:00 [06:36:00.342]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.465]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:64644a4888915b5b timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:64644a4888915b5b"}
06:36:00 [06:36:00.474]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.475]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:92779bb5e7dda1c7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:92779bb5e7dda1c7"}
06:36:00 [06:36:00.476]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJOs1Wc/1 with map size 1073741824
06:36:00 [06:36:00.478]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.603]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:92779bb5e7dda1c7 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:92779bb5e7dda1c7"}
06:36:00 [06:36:00.609]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bf0000cade72c5b2 epochNumber=10
{"provingJobId":"1:BASE_PARITY:bf0000cade72c5b2"}
06:36:00 [06:36:00.614]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testJOs1Wc/10 with map size 1073741824
06:36:00 [06:36:00.618]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.655]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:36:00 [06:36:00.656]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
06:36:00 [06:36:00.780]
INFO:
prover-client:proving-broker Proving Broker started
06:36:00 [06:36:00.781]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:83620307af6862d4 epochNumber=1
{"provingJobId":"1:BASE_PARITY:83620307af6862d4"}
06:36:00 [06:36:00.781]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testldKtSo/1 with map size 1073741824
06:36:00 [06:36:00.785]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:00 [06:36:00.960]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:83620307af6862d4 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:83620307af6862d4"}
06:36:01 [06:36:01.013]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.013]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3b3bfbaa758ece54 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3b3bfbaa758ece54"}
06:36:01 [06:36:01.015]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testPlQ1Ub/1 with map size 1073741824
06:36:01 [06:36:01.017]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.049]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:3b3bfbaa758ece54 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:3b3bfbaa758ece54"}
06:36:01 [06:36:01.055]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.056]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c1f754f0989ef539 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c1f754f0989ef539"}
06:36:01 [06:36:01.058]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5WOIsu/1 with map size 1073741824
06:36:01 [06:36:01.062]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.081]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c1f754f0989ef539 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:c1f754f0989ef539"}
06:36:01 [06:36:01.082]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c1f754f0989ef539 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:c1f754f0989ef539"}
06:36:01 [06:36:01.094]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.095]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1b3a9ed0aa5c3ebc epochNumber=1
{"provingJobId":"1:BASE_PARITY:1b3a9ed0aa5c3ebc"}
06:36:01 [06:36:01.096]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmV8VE6/1 with map size 1073741824
06:36:01 [06:36:01.099]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.121]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1b3a9ed0aa5c3ebc type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1b3a9ed0aa5c3ebc"}
06:36:01 [06:36:01.136]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.138]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5a2ac6e5b5b600e6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5a2ac6e5b5b600e6"}
06:36:01 [06:36:01.140]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMYWaqt/1 with map size 1073741824
06:36:01 [06:36:01.144]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.163]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:5a2ac6e5b5b600e6 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:5a2ac6e5b5b600e6"}
06:36:01 [06:36:01.164]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2c8c385c559978d9 epochNumber=3
{"provingJobId":"1:BASE_PARITY:2c8c385c559978d9"}
06:36:01 [06:36:01.165]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testMYWaqt/3 with map size 1073741824
06:36:01 [06:36:01.167]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.189]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:36:01 [06:36:01.189]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
06:36:01 [06:36:01.210]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:5a2ac6e5b5b600e6 err=test error
{"provingJoId":"1:BASE_PARITY:5a2ac6e5b5b600e6"}
06:36:01 [06:36:01.215]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testL2tXY3/1 with map size 1073741824
06:36:01 [06:36:01.218]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.244]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testL2tXY3/2 with map size 1073741824
06:36:01 [06:36:01.253]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.269]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.271]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:058f70df7e7338c6 settled=false
{"provingJobId":"1:BASE_PARITY:058f70df7e7338c6","status":"pending"}
06:36:01 [06:36:01.272]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e245f1b5531779e4 settled=false
{"provingJobId":"1:BASE_PARITY:e245f1b5531779e4","status":"pending"}
06:36:01 [06:36:01.278]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUq4qLd/1 with map size 1073741824
06:36:01 [06:36:01.281]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.299]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testUq4qLd/2 with map size 1073741824
06:36:01 [06:36:01.304]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.348]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.353]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6d9b553350b78f27 settled=true
{"provingJobId":"1:BASE_PARITY:6d9b553350b78f27","status":"fulfilled"}
06:36:01 [06:36:01.356]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:135bf954dd6164cf settled=true
{"provingJobId":"2:BASE_PARITY:135bf954dd6164cf","status":"fulfilled"}
06:36:01 [06:36:01.362]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testtRpDmM/1 with map size 1073741824
06:36:01 [06:36:01.366]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.395]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testtRpDmM/2 with map size 1073741824
06:36:01 [06:36:01.398]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.423]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.426]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cf96954e66f55409 settled=true
{"provingJobId":"1:BASE_PARITY:cf96954e66f55409","status":"fulfilled"}
06:36:01 [06:36:01.433]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:251bff28ac1680c5 settled=false
{"provingJobId":"1:BASE_PARITY:251bff28ac1680c5","status":"pending"}
06:36:01 [06:36:01.446]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.447]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e593287b27bfe8f6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:e593287b27bfe8f6"}
06:36:01 [06:36:01.451]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoSBJbm/1 with map size 1073741824
06:36:01 [06:36:01.454]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.483]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.484]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c87a6a137a0b2c8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6c87a6a137a0b2c8"}
06:36:01 [06:36:01.485]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:6c87a6a137a0b2c8: Error: db error: Error: db error
06:36:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:6c87a6a137a0b2c8"}
06:36:01 [06:36:01.489]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.489]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:59897a2d80c570d8 epochNumber=1
{"provingJobId":"1:BASE_PARITY:59897a2d80c570d8"}
06:36:01 [06:36:01.491]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testfqUjJM/1 with map size 1073741824
06:36:01 [06:36:01.494]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.512]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:59897a2d80c570d8 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:59897a2d80c570d8"}
06:36:01 [06:36:01.513]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:59897a2d80c570d8 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:59897a2d80c570d8"}
06:36:01 [06:36:01.526]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.526]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9e35fe9d41b2f481 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9e35fe9d41b2f481"}
06:36:01 [06:36:01.530]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSFYZV7/1 with map size 1073741824
06:36:01 [06:36:01.544]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.570]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9e35fe9d41b2f481 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9e35fe9d41b2f481"}
06:36:01 [06:36:01.571]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9e35fe9d41b2f481 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:9e35fe9d41b2f481"}
06:36:01 [06:36:01.572]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:9e35fe9d41b2f481: Error: db error
06:36:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:9e35fe9d41b2f481"}
06:36:01 [06:36:01.579]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.579]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:36fb9984eb5e0348 epochNumber=1
{"provingJobId":"1:BASE_PARITY:36fb9984eb5e0348"}
06:36:01 [06:36:01.580]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzpbkYc/1 with map size 1073741824
06:36:01 [06:36:01.585]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.607]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:36fb9984eb5e0348 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:36fb9984eb5e0348"}
06:36:01 [06:36:01.608]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:36fb9984eb5e0348 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:36fb9984eb5e0348"}
06:36:01 [06:36:01.619]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.620]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a3a3ba0803287cfb epochNumber=1
{"provingJobId":"1:BASE_PARITY:a3a3ba0803287cfb"}
06:36:01 [06:36:01.622]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2m2Gi9/1 with map size 1073741824
06:36:01 [06:36:01.625]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.649]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a3a3ba0803287cfb type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:a3a3ba0803287cfb"}
06:36:01 [06:36:01.650]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a3a3ba0803287cfb type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a3a3ba0803287cfb"}
06:36:01 [06:36:01.651]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:a3a3ba0803287cfb jobErr=test error: Error: db error
06:36:01 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:a3a3ba0803287cfb"}
06:36:01 [06:36:01.658]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.659]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ce6b38f537121d80 not found
{"provingJobId":"1:BASE_PARITY:ce6b38f537121d80"}
06:36:01 [06:36:01.663]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.663]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:aae2f85d09c732e6 err=test error
{"provingJoId":"1:BASE_PARITY:aae2f85d09c732e6"}
06:36:01 [06:36:01.668]
INFO:
prover-client:proving-broker Proving Broker started
06:36:01 [06:36:01.678]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d99c1c286d371f48 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d99c1c286d371f48"}
06:36:01 [06:36:01.680]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmMUOuo/1 with map size 1073741824
06:36:01 [06:36:01.684]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.705]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d99c1c286d371f48 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:d99c1c286d371f48"}
06:36:01 [06:36:01.706]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d99c1c286d371f48 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:d99c1c286d371f48"}
06:36:01 [06:36:01.722]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:e18393e88e2611c6 epochNumber=2
{"provingJobId":"2:BASE_PARITY:e18393e88e2611c6"}
06:36:01 [06:36:01.723]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testmMUOuo/2 with map size 1073741824
06:36:01 [06:36:01.726]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.750]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:e18393e88e2611c6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:e18393e88e2611c6"}
06:36:01 [06:36:01.750]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:e18393e88e2611c6 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:e18393e88e2611c6"}
06:36:01 [06:36:01.820]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:728c140d058a8ea2 epochNumber=3
{"provingJobId":"3:BASE_PARITY:728c140d058a8ea2"}
06:36:01 [06:36:01.821]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testmMUOuo/3 with map size 1073741824
06:36:01 [06:36:01.825]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.874]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:36:01 [06:36:01.874]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
06:36:01 [06:36:01.922]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:e147ec03c7add661 epochNumber=4
{"provingJobId":"4:BASE_PARITY:e147ec03c7add661"}
06:36:01 [06:36:01.923]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testmMUOuo/4 with map size 1073741824
06:36:01 [06:36:01.934]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:01 [06:36:01.978]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:36:01 [06:36:01.978]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
06:36:02 [06:36:02.023]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:4dd66f244688dbd7 epochNumber=5
{"provingJobId":"5:BASE_PARITY:4dd66f244688dbd7"}
06:36:02 [06:36:02.025]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testmMUOuo/5 with map size 1073741824
06:36:02 [06:36:02.037]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
06:36:02 [06:36:02.083]
INFO:
prover-client:proving-broker Cleaned up jobs=1
06:36:02 [06:36:02.084]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
06:36:02
PASS src/proving_broker/proving_broker.test.ts (
13.309 s)
06:36:02 ProvingBroker
06:36:02 Producer API
06:36:02
✓ refuses stale jobs (50 ms)
06:36:02
✓ enqueues jobs (4 ms)
06:36:02
✓ ignores duplicate jobs (4 ms)
06:36:02
✓ reports correct status when enqueuing repeat jobs (4 ms)
06:36:02
✓ throws an error in case of duplicate job IDs (33 ms)
06:36:02
✓ returns not-found status for non-existing jobs (2 ms)
06:36:02
✓ cancels jobs in queue (4 ms)
06:36:02
✓ cancels jobs in-progress (2 ms)
06:36:02
✓ returns job result if successful (3 ms)
06:36:02
✓ returns job error if failed (2 ms)
06:36:02
✓ correctly returns job status for concurrent writes (3 ms)
06:36:02
✓ refuses stale jobs (39 ms)
06:36:02
✓ enqueues jobs (32 ms)
06:36:02
✓ ignores duplicate jobs (26 ms)
06:36:02
✓ reports correct status when enqueuing repeat jobs (33 ms)
06:36:02
✓ throws an error in case of duplicate job IDs (25 ms)
06:36:02
✓ returns not-found status for non-existing jobs (3 ms)
06:36:02
✓ cancels jobs in queue (30 ms)
06:36:02
✓ cancels jobs in-progress (30 ms)
06:36:02
✓ returns job result if successful (29 ms)
06:36:02
✓ returns job error if failed (34 ms)
06:36:02
✓ correctly returns job status for concurrent writes (39 ms)
06:36:02 Consumer API
06:36:02
✓ returns undefined if no jobs are available (1 ms)
06:36:02
✓ returns jobs in priority order (2 ms)
06:36:02
✓ returns undefined if no jobs are available for the given allowList (1 ms)
06:36:02
✓ returns a job if it is in the allowList (2 ms)
06:36:02
✓ returns the most important job if it is in the allowList (2 ms)
06:36:02
✓ returns any job if filter is empty (3 ms)
06:36:02
✓ returns a new job when reporting job success (2 ms)
06:36:02
✓ returns a new job when reporting permanent error (2 ms)
06:36:02
✓ returns a new job when reporting retry-able error (3 ms)
06:36:02
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
06:36:02
✓ returns a new job if job is already in progress elsewhere (1057 ms)
06:36:02
✓ avoids sending the same job to a new agent after a restart (1004 ms)
06:36:02
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
06:36:02
✓ tracks job result if in progress (3 ms)
06:36:02
✓ tracks job result even if job is in queue (3 ms)
06:36:02
✓ ignores reported job error if unknown job (1 ms)
06:36:02
✓ ignores job result if unknown job (2 ms)
06:36:02
✓ returns undefined if no jobs are available (2 ms)
06:36:02
✓ returns jobs in priority order (61 ms)
06:36:02
✓ returns undefined if no jobs are available for the given allowList (24 ms)
06:36:02
✓ returns a job if it is in the allowList (61 ms)
06:36:02
✓ returns the most important job if it is in the allowList (59 ms)
06:36:02
✓ returns any job if filter is empty (57 ms)
06:36:02
✓ returns a new job when reporting job success (37 ms)
06:36:02
✓ returns a new job when reporting permanent error (39 ms)
06:36:02
✓ returns a new job when reporting retry-able error (42 ms)
06:36:02
✓ returns a new job when reporting progress if current one is cancelled (52 ms)
06:36:02
✓ returns a new job if job is already in progress elsewhere (1107 ms)
06:36:02
✓ avoids sending the same job to a new agent after a restart (1051 ms)
06:36:02
✓ avoids sending a completed job to a new agent after a restart (1054 ms)
06:36:02
✓ tracks job result if in progress (53 ms)
06:36:02
✓ tracks job result even if job is in queue (56 ms)
06:36:02
✓ ignores reported job error if unknown job (4 ms)
06:36:02
✓ ignores job result if unknown job (5 ms)
06:36:02 Timeouts
06:36:02
✓ tracks in progress jobs (2 ms)
06:36:02
✓ re-enqueues jobs that time out (108 ms)
06:36:02
✓ cancel stale jobs that time out (237 ms)
06:36:02
✓ keeps the jobs in progress while it is alive (204 ms)
06:36:02
✓ tracks in progress jobs (22 ms)
06:36:02
✓ re-enqueues jobs that time out (135 ms)
06:36:02
✓ cancel stale jobs that time out (306 ms)
06:36:02
✓ keeps the jobs in progress while it is alive (232 ms)
06:36:02 Retries
06:36:02
✓ retries jobs (3 ms)
06:36:02
✓ retries up to a maximum number of times (2 ms)
06:36:02
✓ passing retry=false does not retry (2 ms)
06:36:02
✓ does not retry if job is stale (28 ms)
06:36:02
✓ retries jobs (40 ms)
06:36:02
✓ retries up to a maximum number of times (38 ms)
06:36:02
✓ passing retry=false does not retry (41 ms)
06:36:02
✓ does not retry if job is stale (80 ms)
06:36:02 Database management
06:36:02
✓ re-enqueues proof requests on start (3 ms)
06:36:02
✓ restores proof results on start (2 ms)
06:36:02
✓ only re-enqueues unfinished jobs (2 ms)
06:36:02
✓ saves job when enqueued (3 ms)
06:36:02
✓ does not retain job if database fails to save (3 ms)
06:36:02
✓ saves job result (2 ms)
06:36:02
✓ saves result even if database fails to save (3 ms)
06:36:02
✓ saves job error (2 ms)
06:36:02
✓ saves job error even if database fails to save (3 ms)
06:36:02
✓ does not save job result if job is unknown (2 ms)
06:36:02
✓ does not save job error if job is unknown (1 ms)
06:36:02
✓ cleans up old jobs periodically (259 ms)
06:36:02
✓ re-enqueues proof requests on start (64 ms)
06:36:02
✓ restores proof results on start (84 ms)
06:36:02
✓ only re-enqueues unfinished jobs (82 ms)
06:36:02
✓ saves job when enqueued (37 ms)
06:36:02
✓ does not retain job if database fails to save (5 ms)
06:36:02
✓ saves job result (36 ms)
06:36:02
✓ saves result even if database fails to save (50 ms)
06:36:02
✓ saves job error (42 ms)
06:36:02
✓ saves job error even if database fails to save (39 ms)
06:36:02
✓ does not save job result if job is unknown (4 ms)
06:36:02
✓ does not save job error if job is unknown (4 ms)
06:36:02
✓ cleans up old jobs periodically (453 ms)
06:36:02
06:36:02
Test Suites: 1 passed, 1 total
06:36:02
Tests: 96 passed, 96 total
06:36:02
Snapshots: 0 total
06:36:02
Time: 13.397 s
06:36:02
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
06:36:02
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?