Parent Log:
http://ci.aztec-labs.com/a36edd533f52a26c
Command: 47858aa9bf665f8a yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/1bb4f0588a9dc5b1df622753aab1d907b3f12c7a
Env: REF_NAME=gh-readonly-queue/next/pr-15245-534f06e45f48274efd53a2204c90cb0b91a53efb CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Wed Jun 25 08:19:48 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15245_amd64_x1-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next
08:19:52 [08:19:52.794]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.802]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:db5946701a98152c epochNumber=42
{"provingJobId":"1:BASE_PARITY:db5946701a98152c"}
08:19:52 [08:19:52.806]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:f670a373f158663b epochNumber=1
{"provingJobId":"1:BASE_PARITY:f670a373f158663b"}
08:19:52 [08:19:52.851]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.852]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:918f38b376e72b10 epochNumber=1
{"provingJobId":"1:BASE_PARITY:918f38b376e72b10"}
08:19:52 [08:19:52.854]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0767027cb4cd4adc epochNumber=1
{"provingJobId":"1:BASE_PARITY:0767027cb4cd4adc"}
08:19:52 [08:19:52.856]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.857]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d0c39e21e08a1502 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d0c39e21e08a1502"}
08:19:52 [08:19:52.858]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:d0c39e21e08a1502 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:d0c39e21e08a1502"}
08:19:52 [08:19:52.862]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.863]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a37d161360675d29 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a37d161360675d29"}
08:19:52 [08:19:52.864]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a37d161360675d29 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a37d161360675d29"}
08:19:52 [08:19:52.865]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a37d161360675d29 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:a37d161360675d29"}
08:19:52 [08:19:52.866]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a37d161360675d29 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a37d161360675d29"}
08:19:52 [08:19:52.867]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.868]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6aec3e5b5903ce22 epochNumber=1
{"provingJobId":"1:BASE_PARITY:6aec3e5b5903ce22"}
08:19:52 [08:19:52.898]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.900]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.901]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:630c90aeb504cd9f epochNumber=1
{"provingJobId":"1:BASE_PARITY:630c90aeb504cd9f"}
08:19:52 [08:19:52.902]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:630c90aeb504cd9f
{"provingJobId":"1:BASE_PARITY:630c90aeb504cd9f"}
08:19:52 [08:19:52.902]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:630c90aeb504cd9f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:630c90aeb504cd9f"}
08:19:52 [08:19:52.902]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:630c90aeb504cd9f type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:630c90aeb504cd9f"}
08:19:52 [08:19:52.904]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.905]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:54b269b5e48ee501 epochNumber=1
{"provingJobId":"1:BASE_PARITY:54b269b5e48ee501"}
08:19:52 [08:19:52.906]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:54b269b5e48ee501
{"provingJobId":"1:BASE_PARITY:54b269b5e48ee501"}
08:19:52 [08:19:52.906]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:54b269b5e48ee501 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:54b269b5e48ee501"}
08:19:52 [08:19:52.908]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.909]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4712bdb708b3cf8f epochNumber=1
{"provingJobId":"1:BASE_PARITY:4712bdb708b3cf8f"}
08:19:52 [08:19:52.909]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4712bdb708b3cf8f type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4712bdb708b3cf8f"}
08:19:52 [08:19:52.918]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4712bdb708b3cf8f type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:4712bdb708b3cf8f"}
08:19:52 [08:19:52.921]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.922]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9a74e6dcfa608350 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9a74e6dcfa608350"}
08:19:52 [08:19:52.923]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9a74e6dcfa608350 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:9a74e6dcfa608350"}
08:19:52 [08:19:52.923]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:9a74e6dcfa608350 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:9a74e6dcfa608350"}
08:19:52 [08:19:52.925]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0a5a32e860d15a01 epochNumber=0
{"provingJobId":"1:BASE_PARITY:0a5a32e860d15a01"}
08:19:52 [08:19:52.926]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0a5a32e860d15a01 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0a5a32e860d15a01"}
08:19:52 [08:19:52.926]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1c0d82876815461e epochNumber=0
{"provingJobId":"1:BASE_PARITY:1c0d82876815461e"}
08:19:52 [08:19:52.927]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:0a5a32e860d15a01 epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:0a5a32e860d15a01"}
08:19:52 [08:19:52.927]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:83ed768d4876179c epochNumber=0
{"provingJobId":"1:BASE_PARITY:83ed768d4876179c"}
08:19:52 [08:19:52.929]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.930]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.931]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7bda39008cfeffa epochNumber=2
{"provingJobId":"1:BASE_PARITY:b7bda39008cfeffa"}
08:19:52 [08:19:52.931]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5eede08d70d1dddf epochNumber=1
{"provingJobId":"1:BASE_PARITY:5eede08d70d1dddf"}
08:19:52 [08:19:52.933]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.934]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0d193207f414172d epochNumber=1
{"provingJobId":"1:BASE_PARITY:0d193207f414172d"}
08:19:52 [08:19:52.935]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.935]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c75ed0a019fa4862 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c75ed0a019fa4862"}
08:19:52 [08:19:52.936]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:69216460af04803b epochNumber=1
{"provingJobId":"1:BASE_PARITY:69216460af04803b"}
08:19:52 [08:19:52.936]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:526f3d5b2307d71a epochNumber=2
{"provingJobId":"1:BASE_PARITY:526f3d5b2307d71a"}
08:19:52 [08:19:52.936]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2f93a6526bed7133 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2f93a6526bed7133"}
08:19:52 [08:19:52.938]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a11f0f73b63ff812 epochNumber=1
{"provingJobId":"1:BASE_PARITY:a11f0f73b63ff812"}
08:19:52 [08:19:52.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dbc0d7397efedd1a epochNumber=1
{"provingJobId":"1:BASE_PARITY:dbc0d7397efedd1a"}
08:19:52 [08:19:52.939]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9140d73c1bda1e08 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9140d73c1bda1e08"}
08:19:52 [08:19:52.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:279f73e03f7e89bb epochNumber=1
{"provingJobId":"1:BASE_PARITY:279f73e03f7e89bb"}
08:19:52 [08:19:52.941]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.941]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:e7ce8b6b3a17ae6f epochNumber=1
{"provingJobId":"1:BASE_PARITY:e7ce8b6b3a17ae6f"}
08:19:52 [08:19:52.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:30eca8bf2a018890 epochNumber=1
{"provingJobId":"1:BASE_PARITY:30eca8bf2a018890"}
08:19:52 [08:19:52.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b17203a66573ce5a epochNumber=2
{"provingJobId":"1:BASE_PARITY:b17203a66573ce5a"}
08:19:52 [08:19:52.942]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4e998f12b6bf0d59 epochNumber=1
{"provingJobId":"1:BASE_PARITY:4e998f12b6bf0d59"}
08:19:52 [08:19:52.943]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.944]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:61e10b20e6cd50d7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:61e10b20e6cd50d7"}
08:19:52 [08:19:52.944]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9422fd0dd1b4a501 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9422fd0dd1b4a501"}
08:19:52 [08:19:52.945]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:61e10b20e6cd50d7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:61e10b20e6cd50d7"}
08:19:52 [08:19:52.946]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.946]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:735dda9dfd400401 epochNumber=1
{"provingJobId":"1:BASE_PARITY:735dda9dfd400401"}
08:19:52 [08:19:52.947]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:977ffff0cdb4193d epochNumber=1
{"provingJobId":"1:BASE_PARITY:977ffff0cdb4193d"}
08:19:52 [08:19:52.947]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:735dda9dfd400401 type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:735dda9dfd400401"}
08:19:52 [08:19:52.948]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.948]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:68915bbdc71d89cd epochNumber=1
{"provingJobId":"1:BASE_PARITY:68915bbdc71d89cd"}
08:19:52 [08:19:52.948]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:719cf094daca6c9c epochNumber=1
{"provingJobId":"1:BASE_PARITY:719cf094daca6c9c"}
08:19:52 [08:19:52.949]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:68915bbdc71d89cd type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:68915bbdc71d89cd"}
08:19:52 [08:19:52.949]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.950]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3975a63b3c16ebb0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3975a63b3c16ebb0"}
08:19:52 [08:19:52.950]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:3975a63b3c16ebb0
{"provingJobId":"1:BASE_PARITY:3975a63b3c16ebb0"}
08:19:52 [08:19:52.951]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3975a63b3c16ebb0 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:3975a63b3c16ebb0"}
08:19:52 [08:19:52.951]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f89b9a249445009b epochNumber=1
{"provingJobId":"1:BASE_PARITY:f89b9a249445009b"}
08:19:52 [08:19:52.951]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3975a63b3c16ebb0 has already been completed
{"provingJobId":"1:BASE_PARITY:3975a63b3c16ebb0"}
08:19:52 [08:19:52.952]
INFO:
prover-client:proving-broker Proving Broker started
08:19:52 [08:19:52.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4349c63fe3669e3e epochNumber=1
{"provingJobId":"1:BASE_PARITY:4349c63fe3669e3e"}
08:19:52 [08:19:52.953]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bab8a7514b20e88b epochNumber=2
{"provingJobId":"1:BASE_PARITY:bab8a7514b20e88b"}
08:19:54 [08:19:54.004]
INFO:
prover-client:proving-broker Proving Broker started
08:19:54 [08:19:54.005]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4349c63fe3669e3e settled=false
{"provingJobId":"1:BASE_PARITY:4349c63fe3669e3e","status":"pending"}
08:19:54 [08:19:54.006]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bab8a7514b20e88b settled=false
{"provingJobId":"1:BASE_PARITY:bab8a7514b20e88b","status":"pending"}
08:19:54 [08:19:54.007]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4349c63fe3669e3e type=BASE_PARITY startedAt=1750839592953 older agent has taken job
{"provingJobId":"1:BASE_PARITY:4349c63fe3669e3e"}
08:19:54 [08:19:54.007]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4349c63fe3669e3e type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:4349c63fe3669e3e"}
08:19:54 [08:19:54.009]
INFO:
prover-client:proving-broker Proving Broker started
08:19:54 [08:19:54.009]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d7e3561911869fc2 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d7e3561911869fc2"}
08:19:54 [08:19:54.010]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:18910880217fe32e epochNumber=2
{"provingJobId":"1:BASE_PARITY:18910880217fe32e"}
08:19:55 [08:19:55.011]
INFO:
prover-client:proving-broker Proving Broker started
08:19:55 [08:19:55.011]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:d7e3561911869fc2 settled=false
{"provingJobId":"1:BASE_PARITY:d7e3561911869fc2","status":"pending"}
08:19:55 [08:19:55.012]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:18910880217fe32e settled=false
{"provingJobId":"1:BASE_PARITY:18910880217fe32e","status":"pending"}
08:19:55 [08:19:55.012]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:d7e3561911869fc2 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:d7e3561911869fc2"}
08:19:55 [08:19:55.014]
INFO:
prover-client:proving-broker Proving Broker started
08:19:55 [08:19:55.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:463d1ed9a62409dd epochNumber=1
{"provingJobId":"1:BASE_PARITY:463d1ed9a62409dd"}
08:19:55 [08:19:55.015]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:da7d76f82904d5dd epochNumber=2
{"provingJobId":"1:BASE_PARITY:da7d76f82904d5dd"}
08:19:56 [08:19:56.015]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.016]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:463d1ed9a62409dd settled=false
{"provingJobId":"1:BASE_PARITY:463d1ed9a62409dd","status":"pending"}
08:19:56 [08:19:56.016]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:da7d76f82904d5dd settled=false
{"provingJobId":"1:BASE_PARITY:da7d76f82904d5dd","status":"pending"}
08:19:56 [08:19:56.017]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:463d1ed9a62409dd type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:463d1ed9a62409dd"}
08:19:56 [08:19:56.017]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:463d1ed9a62409dd type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:463d1ed9a62409dd"}
08:19:56 [08:19:56.019]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.019]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1f95cac31c605a85 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1f95cac31c605a85"}
08:19:56 [08:19:56.020]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6ed4e153818a364a epochNumber=2
{"provingJobId":"1:BASE_PARITY:6ed4e153818a364a"}
08:19:56 [08:19:56.020]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1f95cac31c605a85 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1f95cac31c605a85"}
08:19:56 [08:19:56.021]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6ed4e153818a364a type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:6ed4e153818a364a"}
08:19:56 [08:19:56.022]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.022]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:546aba925f1dcc8a epochNumber=1
{"provingJobId":"1:BASE_PARITY:546aba925f1dcc8a"}
08:19:56 [08:19:56.023]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1e418d8973c31ab8 epochNumber=2
{"provingJobId":"1:BASE_PARITY:1e418d8973c31ab8"}
08:19:56 [08:19:56.023]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:546aba925f1dcc8a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:546aba925f1dcc8a"}
08:19:56 [08:19:56.023]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:546aba925f1dcc8a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:546aba925f1dcc8a"}
08:19:56 [08:19:56.023]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1e418d8973c31ab8 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:1e418d8973c31ab8"}
08:19:56 [08:19:56.024]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.025]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0f53cd03d0b8d312 err=test error
{"provingJoId":"1:BASE_PARITY:0f53cd03d0b8d312"}
08:19:56 [08:19:56.026]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.026]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:ed6c247aa5015794 not found
{"provingJobId":"1:BASE_PARITY:ed6c247aa5015794"}
08:19:56 [08:19:56.027]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.028]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b7f7f148a6aefc53 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b7f7f148a6aefc53"}
08:19:56 [08:19:56.029]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.030]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:84846180838d1a76 epochNumber=1
{"provingJobId":"1:BASE_PARITY:84846180838d1a76"}
08:19:56 [08:19:56.131]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:84846180838d1a76 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:84846180838d1a76"}
08:19:56 [08:19:56.138]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.139]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:64c8bd79ef1f107d epochNumber=1
{"provingJobId":"1:BASE_PARITY:64c8bd79ef1f107d"}
08:19:56 [08:19:56.241]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:64c8bd79ef1f107d timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:64c8bd79ef1f107d"}
08:19:56 [08:19:56.246]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a05792f323c51b3b epochNumber=10
{"provingJobId":"1:BASE_PARITY:a05792f323c51b3b"}
08:19:56 [08:19:56.267]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:19:56 [08:19:56.374]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.375]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9120dc3a134d29b5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9120dc3a134d29b5"}
08:19:56 [08:19:56.528]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:9120dc3a134d29b5 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:9120dc3a134d29b5"}
08:19:56 [08:19:56.580]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.580]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bb2515653d27525b epochNumber=1
{"provingJobId":"1:BASE_PARITY:bb2515653d27525b"}
08:19:56 [08:19:56.581]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:bb2515653d27525b type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:bb2515653d27525b"}
08:19:56 [08:19:56.583]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.583]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0cc2199c0963f720 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0cc2199c0963f720"}
08:19:56 [08:19:56.584]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0cc2199c0963f720 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:0cc2199c0963f720"}
08:19:56 [08:19:56.585]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:0cc2199c0963f720 type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:0cc2199c0963f720"}
08:19:56 [08:19:56.586]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.586]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:297fd9c5a3fdd7e7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:297fd9c5a3fdd7e7"}
08:19:56 [08:19:56.587]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:297fd9c5a3fdd7e7 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:297fd9c5a3fdd7e7"}
08:19:56 [08:19:56.588]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.589]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:357405f3bbc2399a epochNumber=1
{"provingJobId":"1:BASE_PARITY:357405f3bbc2399a"}
08:19:56 [08:19:56.589]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:357405f3bbc2399a type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:357405f3bbc2399a"}
08:19:56 [08:19:56.590]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a8a0f762c612bd1e epochNumber=3
{"provingJobId":"1:BASE_PARITY:a8a0f762c612bd1e"}
08:19:56 [08:19:56.613]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:19:56 [08:19:56.616]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:357405f3bbc2399a err=test error
{"provingJoId":"1:BASE_PARITY:357405f3bbc2399a"}
08:19:56 [08:19:56.617]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.617]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f2825009e271a43e settled=false
{"provingJobId":"1:BASE_PARITY:f2825009e271a43e","status":"pending"}
08:19:56 [08:19:56.617]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:13ffc11626516b82 settled=false
{"provingJobId":"1:BASE_PARITY:13ffc11626516b82","status":"pending"}
08:19:56 [08:19:56.619]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.619]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0862187b2d2ce811 settled=true
{"provingJobId":"1:BASE_PARITY:0862187b2d2ce811","status":"fulfilled"}
08:19:56 [08:19:56.619]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:25bc0bc4d3a3f027 settled=true
{"provingJobId":"2:BASE_PARITY:25bc0bc4d3a3f027","status":"fulfilled"}
08:19:56 [08:19:56.620]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.620]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0ee74d92ec252e70 settled=true
{"provingJobId":"1:BASE_PARITY:0ee74d92ec252e70","status":"fulfilled"}
08:19:56 [08:19:56.621]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f5fb2f934a1a1402 settled=false
{"provingJobId":"1:BASE_PARITY:f5fb2f934a1a1402","status":"pending"}
08:19:56 [08:19:56.622]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.623]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:7e34f7c7651701b5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:7e34f7c7651701b5"}
08:19:56 [08:19:56.624]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.624]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:15181f0c3b775144 epochNumber=1
{"provingJobId":"1:BASE_PARITY:15181f0c3b775144"}
08:19:56 [08:19:56.625]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:15181f0c3b775144: Error: db error: Error: db error
08:19:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:15181f0c3b775144"}
08:19:56 [08:19:56.626]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.627]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:37dab4d679ece5ab epochNumber=1
{"provingJobId":"1:BASE_PARITY:37dab4d679ece5ab"}
08:19:56 [08:19:56.627]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:37dab4d679ece5ab type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:37dab4d679ece5ab"}
08:19:56 [08:19:56.627]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:37dab4d679ece5ab type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:37dab4d679ece5ab"}
08:19:56 [08:19:56.628]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.629]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:734268ca4b3e04c0 epochNumber=1
{"provingJobId":"1:BASE_PARITY:734268ca4b3e04c0"}
08:19:56 [08:19:56.629]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:734268ca4b3e04c0 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:734268ca4b3e04c0"}
08:19:56 [08:19:56.629]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:734268ca4b3e04c0 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:734268ca4b3e04c0"}
08:19:56 [08:19:56.630]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:734268ca4b3e04c0: Error: db error
08:19:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:734268ca4b3e04c0"}
08:19:56 [08:19:56.631]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.631]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:fe245c449e139ce6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:fe245c449e139ce6"}
08:19:56 [08:19:56.632]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:fe245c449e139ce6 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:fe245c449e139ce6"}
08:19:56 [08:19:56.632]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:fe245c449e139ce6 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:fe245c449e139ce6"}
08:19:56 [08:19:56.633]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.633]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:4f48210aff4cdd7c epochNumber=1
{"provingJobId":"1:BASE_PARITY:4f48210aff4cdd7c"}
08:19:56 [08:19:56.634]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:4f48210aff4cdd7c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:4f48210aff4cdd7c"}
08:19:56 [08:19:56.634]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4f48210aff4cdd7c type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:4f48210aff4cdd7c"}
08:19:56 [08:19:56.634]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:4f48210aff4cdd7c jobErr=test error: Error: db error
08:19:56 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:4f48210aff4cdd7c"}
08:19:56 [08:19:56.635]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.636]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:182a6c162091ae71 not found
{"provingJobId":"1:BASE_PARITY:182a6c162091ae71"}
08:19:56 [08:19:56.637]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.637]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:02186a284f605b44 err=test error
{"provingJoId":"1:BASE_PARITY:02186a284f605b44"}
08:19:56 [08:19:56.638]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.649]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2dfbde7635868750 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2dfbde7635868750"}
08:19:56 [08:19:56.650]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2dfbde7635868750 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2dfbde7635868750"}
08:19:56 [08:19:56.650]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2dfbde7635868750 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2dfbde7635868750"}
08:19:56 [08:19:56.661]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:488a9f0d2a6083c7 epochNumber=2
{"provingJobId":"2:BASE_PARITY:488a9f0d2a6083c7"}
08:19:56 [08:19:56.662]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:488a9f0d2a6083c7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:488a9f0d2a6083c7"}
08:19:56 [08:19:56.662]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:488a9f0d2a6083c7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:488a9f0d2a6083c7"}
08:19:56 [08:19:56.724]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:5ce1d953f70c3c92 epochNumber=3
{"provingJobId":"3:BASE_PARITY:5ce1d953f70c3c92"}
08:19:56 [08:19:56.739]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:19:56 [08:19:56.787]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:35c3b52551396e6d epochNumber=4
{"provingJobId":"4:BASE_PARITY:35c3b52551396e6d"}
08:19:56 [08:19:56.791]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:19:56 [08:19:56.850]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:f836f8f654d34ef4 epochNumber=5
{"provingJobId":"5:BASE_PARITY:f836f8f654d34ef4"}
08:19:56 [08:19:56.867]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:19:56 [08:19:56.904]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.905]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b622cd028a168da7 epochNumber=42
{"provingJobId":"1:BASE_PARITY:b622cd028a168da7"}
08:19:56 [08:19:56.905]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testrBtxKm/42 with map size 1073741824
08:19:56 [08:19:56.909]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:56 [08:19:56.935]
WARN:
prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:0befd5e6a9371990 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0befd5e6a9371990"}
08:19:56 [08:19:56.940]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.940]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b21edb3de1958a67 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b21edb3de1958a67"}
08:19:56 [08:19:56.941]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testywRoGW/1 with map size 1073741824
08:19:56 [08:19:56.943]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:56 [08:19:56.962]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:813e30209240d57a epochNumber=1
{"provingJobId":"1:BASE_PARITY:813e30209240d57a"}
08:19:56 [08:19:56.971]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.972]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f8eb61a7a5c01881 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f8eb61a7a5c01881"}
08:19:56 [08:19:56.972]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-tests3QP9s/1 with map size 1073741824
08:19:56 [08:19:56.975]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:56 [08:19:56.993]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f8eb61a7a5c01881 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f8eb61a7a5c01881"}
08:19:56 [08:19:56.997]
INFO:
prover-client:proving-broker Proving Broker started
08:19:56 [08:19:56.998]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f317f914a278ea91 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f317f914a278ea91"}
08:19:56 [08:19:56.998]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0DwLTa/1 with map size 1073741824
08:19:57 [08:19:57.001]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.017]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f317f914a278ea91 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f317f914a278ea91"}
08:19:57 [08:19:57.018]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:f317f914a278ea91 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:f317f914a278ea91"}
08:19:57 [08:19:57.024]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:f317f914a278ea91 epochNumber=1. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:f317f914a278ea91"}
08:19:57 [08:19:57.029]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.029]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:386d4871a8b42524 epochNumber=1
{"provingJobId":"1:BASE_PARITY:386d4871a8b42524"}
08:19:57 [08:19:57.030]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test20En2b/1 with map size 1073741824
08:19:57 [08:19:57.032]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.053]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.055]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.056]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2308d21fffc9034c epochNumber=1
{"provingJobId":"1:BASE_PARITY:2308d21fffc9034c"}
08:19:57 [08:19:57.056]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLTSrnb/1 with map size 1073741824
08:19:57 [08:19:57.059]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.076]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:2308d21fffc9034c
{"provingJobId":"1:BASE_PARITY:2308d21fffc9034c"}
08:19:57 [08:19:57.076]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2308d21fffc9034c type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2308d21fffc9034c"}
08:19:57 [08:19:57.077]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2308d21fffc9034c type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:2308d21fffc9034c"}
08:19:57 [08:19:57.086]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.087]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b890c75544b02218 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b890c75544b02218"}
08:19:57 [08:19:57.088]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhe9bjd/1 with map size 1073741824
08:19:57 [08:19:57.090]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.107]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:b890c75544b02218
{"provingJobId":"1:BASE_PARITY:b890c75544b02218"}
08:19:57 [08:19:57.108]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b890c75544b02218 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:b890c75544b02218"}
08:19:57 [08:19:57.117]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.117]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1414160f7715228a epochNumber=1
{"provingJobId":"1:BASE_PARITY:1414160f7715228a"}
08:19:57 [08:19:57.118]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test9JiBhM/1 with map size 1073741824
08:19:57 [08:19:57.120]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.136]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1414160f7715228a type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:1414160f7715228a"}
08:19:57 [08:19:57.137]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1414160f7715228a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:1414160f7715228a"}
08:19:57 [08:19:57.144]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.145]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8841cfb28a4025cf epochNumber=1
{"provingJobId":"1:BASE_PARITY:8841cfb28a4025cf"}
08:19:57 [08:19:57.145]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testt4zeSk/1 with map size 1073741824
08:19:57 [08:19:57.147]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.164]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8841cfb28a4025cf type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8841cfb28a4025cf"}
08:19:57 [08:19:57.165]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8841cfb28a4025cf type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:8841cfb28a4025cf"}
08:19:57 [08:19:57.173]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.174]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a217bbec077719ae epochNumber=0
{"provingJobId":"1:BASE_PARITY:a217bbec077719ae"}
08:19:57 [08:19:57.174]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testKgloey/0 with map size 1073741824
08:19:57 [08:19:57.176]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.195]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a217bbec077719ae epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a217bbec077719ae"}
08:19:57 [08:19:57.196]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:bff2ec91a3909647 epochNumber=0
{"provingJobId":"1:BASE_PARITY:bff2ec91a3909647"}
08:19:57 [08:19:57.196]
WARN:
prover-client:proving-broker Cached proving job id=1:BASE_PARITY:a217bbec077719ae epochNumber=0. Not enqueuing again
{"provingJobId":"1:BASE_PARITY:a217bbec077719ae"}
08:19:57 [08:19:57.197]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b23e1734a79a1511 epochNumber=0
{"provingJobId":"1:BASE_PARITY:b23e1734a79a1511"}
08:19:57 [08:19:57.218]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.220]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.221]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:617543e2c55bb631 epochNumber=2
{"provingJobId":"1:BASE_PARITY:617543e2c55bb631"}
08:19:57 [08:19:57.221]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testsPAx4A/2 with map size 1073741824
08:19:57 [08:19:57.223]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.240]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0f473f9ba1b4a4e7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:0f473f9ba1b4a4e7"}
08:19:57 [08:19:57.241]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsPAx4A/1 with map size 1073741824
08:19:57 [08:19:57.243]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.261]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.262]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:827c587f5b677cf3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:827c587f5b677cf3"}
08:19:57 [08:19:57.262]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhZXT5J/1 with map size 1073741824
08:19:57 [08:19:57.266]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.296]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.297]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:75d89dea9c4ce7a9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:75d89dea9c4ce7a9"}
08:19:57 [08:19:57.298]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testmped9d/1 with map size 1073741824
08:19:57 [08:19:57.300]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.320]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:cb62f0acaffc87f7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:cb62f0acaffc87f7"}
08:19:57 [08:19:57.327]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:409477d18008f35f epochNumber=2
{"provingJobId":"1:BASE_PARITY:409477d18008f35f"}
08:19:57 [08:19:57.328]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testmped9d/2 with map size 1073741824
08:19:57 [08:19:57.332]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.350]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:214b0167d12d6204 epochNumber=1
{"provingJobId":"1:BASE_PARITY:214b0167d12d6204"}
08:19:57 [08:19:57.359]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.360]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0599f732a5625cdd epochNumber=1
{"provingJobId":"1:BASE_PARITY:0599f732a5625cdd"}
08:19:57 [08:19:57.361]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testN8lEkX/1 with map size 1073741824
08:19:57 [08:19:57.363]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.383]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c897aa85c9d557ad epochNumber=1
{"provingJobId":"1:BASE_PARITY:c897aa85c9d557ad"}
08:19:57 [08:19:57.389]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a105f03db20e6d39 epochNumber=2
{"provingJobId":"1:BASE_PARITY:a105f03db20e6d39"}
08:19:57 [08:19:57.390]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testN8lEkX/2 with map size 1073741824
08:19:57 [08:19:57.392]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.409]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:22b7da259704624f epochNumber=1
{"provingJobId":"1:BASE_PARITY:22b7da259704624f"}
08:19:57 [08:19:57.419]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.420]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:dae497ab5b918a02 epochNumber=1
{"provingJobId":"1:BASE_PARITY:dae497ab5b918a02"}
08:19:57 [08:19:57.420]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrwdNTS/1 with map size 1073741824
08:19:57 [08:19:57.423]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.439]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9520c6172961ece5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:9520c6172961ece5"}
08:19:57 [08:19:57.445]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:ddec3fc3ba1501ea epochNumber=2
{"provingJobId":"1:BASE_PARITY:ddec3fc3ba1501ea"}
08:19:57 [08:19:57.446]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testrwdNTS/2 with map size 1073741824
08:19:57 [08:19:57.452]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.466]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:51e698bfaf680c82 epochNumber=1
{"provingJobId":"1:BASE_PARITY:51e698bfaf680c82"}
08:19:57 [08:19:57.475]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.476]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:0230a8d7094f452a epochNumber=1
{"provingJobId":"1:BASE_PARITY:0230a8d7094f452a"}
08:19:57 [08:19:57.477]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjg2Crt/1 with map size 1073741824
08:19:57 [08:19:57.479]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.496]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c4f2d2faf4bba802 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c4f2d2faf4bba802"}
08:19:57 [08:19:57.502]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:0230a8d7094f452a type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:0230a8d7094f452a"}
08:19:57 [08:19:57.511]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.512]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:35c5b5137167627a epochNumber=1
{"provingJobId":"1:BASE_PARITY:35c5b5137167627a"}
08:19:57 [08:19:57.512]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdTCOQG/1 with map size 1073741824
08:19:57 [08:19:57.513]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.529]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:66812e99d8b3815c epochNumber=1
{"provingJobId":"1:BASE_PARITY:66812e99d8b3815c"}
08:19:57 [08:19:57.535]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:35c5b5137167627a type=BASE_PARITY totalAttempts=1 err=result
{"provingJobId":"1:BASE_PARITY:35c5b5137167627a"}
08:19:57 [08:19:57.543]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.544]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:d6ec09abb79ddcb5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:d6ec09abb79ddcb5"}
08:19:57 [08:19:57.545]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6NKsvz/1 with map size 1073741824
08:19:57 [08:19:57.546]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.566]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:6c18f2fc33a9819e epochNumber=1
{"provingJobId":"1:BASE_PARITY:6c18f2fc33a9819e"}
08:19:57 [08:19:57.572]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d6ec09abb79ddcb5 type=BASE_PARITY retry=1 err=result
{"provingJobId":"1:BASE_PARITY:d6ec09abb79ddcb5"}
08:19:57 [08:19:57.577]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.577]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1aaf42592fe01409 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1aaf42592fe01409"}
08:19:57 [08:19:57.578]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkOKHoy/1 with map size 1073741824
08:19:57 [08:19:57.580]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.599]
INFO:
prover-client:proving-broker Cancelling job id=1:BASE_PARITY:1aaf42592fe01409
{"provingJobId":"1:BASE_PARITY:1aaf42592fe01409"}
08:19:57 [08:19:57.599]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1aaf42592fe01409 type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"1:BASE_PARITY:1aaf42592fe01409"}
08:19:57 [08:19:57.605]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c25fcecb891f5b96 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c25fcecb891f5b96"}
08:19:57 [08:19:57.612]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1aaf42592fe01409 has already been completed
{"provingJobId":"1:BASE_PARITY:1aaf42592fe01409"}
08:19:57 [08:19:57.615]
INFO:
prover-client:proving-broker Proving Broker started
08:19:57 [08:19:57.615]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f7d8424fb8cbc3d3 epochNumber=1
{"provingJobId":"1:BASE_PARITY:f7d8424fb8cbc3d3"}
08:19:57 [08:19:57.617]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testX86EZP/1 with map size 1073741824
08:19:57 [08:19:57.619]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:57 [08:19:57.635]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:9a46a71dd35cb695 epochNumber=2
{"provingJobId":"1:BASE_PARITY:9a46a71dd35cb695"}
08:19:57 [08:19:57.636]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testX86EZP/2 with map size 1073741824
08:19:57 [08:19:57.639]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:58 [08:19:58.709]
INFO:
prover-client:proving-broker Proving Broker started
08:19:58 [08:19:58.715]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f7d8424fb8cbc3d3 settled=false
{"provingJobId":"1:BASE_PARITY:f7d8424fb8cbc3d3","status":"pending"}
08:19:58 [08:19:58.717]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9a46a71dd35cb695 settled=false
{"provingJobId":"1:BASE_PARITY:9a46a71dd35cb695","status":"pending"}
08:19:58 [08:19:58.717]
INFO:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f7d8424fb8cbc3d3 type=BASE_PARITY startedAt=1750839597657 older agent has taken job
{"provingJobId":"1:BASE_PARITY:f7d8424fb8cbc3d3"}
08:19:58 [08:19:58.718]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:f7d8424fb8cbc3d3 type=BASE_PARITY already being worked on by another agent. Sending new one
{"provingJobId":"1:BASE_PARITY:f7d8424fb8cbc3d3"}
08:19:58 [08:19:58.721]
INFO:
prover-client:proving-broker Proving Broker started
08:19:58 [08:19:58.723]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1bd121c4f6c15cb9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:1bd121c4f6c15cb9"}
08:19:58 [08:19:58.724]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6OZ6l8/1 with map size 1073741824
08:19:58 [08:19:58.728]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:58 [08:19:58.742]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:221386a3cd6e34a6 epochNumber=2
{"provingJobId":"1:BASE_PARITY:221386a3cd6e34a6"}
08:19:58 [08:19:58.743]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test6OZ6l8/2 with map size 1073741824
08:19:58 [08:19:58.745]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:59 [08:19:59.763]
INFO:
prover-client:proving-broker Proving Broker started
08:19:59 [08:19:59.765]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1bd121c4f6c15cb9 settled=false
{"provingJobId":"1:BASE_PARITY:1bd121c4f6c15cb9","status":"pending"}
08:19:59 [08:19:59.766]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:221386a3cd6e34a6 settled=false
{"provingJobId":"1:BASE_PARITY:221386a3cd6e34a6","status":"pending"}
08:19:59 [08:19:59.767]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1bd121c4f6c15cb9 type=BASE_PARITY not found in the in-progress cache, adding it
{"provingJobId":"1:BASE_PARITY:1bd121c4f6c15cb9"}
08:19:59 [08:19:59.771]
INFO:
prover-client:proving-broker Proving Broker started
08:19:59 [08:19:59.772]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:007d06c38f4d2bff epochNumber=1
{"provingJobId":"1:BASE_PARITY:007d06c38f4d2bff"}
08:19:59 [08:19:59.775]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testRYiomZ/1 with map size 1073741824
08:19:59 [08:19:59.777]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:19:59 [08:19:59.797]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1cd81196e0c1675f epochNumber=2
{"provingJobId":"1:BASE_PARITY:1cd81196e0c1675f"}
08:19:59 [08:19:59.798]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testRYiomZ/2 with map size 1073741824
08:19:59 [08:19:59.801]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:00 [08:20:00.821]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.823]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:007d06c38f4d2bff settled=false
{"provingJobId":"1:BASE_PARITY:007d06c38f4d2bff","status":"pending"}
08:20:00 [08:20:00.825]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1cd81196e0c1675f settled=false
{"provingJobId":"1:BASE_PARITY:1cd81196e0c1675f","status":"pending"}
08:20:00 [08:20:00.826]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:007d06c38f4d2bff type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:007d06c38f4d2bff"}
08:20:00 [08:20:00.826]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:007d06c38f4d2bff type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:007d06c38f4d2bff"}
08:20:00 [08:20:00.837]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.838]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:726b02fc2645c34d epochNumber=1
{"provingJobId":"1:BASE_PARITY:726b02fc2645c34d"}
08:20:00 [08:20:00.839]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBSUGBX/1 with map size 1073741824
08:20:00 [08:20:00.841]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:00 [08:20:00.860]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:02fbe229c0982bf5 epochNumber=2
{"provingJobId":"1:BASE_PARITY:02fbe229c0982bf5"}
08:20:00 [08:20:00.861]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBSUGBX/2 with map size 1073741824
08:20:00 [08:20:00.867]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:00 [08:20:00.896]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:726b02fc2645c34d type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:726b02fc2645c34d"}
08:20:00 [08:20:00.903]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:02fbe229c0982bf5 type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:02fbe229c0982bf5"}
08:20:00 [08:20:00.914]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.915]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:38b9d02ab29d0ff9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:38b9d02ab29d0ff9"}
08:20:00 [08:20:00.915]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test5Txqi1/1 with map size 1073741824
08:20:00 [08:20:00.917]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:00 [08:20:00.934]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:a027b209a2d1bc5e epochNumber=2
{"provingJobId":"1:BASE_PARITY:a027b209a2d1bc5e"}
08:20:00 [08:20:00.934]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test5Txqi1/2 with map size 1073741824
08:20:00 [08:20:00.936]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:00 [08:20:00.953]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:38b9d02ab29d0ff9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:38b9d02ab29d0ff9"}
08:20:00 [08:20:00.954]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:38b9d02ab29d0ff9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:38b9d02ab29d0ff9"}
08:20:00 [08:20:00.960]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:a027b209a2d1bc5e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:a027b209a2d1bc5e"}
08:20:00 [08:20:00.970]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.971]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:7ca1a5d7c82a7f12 err=test error
{"provingJoId":"1:BASE_PARITY:7ca1a5d7c82a7f12"}
08:20:00 [08:20:00.976]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.978]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:a990e3c0e12f6731 not found
{"provingJobId":"1:BASE_PARITY:a990e3c0e12f6731"}
08:20:00 [08:20:00.983]
INFO:
prover-client:proving-broker Proving Broker started
08:20:00 [08:20:00.984]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1de447ffdaf7360f epochNumber=1
{"provingJobId":"1:BASE_PARITY:1de447ffdaf7360f"}
08:20:00 [08:20:00.986]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testMivjN7/1 with map size 1073741824
08:20:00 [08:20:00.991]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.012]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.013]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:3c1f52b54e778408 epochNumber=1
{"provingJobId":"1:BASE_PARITY:3c1f52b54e778408"}
08:20:01 [08:20:01.014]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBkCJYO/1 with map size 1073741824
08:20:01 [08:20:01.015]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.139]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:3c1f52b54e778408 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:3c1f52b54e778408"}
08:20:01 [08:20:01.144]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.145]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:5544bf7aa80e2108 epochNumber=1
{"provingJobId":"1:BASE_PARITY:5544bf7aa80e2108"}
08:20:01 [08:20:01.146]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhrkjrw/1 with map size 1073741824
08:20:01 [08:20:01.148]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.271]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:5544bf7aa80e2108 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:5544bf7aa80e2108"}
08:20:01 [08:20:01.274]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:aa4e68c533cf8cde epochNumber=10
{"provingJobId":"1:BASE_PARITY:aa4e68c533cf8cde"}
08:20:01 [08:20:01.280]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testhrkjrw/10 with map size 1073741824
08:20:01 [08:20:01.285]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.323]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:20:01 [08:20:01.324]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
08:20:01 [08:20:01.432]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.433]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:01e31f47b7daff50 epochNumber=1
{"provingJobId":"1:BASE_PARITY:01e31f47b7daff50"}
08:20:01 [08:20:01.433]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test7hJ6SB/1 with map size 1073741824
08:20:01 [08:20:01.435]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.620]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:01e31f47b7daff50 timed out. Adding it back to the queue.
{"provingJobId":"1:BASE_PARITY:01e31f47b7daff50"}
08:20:01 [08:20:01.659]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.660]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8e3dec28efca5ad5 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8e3dec28efca5ad5"}
08:20:01 [08:20:01.660]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testp3hIBo/1 with map size 1073741824
08:20:01 [08:20:01.662]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.686]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8e3dec28efca5ad5 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:8e3dec28efca5ad5"}
08:20:01 [08:20:01.691]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.692]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:f5ca64bce70fa7fc epochNumber=1
{"provingJobId":"1:BASE_PARITY:f5ca64bce70fa7fc"}
08:20:01 [08:20:01.698]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testby9FSn/1 with map size 1073741824
08:20:01 [08:20:01.701]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.718]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:f5ca64bce70fa7fc type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:f5ca64bce70fa7fc"}
08:20:01 [08:20:01.720]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f5ca64bce70fa7fc type=BASE_PARITY totalAttempts=2 err=test error
{"provingJobId":"1:BASE_PARITY:f5ca64bce70fa7fc"}
08:20:01 [08:20:01.730]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.731]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:473a240be99c025d epochNumber=1
{"provingJobId":"1:BASE_PARITY:473a240be99c025d"}
08:20:01 [08:20:01.733]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl7RhBh/1 with map size 1073741824
08:20:01 [08:20:01.735]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.755]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:473a240be99c025d type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:473a240be99c025d"}
08:20:01 [08:20:01.764]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.765]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:b374658be83dc7d6 epochNumber=1
{"provingJobId":"1:BASE_PARITY:b374658be83dc7d6"}
08:20:01 [08:20:01.765]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testAT3myX/1 with map size 1073741824
08:20:01 [08:20:01.771]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.797]
INFO:
prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:b374658be83dc7d6 type=BASE_PARITY retry=1 err=test error
{"provingJobId":"1:BASE_PARITY:b374658be83dc7d6"}
08:20:01 [08:20:01.799]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:1fddbdeea33ad424 epochNumber=3
{"provingJobId":"1:BASE_PARITY:1fddbdeea33ad424"}
08:20:01 [08:20:01.800]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testAT3myX/3 with map size 1073741824
08:20:01 [08:20:01.804]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.841]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:20:01 [08:20:01.842]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
08:20:01 [08:20:01.848]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:b374658be83dc7d6 err=test error
{"provingJoId":"1:BASE_PARITY:b374658be83dc7d6"}
08:20:01 [08:20:01.854]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test18XIct/1 with map size 1073741824
08:20:01 [08:20:01.857]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.884]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test18XIct/2 with map size 1073741824
08:20:01 [08:20:01.887]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.907]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.909]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2d5fbd8ffa603bde settled=false
{"provingJobId":"1:BASE_PARITY:2d5fbd8ffa603bde","status":"pending"}
08:20:01 [08:20:01.911]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c7639d8f373a309b settled=false
{"provingJobId":"1:BASE_PARITY:c7639d8f373a309b","status":"pending"}
08:20:01 [08:20:01.918]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testycIXwA/1 with map size 1073741824
08:20:01 [08:20:01.921]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.941]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testycIXwA/2 with map size 1073741824
08:20:01 [08:20:01.944]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:01 [08:20:01.994]
INFO:
prover-client:proving-broker Proving Broker started
08:20:01 [08:20:01.997]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:720f343e6c36da73 settled=true
{"provingJobId":"1:BASE_PARITY:720f343e6c36da73","status":"fulfilled"}
08:20:01 [08:20:01.998]
INFO:
prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:b9cb04aefb1f5a7f settled=true
{"provingJobId":"2:BASE_PARITY:b9cb04aefb1f5a7f","status":"fulfilled"}
08:20:02 [08:20:02.006]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test18xDBC/1 with map size 1073741824
08:20:02 [08:20:02.009]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.041]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test18xDBC/2 with map size 1073741824
08:20:02 [08:20:02.043]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.064]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.066]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8f6676b4c9fcf8a3 settled=true
{"provingJobId":"1:BASE_PARITY:8f6676b4c9fcf8a3","status":"fulfilled"}
08:20:02 [08:20:02.070]
INFO:
prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:659849cbf2fb2451 settled=false
{"provingJobId":"1:BASE_PARITY:659849cbf2fb2451","status":"pending"}
08:20:02 [08:20:02.080]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.081]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c21261223b602977 epochNumber=1
{"provingJobId":"1:BASE_PARITY:c21261223b602977"}
08:20:02 [08:20:02.082]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test1AobJn/1 with map size 1073741824
08:20:02 [08:20:02.092]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.116]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.117]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:55f3b977e7cb9085 epochNumber=1
{"provingJobId":"1:BASE_PARITY:55f3b977e7cb9085"}
08:20:02 [08:20:02.118]
ERROR:
prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:55f3b977e7cb9085: Error: db error: Error: db error
08:20:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63)
{"provingJobId":"1:BASE_PARITY:55f3b977e7cb9085"}
08:20:02 [08:20:02.121]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.121]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:8be8df671549a0e7 epochNumber=1
{"provingJobId":"1:BASE_PARITY:8be8df671549a0e7"}
08:20:02 [08:20:02.122]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrXb91A/1 with map size 1073741824
08:20:02 [08:20:02.124]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.144]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:8be8df671549a0e7 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:8be8df671549a0e7"}
08:20:02 [08:20:02.145]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8be8df671549a0e7 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:8be8df671549a0e7"}
08:20:02 [08:20:02.157]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.158]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:2e8318ab91f7dc99 epochNumber=1
{"provingJobId":"1:BASE_PARITY:2e8318ab91f7dc99"}
08:20:02 [08:20:02.159]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0aXYmJ/1 with map size 1073741824
08:20:02 [08:20:02.163]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.183]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:2e8318ab91f7dc99 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:2e8318ab91f7dc99"}
08:20:02 [08:20:02.184]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2e8318ab91f7dc99 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:2e8318ab91f7dc99"}
08:20:02 [08:20:02.185]
ERROR:
prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:2e8318ab91f7dc99: Error: db error
08:20:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69)
{"provingJobId":"1:BASE_PARITY:2e8318ab91f7dc99"}
08:20:02 [08:20:02.192]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.200]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:11bf5e2e781461ad epochNumber=1
{"provingJobId":"1:BASE_PARITY:11bf5e2e781461ad"}
08:20:02 [08:20:02.203]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testaz5tKJ/1 with map size 1073741824
08:20:02 [08:20:02.210]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.234]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:11bf5e2e781461ad type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:11bf5e2e781461ad"}
08:20:02 [08:20:02.234]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:11bf5e2e781461ad type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:11bf5e2e781461ad"}
08:20:02 [08:20:02.249]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.250]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:c9e0dc6d917cbb6e epochNumber=1
{"provingJobId":"1:BASE_PARITY:c9e0dc6d917cbb6e"}
08:20:02 [08:20:02.251]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testuQ8sQm/1 with map size 1073741824
08:20:02 [08:20:02.254]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.276]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:c9e0dc6d917cbb6e type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:c9e0dc6d917cbb6e"}
08:20:02 [08:20:02.277]
INFO:
prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c9e0dc6d917cbb6e type=BASE_PARITY totalAttempts=1 err=test error
{"provingJobId":"1:BASE_PARITY:c9e0dc6d917cbb6e"}
08:20:02 [08:20:02.278]
ERROR:
prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:c9e0dc6d917cbb6e jobErr=test error: Error: db error
08:20:02 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68)
{"provingJobId":"1:BASE_PARITY:c9e0dc6d917cbb6e"}
08:20:02 [08:20:02.290]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.291]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:1bb3e22236246ea4 not found
{"provingJobId":"1:BASE_PARITY:1bb3e22236246ea4"}
08:20:02 [08:20:02.295]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.296]
WARN:
prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:278ceec4eb62c89c err=test error
{"provingJoId":"1:BASE_PARITY:278ceec4eb62c89c"}
08:20:02 [08:20:02.301]
INFO:
prover-client:proving-broker Proving Broker started
08:20:02 [08:20:02.312]
INFO:
prover-client:proving-broker New proving job id=1:BASE_PARITY:96ef53b1e380ecf9 epochNumber=1
{"provingJobId":"1:BASE_PARITY:96ef53b1e380ecf9"}
08:20:02 [08:20:02.313]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testpAEcnX/1 with map size 1073741824
08:20:02 [08:20:02.325]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.349]
WARN:
prover-client:proving-broker Proving job id=1:BASE_PARITY:96ef53b1e380ecf9 type=BASE_PARITY not in the in-progress set
{"provingJobId":"1:BASE_PARITY:96ef53b1e380ecf9"}
08:20:02 [08:20:02.349]
INFO:
prover-client:proving-broker Proving job complete id=1:BASE_PARITY:96ef53b1e380ecf9 type=BASE_PARITY totalAttempts=1
{"provingJobId":"1:BASE_PARITY:96ef53b1e380ecf9"}
08:20:02 [08:20:02.366]
INFO:
prover-client:proving-broker New proving job id=2:BASE_PARITY:fb51ec58f62c792a epochNumber=2
{"provingJobId":"2:BASE_PARITY:fb51ec58f62c792a"}
08:20:02 [08:20:02.368]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testpAEcnX/2 with map size 1073741824
08:20:02 [08:20:02.375]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.401]
WARN:
prover-client:proving-broker Proving job id=2:BASE_PARITY:fb51ec58f62c792a type=BASE_PARITY not in the in-progress set
{"provingJobId":"2:BASE_PARITY:fb51ec58f62c792a"}
08:20:02 [08:20:02.402]
INFO:
prover-client:proving-broker Proving job complete id=2:BASE_PARITY:fb51ec58f62c792a type=BASE_PARITY totalAttempts=1
{"provingJobId":"2:BASE_PARITY:fb51ec58f62c792a"}
08:20:02 [08:20:02.470]
INFO:
prover-client:proving-broker New proving job id=3:BASE_PARITY:86d99c8b87e38072 epochNumber=3
{"provingJobId":"3:BASE_PARITY:86d99c8b87e38072"}
08:20:02 [08:20:02.471]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testpAEcnX/3 with map size 1073741824
08:20:02 [08:20:02.475]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.509]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:20:02 [08:20:02.509]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 1
08:20:02 [08:20:02.560]
INFO:
prover-client:proving-broker New proving job id=4:BASE_PARITY:b5fc24843742feed epochNumber=4
{"provingJobId":"4:BASE_PARITY:b5fc24843742feed"}
08:20:02 [08:20:02.561]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testpAEcnX/4 with map size 1073741824
08:20:02 [08:20:02.563]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.589]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:20:02 [08:20:02.590]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 2
08:20:02 [08:20:02.644]
INFO:
prover-client:proving-broker New proving job id=5:BASE_PARITY:177afac2212f5353 epochNumber=5
{"provingJobId":"5:BASE_PARITY:177afac2212f5353"}
08:20:02 [08:20:02.645]
INFO:
prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testpAEcnX/5 with map size 1073741824
08:20:02 [08:20:02.647]
INFO:
kv-store:lmdb-v2 Starting data store with maxReaders 16
08:20:02 [08:20:02.669]
INFO:
prover-client:proving-broker Cleaned up jobs=1
08:20:02 [08:20:02.670]
INFO:
prover-client:proving-broker-database Deleting broker database for epoch 3
08:20:02
PASS src/proving_broker/proving_broker.test.ts (
13.257 s)
08:20:02 ProvingBroker
08:20:02 Producer API
08:20:02
✓ refuses stale jobs (58 ms)
08:20:02
✓ enqueues jobs (5 ms)
08:20:02
✓ ignores duplicate jobs (6 ms)
08:20:02
✓ reports correct status when enqueuing repeat jobs (6 ms)
08:20:02
✓ throws an error in case of duplicate job IDs (30 ms)
08:20:02
✓ returns not-found status for non-existing jobs (2 ms)
08:20:02
✓ cancels jobs in queue (3 ms)
08:20:02
✓ cancels jobs in-progress (2 ms)
08:20:02
✓ returns job result if successful (13 ms)
08:20:02
✓ returns job error if failed (3 ms)
08:20:02
✓ correctly returns job status for concurrent writes (4 ms)
08:20:02
✓ refuses stale jobs (36 ms)
08:20:02
✓ enqueues jobs (31 ms)
08:20:02
✓ ignores duplicate jobs (26 ms)
08:20:02
✓ reports correct status when enqueuing repeat jobs (30 ms)
08:20:02
✓ throws an error in case of duplicate job IDs (26 ms)
08:20:02
✓ returns not-found status for non-existing jobs (2 ms)
08:20:02
✓ cancels jobs in queue (30 ms)
08:20:02
✓ cancels jobs in-progress (29 ms)
08:20:02
✓ returns job result if successful (28 ms)
08:20:02
✓ returns job error if failed (28 ms)
08:20:02
✓ correctly returns job status for concurrent writes (45 ms)
08:20:02 Consumer API
08:20:02
✓ returns undefined if no jobs are available (2 ms)
08:20:02
✓ returns jobs in priority order (2 ms)
08:20:02
✓ returns undefined if no jobs are available for the given allowList (1 ms)
08:20:02
✓ returns a job if it is in the allowList (3 ms)
08:20:02
✓ returns the most important job if it is in the allowList (3 ms)
08:20:02
✓ returns any job if filter is empty (2 ms)
08:20:02
✓ returns a new job when reporting job success (2 ms)
08:20:02
✓ returns a new job when reporting permanent error (2 ms)
08:20:02
✓ returns a new job when reporting retry-able error (2 ms)
08:20:02
✓ returns a new job when reporting progress if current one is cancelled (3 ms)
08:20:02
✓ returns a new job if job is already in progress elsewhere (1056 ms)
08:20:02
✓ avoids sending the same job to a new agent after a restart (1005 ms)
08:20:02
✓ avoids sending a completed job to a new agent after a restart (1004 ms)
08:20:02
✓ tracks job result if in progress (3 ms)
08:20:02
✓ tracks job result even if job is in queue (3 ms)
08:20:02
✓ ignores reported job error if unknown job (1 ms)
08:20:02
✓ ignores job result if unknown job (1 ms)
08:20:02
✓ returns undefined if no jobs are available (2 ms)
08:20:02
✓ returns jobs in priority order (40 ms)
08:20:02
✓ returns undefined if no jobs are available for the given allowList (34 ms)
08:20:02
✓ returns a job if it is in the allowList (62 ms)
08:20:02
✓ returns the most important job if it is in the allowList (59 ms)
08:20:02
✓ returns any job if filter is empty (56 ms)
08:20:02
✓ returns a new job when reporting job success (36 ms)
08:20:02
✓ returns a new job when reporting permanent error (32 ms)
08:20:02
✓ returns a new job when reporting retry-able error (32 ms)
08:20:02
✓ returns a new job when reporting progress if current one is cancelled (38 ms)
08:20:02
✓ returns a new job if job is already in progress elsewhere (1106 ms)
08:20:02
✓ avoids sending the same job to a new agent after a restart (1049 ms)
08:20:02
✓ avoids sending a completed job to a new agent after a restart (1065 ms)
08:20:02
✓ tracks job result if in progress (77 ms)
08:20:02
✓ tracks job result even if job is in queue (56 ms)
08:20:02
✓ ignores reported job error if unknown job (4 ms)
08:20:02
✓ ignores job result if unknown job (7 ms)
08:20:02 Timeouts
08:20:02
✓ tracks in progress jobs (2 ms)
08:20:02
✓ re-enqueues jobs that time out (108 ms)
08:20:02
✓ cancel stale jobs that time out (235 ms)
08:20:02
✓ keeps the jobs in progress while it is alive (205 ms)
08:20:02
✓ tracks in progress jobs (30 ms)
08:20:02
✓ re-enqueues jobs that time out (131 ms)
08:20:02
✓ cancel stale jobs that time out (287 ms)
08:20:02
✓ keeps the jobs in progress while it is alive (226 ms)
08:20:02 Retries
08:20:02
✓ retries jobs (3 ms)
08:20:02
✓ retries up to a maximum number of times (3 ms)
08:20:02
✓ passing retry=false does not retry (3 ms)
08:20:02
✓ does not retry if job is stale (28 ms)
08:20:02
✓ retries jobs (32 ms)
08:20:02
✓ retries up to a maximum number of times (38 ms)
08:20:02
✓ passing retry=false does not retry (34 ms)
08:20:02
✓ does not retry if job is stale (88 ms)
08:20:02 Database management
08:20:02
✓ re-enqueues proof requests on start (2 ms)
08:20:02
✓ restores proof results on start (2 ms)
08:20:02
✓ only re-enqueues unfinished jobs (1 ms)
08:20:02
✓ saves job when enqueued (3 ms)
08:20:02
✓ does not retain job if database fails to save (2 ms)
08:20:02
✓ saves job result (2 ms)
08:20:02
✓ saves result even if database fails to save (2 ms)
08:20:02
✓ saves job error (3 ms)
08:20:02
✓ saves job error even if database fails to save (2 ms)
08:20:02
✓ does not save job result if job is unknown (1 ms)
08:20:02
✓ does not save job error if job is unknown (1 ms)
08:20:02
✓ cleans up old jobs periodically (263 ms)
08:20:02
✓ re-enqueues proof requests on start (62 ms)
08:20:02
✓ restores proof results on start (87 ms)
08:20:02
✓ only re-enqueues unfinished jobs (75 ms)
08:20:02
✓ saves job when enqueued (34 ms)
08:20:02
✓ does not retain job if database fails to save (7 ms)
08:20:02
✓ saves job result (34 ms)
08:20:02
✓ saves result even if database fails to save (33 ms)
08:20:02
✓ saves job error (57 ms)
08:20:02
✓ saves job error even if database fails to save (37 ms)
08:20:02
✓ does not save job result if job is unknown (9 ms)
08:20:02
✓ does not save job error if job is unknown (5 ms)
08:20:02
✓ cleans up old jobs periodically (424 ms)
08:20:02
08:20:02
Test Suites: 1 passed, 1 total
08:20:02
Tests: 96 passed, 96 total
08:20:02
Snapshots: 0 total
08:20:02
Time: 13.344 s
08:20:02
Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts
.
08:20:02
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?