Parent Log: http://ci.aztec-labs.com/78cea4cf271fb3a9 Command: 128a905c1f9c1691 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/76ca48a2187e3506bb464eae574e49476c2876ca Env: REF_NAME=gh-readonly-queue/next/pr-15045-9d6fb0bd1594726370a5f3a56591963452264642 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Fri Jun 13 18:05:35 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15045_amd64_x2-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 18:05:39 [18:05:39.092] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.097] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:279ad33b670fdacb epochNumber=42 {"provingJobId":"1:BASE_PARITY:279ad33b670fdacb"} 18:05:39 [18:05:39.100] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:9ecd45500b7cbab9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9ecd45500b7cbab9"} 18:05:39 [18:05:39.136] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.137] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c6673b34ea2cff81 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c6673b34ea2cff81"} 18:05:39 [18:05:39.138] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cf86a726dee2dfa1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cf86a726dee2dfa1"} 18:05:39 [18:05:39.140] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.141] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8985d1ac04d6d962 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8985d1ac04d6d962"} 18:05:39 [18:05:39.142] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:8985d1ac04d6d962 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:8985d1ac04d6d962"} 18:05:39 [18:05:39.143] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.144] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a075a182873b6a1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a075a182873b6a1"} 18:05:39 [18:05:39.145] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3a075a182873b6a1 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3a075a182873b6a1"} 18:05:39 [18:05:39.145] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3a075a182873b6a1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3a075a182873b6a1"} 18:05:39 [18:05:39.146] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:3a075a182873b6a1 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:3a075a182873b6a1"} 18:05:39 [18:05:39.147] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.147] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:73fa021629b4adc7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:73fa021629b4adc7"} 18:05:39 [18:05:39.166] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.168] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.168] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ff41041957d92a66 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ff41041957d92a66"} 18:05:39 [18:05:39.169] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:ff41041957d92a66 {"provingJobId":"1:BASE_PARITY:ff41041957d92a66"} 18:05:39 [18:05:39.169] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ff41041957d92a66 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ff41041957d92a66"} 18:05:39 [18:05:39.169] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ff41041957d92a66 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:ff41041957d92a66"} 18:05:39 [18:05:39.170] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.171] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:efc875977ba4096a epochNumber=1 {"provingJobId":"1:BASE_PARITY:efc875977ba4096a"} 18:05:39 [18:05:39.171] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:efc875977ba4096a {"provingJobId":"1:BASE_PARITY:efc875977ba4096a"} 18:05:39 [18:05:39.172] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:efc875977ba4096a type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:efc875977ba4096a"} 18:05:39 [18:05:39.173] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.174] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d7965954a19f3f9d epochNumber=1 {"provingJobId":"1:BASE_PARITY:d7965954a19f3f9d"} 18:05:39 [18:05:39.174] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d7965954a19f3f9d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:d7965954a19f3f9d"} 18:05:39 [18:05:39.174] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:d7965954a19f3f9d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:d7965954a19f3f9d"} 18:05:39 [18:05:39.175] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.176] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:437f2171ee57b8ef epochNumber=1 {"provingJobId":"1:BASE_PARITY:437f2171ee57b8ef"} 18:05:39 [18:05:39.176] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:437f2171ee57b8ef type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:437f2171ee57b8ef"} 18:05:39 [18:05:39.176] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:437f2171ee57b8ef type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:437f2171ee57b8ef"} 18:05:39 [18:05:39.177] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.178] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:937179d44648ab4d epochNumber=0 {"provingJobId":"1:BASE_PARITY:937179d44648ab4d"} 18:05:39 [18:05:39.178] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:937179d44648ab4d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:937179d44648ab4d"} 18:05:39 [18:05:39.178] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7daa6f99bb90eedf epochNumber=0 {"provingJobId":"1:BASE_PARITY:7daa6f99bb90eedf"} 18:05:39 [18:05:39.179] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:937179d44648ab4d epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:937179d44648ab4d"} 18:05:39 [18:05:39.179] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1bd173db60591987 epochNumber=0 {"provingJobId":"1:BASE_PARITY:1bd173db60591987"} 18:05:39 [18:05:39.180] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.181] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.181] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d26e0d5c4babc3a1 epochNumber=2 {"provingJobId":"1:BASE_PARITY:d26e0d5c4babc3a1"} 18:05:39 [18:05:39.182] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fa5b2b543738afa8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fa5b2b543738afa8"} 18:05:39 [18:05:39.183] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.183] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ba8448cf203c1804 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ba8448cf203c1804"} 18:05:39 [18:05:39.184] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.184] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dab55f5ea8f48513 epochNumber=1 {"provingJobId":"1:BASE_PARITY:dab55f5ea8f48513"} 18:05:39 [18:05:39.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d77f3dda5d2bab42 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d77f3dda5d2bab42"} 18:05:39 [18:05:39.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c545a5059c74631d epochNumber=2 {"provingJobId":"1:BASE_PARITY:c545a5059c74631d"} 18:05:39 [18:05:39.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c10185de75727d09 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c10185de75727d09"} 18:05:39 [18:05:39.186] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.187] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2fa2f46b0fe589ce epochNumber=1 {"provingJobId":"1:BASE_PARITY:2fa2f46b0fe589ce"} 18:05:39 [18:05:39.187] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:580ba222cf13ad05 epochNumber=1 {"provingJobId":"1:BASE_PARITY:580ba222cf13ad05"} 18:05:39 [18:05:39.187] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:38ad48c452489f9a epochNumber=2 {"provingJobId":"1:BASE_PARITY:38ad48c452489f9a"} 18:05:39 [18:05:39.188] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab7a2b109412ba02 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ab7a2b109412ba02"} 18:05:39 [18:05:39.195] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2b58faca52031d5c epochNumber=1 {"provingJobId":"1:BASE_PARITY:2b58faca52031d5c"} 18:05:39 [18:05:39.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b7c85f28cf4d3fab epochNumber=1 {"provingJobId":"1:BASE_PARITY:b7c85f28cf4d3fab"} 18:05:39 [18:05:39.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:45a8b1cd9a9cbdb3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:45a8b1cd9a9cbdb3"} 18:05:39 [18:05:39.196] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4dcd90d8aff3da86 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4dcd90d8aff3da86"} 18:05:39 [18:05:39.198] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:40259199eb269711 epochNumber=1 {"provingJobId":"1:BASE_PARITY:40259199eb269711"} 18:05:39 [18:05:39.198] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:84e39ef897c6d0fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:84e39ef897c6d0fb"} 18:05:39 [18:05:39.199] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:40259199eb269711 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:40259199eb269711"} 18:05:39 [18:05:39.200] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c732da4524965355 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c732da4524965355"} 18:05:39 [18:05:39.200] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fa93435bff9a7304 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fa93435bff9a7304"} 18:05:39 [18:05:39.200] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c732da4524965355 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:c732da4524965355"} 18:05:39 [18:05:39.201] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.201] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0d88bd2a3445ac18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0d88bd2a3445ac18"} 18:05:39 [18:05:39.202] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:066c0a7082eea47c epochNumber=1 {"provingJobId":"1:BASE_PARITY:066c0a7082eea47c"} 18:05:39 [18:05:39.202] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:0d88bd2a3445ac18 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:0d88bd2a3445ac18"} 18:05:39 [18:05:39.203] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.203] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:12f3ce9adc10479e epochNumber=1 {"provingJobId":"1:BASE_PARITY:12f3ce9adc10479e"} 18:05:39 [18:05:39.204] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:12f3ce9adc10479e {"provingJobId":"1:BASE_PARITY:12f3ce9adc10479e"} 18:05:39 [18:05:39.204] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:12f3ce9adc10479e type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:12f3ce9adc10479e"} 18:05:39 [18:05:39.204] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa43e9dfcfae3517 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aa43e9dfcfae3517"} 18:05:39 [18:05:39.204] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:12f3ce9adc10479e has already been completed {"provingJobId":"1:BASE_PARITY:12f3ce9adc10479e"} 18:05:39 [18:05:39.205] INFO: prover-client:proving-broker Proving Broker started 18:05:39 [18:05:39.205] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c01b98481e28b287 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c01b98481e28b287"} 18:05:39 [18:05:39.206] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0a88a5778f3ef11d epochNumber=2 {"provingJobId":"1:BASE_PARITY:0a88a5778f3ef11d"} 18:05:40 [18:05:40.257] INFO: prover-client:proving-broker Proving Broker started 18:05:40 [18:05:40.258] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:c01b98481e28b287 settled=false {"provingJobId":"1:BASE_PARITY:c01b98481e28b287","status":"pending"} 18:05:40 [18:05:40.258] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:0a88a5778f3ef11d settled=false {"provingJobId":"1:BASE_PARITY:0a88a5778f3ef11d","status":"pending"} 18:05:40 [18:05:40.259] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:c01b98481e28b287 type=BASE_PARITY startedAt=1749837939206 older agent has taken job {"provingJobId":"1:BASE_PARITY:c01b98481e28b287"} 18:05:40 [18:05:40.259] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c01b98481e28b287 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:c01b98481e28b287"} 18:05:40 [18:05:40.260] INFO: prover-client:proving-broker Proving Broker started 18:05:40 [18:05:40.261] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6587b7bf5bf7b0c8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6587b7bf5bf7b0c8"} 18:05:40 [18:05:40.261] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a08ee9bc8431211e epochNumber=2 {"provingJobId":"1:BASE_PARITY:a08ee9bc8431211e"} 18:05:41 [18:05:41.262] INFO: prover-client:proving-broker Proving Broker started 18:05:41 [18:05:41.263] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:6587b7bf5bf7b0c8 settled=false {"provingJobId":"1:BASE_PARITY:6587b7bf5bf7b0c8","status":"pending"} 18:05:41 [18:05:41.263] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:a08ee9bc8431211e settled=false {"provingJobId":"1:BASE_PARITY:a08ee9bc8431211e","status":"pending"} 18:05:41 [18:05:41.263] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6587b7bf5bf7b0c8 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:6587b7bf5bf7b0c8"} 18:05:41 [18:05:41.265] INFO: prover-client:proving-broker Proving Broker started 18:05:41 [18:05:41.265] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:58f2cdcf65016a26 epochNumber=1 {"provingJobId":"1:BASE_PARITY:58f2cdcf65016a26"} 18:05:41 [18:05:41.266] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e2c7eae7d70fd844 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e2c7eae7d70fd844"} 18:05:42 [18:05:42.266] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.267] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:58f2cdcf65016a26 settled=false {"provingJobId":"1:BASE_PARITY:58f2cdcf65016a26","status":"pending"} 18:05:42 [18:05:42.267] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:e2c7eae7d70fd844 settled=false {"provingJobId":"1:BASE_PARITY:e2c7eae7d70fd844","status":"pending"} 18:05:42 [18:05:42.268] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:58f2cdcf65016a26 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:58f2cdcf65016a26"} 18:05:42 [18:05:42.268] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:58f2cdcf65016a26 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:58f2cdcf65016a26"} 18:05:42 [18:05:42.269] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cb1d0d35988421ba epochNumber=1 {"provingJobId":"1:BASE_PARITY:cb1d0d35988421ba"} 18:05:42 [18:05:42.270] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6b435dca26b06587 epochNumber=2 {"provingJobId":"1:BASE_PARITY:6b435dca26b06587"} 18:05:42 [18:05:42.271] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:cb1d0d35988421ba type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:cb1d0d35988421ba"} 18:05:42 [18:05:42.272] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6b435dca26b06587 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:6b435dca26b06587"} 18:05:42 [18:05:42.273] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.273] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:62ca522bae48a538 epochNumber=1 {"provingJobId":"1:BASE_PARITY:62ca522bae48a538"} 18:05:42 [18:05:42.273] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0f5a0a7b665e133 epochNumber=2 {"provingJobId":"1:BASE_PARITY:e0f5a0a7b665e133"} 18:05:42 [18:05:42.274] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:62ca522bae48a538 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:62ca522bae48a538"} 18:05:42 [18:05:42.274] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:62ca522bae48a538 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:62ca522bae48a538"} 18:05:42 [18:05:42.274] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e0f5a0a7b665e133 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:e0f5a0a7b665e133"} 18:05:42 [18:05:42.275] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.276] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:0fc0f022397f2eff err=test error {"provingJoId":"1:BASE_PARITY:0fc0f022397f2eff"} 18:05:42 [18:05:42.277] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.277] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ac62f889610d7459 not found {"provingJobId":"1:BASE_PARITY:ac62f889610d7459"} 18:05:42 [18:05:42.278] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.278] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2ec7486fcfc9e33f epochNumber=1 {"provingJobId":"1:BASE_PARITY:2ec7486fcfc9e33f"} 18:05:42 [18:05:42.279] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.280] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ca13a88292df722f epochNumber=1 {"provingJobId":"1:BASE_PARITY:ca13a88292df722f"} 18:05:42 [18:05:42.385] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ca13a88292df722f timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:ca13a88292df722f"} 18:05:42 [18:05:42.389] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.389] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:28b7fe2aabb165ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:28b7fe2aabb165ea"} 18:05:42 [18:05:42.490] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:28b7fe2aabb165ea timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:28b7fe2aabb165ea"} 18:05:42 [18:05:42.491] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:afd907daa1ee5c25 epochNumber=10 {"provingJobId":"1:BASE_PARITY:afd907daa1ee5c25"} 18:05:42 [18:05:42.516] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:42 [18:05:42.618] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.627] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:484f90949d0d2869 epochNumber=1 {"provingJobId":"1:BASE_PARITY:484f90949d0d2869"} 18:05:42 [18:05:42.802] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:484f90949d0d2869 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:484f90949d0d2869"} 18:05:42 [18:05:42.834] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.834] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ad5e653280418247 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ad5e653280418247"} 18:05:42 [18:05:42.835] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:ad5e653280418247 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:ad5e653280418247"} 18:05:42 [18:05:42.836] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.837] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:82d62b3ddbee31c4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:82d62b3ddbee31c4"} 18:05:42 [18:05:42.838] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:82d62b3ddbee31c4 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:82d62b3ddbee31c4"} 18:05:42 [18:05:42.838] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:82d62b3ddbee31c4 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:82d62b3ddbee31c4"} 18:05:42 [18:05:42.839] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.840] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cacd3fb79df77c46 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cacd3fb79df77c46"} 18:05:42 [18:05:42.840] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:cacd3fb79df77c46 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:cacd3fb79df77c46"} 18:05:42 [18:05:42.841] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.842] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:491304874b5015d3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:491304874b5015d3"} 18:05:42 [18:05:42.842] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:491304874b5015d3 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:491304874b5015d3"} 18:05:42 [18:05:42.843] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1c51b341f5ccd74e epochNumber=3 {"provingJobId":"1:BASE_PARITY:1c51b341f5ccd74e"} 18:05:42 [18:05:42.867] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:42 [18:05:42.868] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:491304874b5015d3 err=test error {"provingJoId":"1:BASE_PARITY:491304874b5015d3"} 18:05:42 [18:05:42.870] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.871] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:8d22b86416a527c2 settled=false {"provingJobId":"1:BASE_PARITY:8d22b86416a527c2","status":"pending"} 18:05:42 [18:05:42.871] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:9b2992f7cdfbcf59 settled=false {"provingJobId":"1:BASE_PARITY:9b2992f7cdfbcf59","status":"pending"} 18:05:42 [18:05:42.874] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.874] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:46ad3c386b228345 settled=true {"provingJobId":"1:BASE_PARITY:46ad3c386b228345","status":"fulfilled"} 18:05:42 [18:05:42.874] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:456569d572a76689 settled=true {"provingJobId":"2:BASE_PARITY:456569d572a76689","status":"fulfilled"} 18:05:42 [18:05:42.876] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.877] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1b40fac295eacfdf settled=true {"provingJobId":"1:BASE_PARITY:1b40fac295eacfdf","status":"fulfilled"} 18:05:42 [18:05:42.877] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4bec969ef769bec4 settled=false {"provingJobId":"1:BASE_PARITY:4bec969ef769bec4","status":"pending"} 18:05:42 [18:05:42.879] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.881] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:41ba335bd51b1bf7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:41ba335bd51b1bf7"} 18:05:42 [18:05:42.884] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.884] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3441468e715b86f1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3441468e715b86f1"} 18:05:42 [18:05:42.885] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:3441468e715b86f1: Error: db error: Error: db error 18:05:42 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:3441468e715b86f1"} 18:05:42 [18:05:42.887] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.887] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:993e54721052a65b epochNumber=1 {"provingJobId":"1:BASE_PARITY:993e54721052a65b"} 18:05:42 [18:05:42.888] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:993e54721052a65b type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:993e54721052a65b"} 18:05:42 [18:05:42.888] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:993e54721052a65b type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:993e54721052a65b"} 18:05:42 [18:05:42.889] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.889] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:40a58fec392ba9b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:40a58fec392ba9b2"} 18:05:42 [18:05:42.889] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:40a58fec392ba9b2 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:40a58fec392ba9b2"} 18:05:42 [18:05:42.890] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:40a58fec392ba9b2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:40a58fec392ba9b2"} 18:05:42 [18:05:42.890] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:40a58fec392ba9b2: Error: db error 18:05:42 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:40a58fec392ba9b2"} 18:05:42 [18:05:42.896] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.896] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:88dac8ff5625940e epochNumber=1 {"provingJobId":"1:BASE_PARITY:88dac8ff5625940e"} 18:05:42 [18:05:42.897] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:88dac8ff5625940e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:88dac8ff5625940e"} 18:05:42 [18:05:42.897] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:88dac8ff5625940e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:88dac8ff5625940e"} 18:05:42 [18:05:42.898] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.899] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5ea458c31e1d224 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5ea458c31e1d224"} 18:05:42 [18:05:42.899] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f5ea458c31e1d224 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:f5ea458c31e1d224"} 18:05:42 [18:05:42.899] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:f5ea458c31e1d224 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:f5ea458c31e1d224"} 18:05:42 [18:05:42.900] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:f5ea458c31e1d224 jobErr=test error: Error: db error 18:05:42 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:f5ea458c31e1d224"} 18:05:42 [18:05:42.901] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.902] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4e32c1305899fa7d not found {"provingJobId":"1:BASE_PARITY:4e32c1305899fa7d"} 18:05:42 [18:05:42.903] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.904] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c6e0c14dfdda745c err=test error {"provingJoId":"1:BASE_PARITY:c6e0c14dfdda745c"} 18:05:42 [18:05:42.905] INFO: prover-client:proving-broker Proving Broker started 18:05:42 [18:05:42.916] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:81cf0b0c0ee3e411 epochNumber=1 {"provingJobId":"1:BASE_PARITY:81cf0b0c0ee3e411"} 18:05:42 [18:05:42.916] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:81cf0b0c0ee3e411 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:81cf0b0c0ee3e411"} 18:05:42 [18:05:42.917] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:81cf0b0c0ee3e411 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:81cf0b0c0ee3e411"} 18:05:42 [18:05:42.927] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:a3d8de5cd0b4c3cd epochNumber=2 {"provingJobId":"2:BASE_PARITY:a3d8de5cd0b4c3cd"} 18:05:42 [18:05:42.927] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:a3d8de5cd0b4c3cd type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:a3d8de5cd0b4c3cd"} 18:05:42 [18:05:42.928] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:a3d8de5cd0b4c3cd type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:a3d8de5cd0b4c3cd"} 18:05:42 [18:05:42.989] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:3a62a80ffec98737 epochNumber=3 {"provingJobId":"3:BASE_PARITY:3a62a80ffec98737"} 18:05:43 [18:05:43.006] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:43 [18:05:43.052] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:3f8f29262e3349c5 epochNumber=4 {"provingJobId":"4:BASE_PARITY:3f8f29262e3349c5"} 18:05:43 [18:05:43.059] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:43 [18:05:43.114] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:008df7be8c987654 epochNumber=5 {"provingJobId":"5:BASE_PARITY:008df7be8c987654"} 18:05:43 [18:05:43.137] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:43 [18:05:43.170] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.171] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fb05e08cd7cb8562 epochNumber=42 {"provingJobId":"1:BASE_PARITY:fb05e08cd7cb8562"} 18:05:43 [18:05:43.172] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testI6diqb/42 with map size 1073741824 18:05:43 [18:05:43.177] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.204] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:d6f0f65234429966 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d6f0f65234429966"} 18:05:43 [18:05:43.211] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.212] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3700b6cb1be9be0b epochNumber=1 {"provingJobId":"1:BASE_PARITY:3700b6cb1be9be0b"} 18:05:43 [18:05:43.213] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testauQfup/1 with map size 1073741824 18:05:43 [18:05:43.215] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.235] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8d639688cc051f0f epochNumber=1 {"provingJobId":"1:BASE_PARITY:8d639688cc051f0f"} 18:05:43 [18:05:43.247] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.248] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cb3c29556485bb81 epochNumber=1 {"provingJobId":"1:BASE_PARITY:cb3c29556485bb81"} 18:05:43 [18:05:43.248] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCSstPm/1 with map size 1073741824 18:05:43 [18:05:43.251] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.271] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:cb3c29556485bb81 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:cb3c29556485bb81"} 18:05:43 [18:05:43.277] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.278] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:eed782249095884c epochNumber=1 {"provingJobId":"1:BASE_PARITY:eed782249095884c"} 18:05:43 [18:05:43.278] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testcTfbxq/1 with map size 1073741824 18:05:43 [18:05:43.282] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.305] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:eed782249095884c epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:eed782249095884c"} 18:05:43 [18:05:43.306] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:eed782249095884c type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:eed782249095884c"} 18:05:43 [18:05:43.314] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:eed782249095884c epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:eed782249095884c"} 18:05:43 [18:05:43.321] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.322] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3594b98f8032b13b epochNumber=1 {"provingJobId":"1:BASE_PARITY:3594b98f8032b13b"} 18:05:43 [18:05:43.323] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testoCu6Sd/1 with map size 1073741824 18:05:43 [18:05:43.325] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.351] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.356] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.358] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1d048abbe2a9e4ed epochNumber=1 {"provingJobId":"1:BASE_PARITY:1d048abbe2a9e4ed"} 18:05:43 [18:05:43.358] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testIzgzxR/1 with map size 1073741824 18:05:43 [18:05:43.366] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.387] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:1d048abbe2a9e4ed {"provingJobId":"1:BASE_PARITY:1d048abbe2a9e4ed"} 18:05:43 [18:05:43.387] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:1d048abbe2a9e4ed type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:1d048abbe2a9e4ed"} 18:05:43 [18:05:43.388] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1d048abbe2a9e4ed type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:1d048abbe2a9e4ed"} 18:05:43 [18:05:43.401] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.402] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6464408296936a78 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6464408296936a78"} 18:05:43 [18:05:43.403] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testWBjoKU/1 with map size 1073741824 18:05:43 [18:05:43.406] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.426] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:6464408296936a78 {"provingJobId":"1:BASE_PARITY:6464408296936a78"} 18:05:43 [18:05:43.427] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:6464408296936a78 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:6464408296936a78"} 18:05:43 [18:05:43.440] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.442] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ea2f0e6c0f97690 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4ea2f0e6c0f97690"} 18:05:43 [18:05:43.443] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testd2VX3T/1 with map size 1073741824 18:05:43 [18:05:43.445] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.462] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4ea2f0e6c0f97690 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4ea2f0e6c0f97690"} 18:05:43 [18:05:43.463] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:4ea2f0e6c0f97690 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:4ea2f0e6c0f97690"} 18:05:43 [18:05:43.472] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.473] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ed33dbb29e0229e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ed33dbb29e0229e8"} 18:05:43 [18:05:43.473] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUONCrx/1 with map size 1073741824 18:05:43 [18:05:43.477] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.496] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ed33dbb29e0229e8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ed33dbb29e0229e8"} 18:05:43 [18:05:43.497] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:ed33dbb29e0229e8 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:ed33dbb29e0229e8"} 18:05:43 [18:05:43.506] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.507] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ab72b7ac9fad4f52 epochNumber=0 {"provingJobId":"1:BASE_PARITY:ab72b7ac9fad4f52"} 18:05:43 [18:05:43.507] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testFzDrmi/0 with map size 1073741824 18:05:43 [18:05:43.509] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.536] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ab72b7ac9fad4f52 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ab72b7ac9fad4f52"} 18:05:43 [18:05:43.537] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:dbf40eec207cdd98 epochNumber=0 {"provingJobId":"1:BASE_PARITY:dbf40eec207cdd98"} 18:05:43 [18:05:43.537] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:ab72b7ac9fad4f52 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:ab72b7ac9fad4f52"} 18:05:43 [18:05:43.538] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:861fe288124c876d epochNumber=0 {"provingJobId":"1:BASE_PARITY:861fe288124c876d"} 18:05:43 [18:05:43.554] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.556] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.556] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4da2ec1db2455391 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4da2ec1db2455391"} 18:05:43 [18:05:43.557] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testhhLi5O/2 with map size 1073741824 18:05:43 [18:05:43.559] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.578] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4f530f77d55c6970 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4f530f77d55c6970"} 18:05:43 [18:05:43.579] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testhhLi5O/1 with map size 1073741824 18:05:43 [18:05:43.582] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.604] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.605] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9907a85ccc9b42cd epochNumber=1 {"provingJobId":"1:BASE_PARITY:9907a85ccc9b42cd"} 18:05:43 [18:05:43.606] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCS9GDr/1 with map size 1073741824 18:05:43 [18:05:43.608] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.636] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.638] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a24607352d81b8cf epochNumber=1 {"provingJobId":"1:BASE_PARITY:a24607352d81b8cf"} 18:05:43 [18:05:43.639] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQ3Dg0k/1 with map size 1073741824 18:05:43 [18:05:43.643] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.661] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bc784a4c498a44fc epochNumber=1 {"provingJobId":"1:BASE_PARITY:bc784a4c498a44fc"} 18:05:43 [18:05:43.670] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2badc0cf914c86d7 epochNumber=2 {"provingJobId":"1:BASE_PARITY:2badc0cf914c86d7"} 18:05:43 [18:05:43.671] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testQ3Dg0k/2 with map size 1073741824 18:05:43 [18:05:43.674] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.693] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5debb2cbe6a03acc epochNumber=1 {"provingJobId":"1:BASE_PARITY:5debb2cbe6a03acc"} 18:05:43 [18:05:43.706] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.708] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9f47d1b031ecea4e epochNumber=1 {"provingJobId":"1:BASE_PARITY:9f47d1b031ecea4e"} 18:05:43 [18:05:43.708] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzQoIgk/1 with map size 1073741824 18:05:43 [18:05:43.714] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.732] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4747844f34f1b252 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4747844f34f1b252"} 18:05:43 [18:05:43.741] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:287015b48649d8b0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:287015b48649d8b0"} 18:05:43 [18:05:43.742] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzQoIgk/2 with map size 1073741824 18:05:43 [18:05:43.744] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.761] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bcf0b2d112f69e18 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bcf0b2d112f69e18"} 18:05:43 [18:05:43.771] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.773] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3a982bd74b335815 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3a982bd74b335815"} 18:05:43 [18:05:43.773] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKtCCPS/1 with map size 1073741824 18:05:43 [18:05:43.775] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.793] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:97b9f79c7f1e7bd3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:97b9f79c7f1e7bd3"} 18:05:43 [18:05:43.799] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:23e4f1765d697580 epochNumber=2 {"provingJobId":"1:BASE_PARITY:23e4f1765d697580"} 18:05:43 [18:05:43.800] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testKtCCPS/2 with map size 1073741824 18:05:43 [18:05:43.803] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.828] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:80fddd5d455a0719 epochNumber=1 {"provingJobId":"1:BASE_PARITY:80fddd5d455a0719"} 18:05:43 [18:05:43.844] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.845] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:20b35cca7fbc9fb3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:20b35cca7fbc9fb3"} 18:05:43 [18:05:43.846] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLzNLKF/1 with map size 1073741824 18:05:43 [18:05:43.849] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.867] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d1de98d3153ee787 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d1de98d3153ee787"} 18:05:43 [18:05:43.874] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:20b35cca7fbc9fb3 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:20b35cca7fbc9fb3"} 18:05:43 [18:05:43.884] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.885] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1ab702f1ead180c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1ab702f1ead180c1"} 18:05:43 [18:05:43.885] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnpMFaj/1 with map size 1073741824 18:05:43 [18:05:43.887] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.904] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8c543def74cef8c6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8c543def74cef8c6"} 18:05:43 [18:05:43.915] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:1ab702f1ead180c1 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:1ab702f1ead180c1"} 18:05:43 [18:05:43.938] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67a3da68b5370180 epochNumber=1 {"provingJobId":"1:BASE_PARITY:67a3da68b5370180"} 18:05:43 [18:05:43.939] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHggx2G/1 with map size 1073741824 18:05:43 [18:05:43.942] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.961] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2598b61896535053 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2598b61896535053"} 18:05:43 [18:05:43.968] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:67a3da68b5370180 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:67a3da68b5370180"} 18:05:43 [18:05:43.973] INFO: prover-client:proving-broker Proving Broker started 18:05:43 [18:05:43.973] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51001c9cc316632c epochNumber=1 {"provingJobId":"1:BASE_PARITY:51001c9cc316632c"} 18:05:43 [18:05:43.974] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testi1QKBM/1 with map size 1073741824 18:05:43 [18:05:43.976] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:43 [18:05:43.996] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:51001c9cc316632c {"provingJobId":"1:BASE_PARITY:51001c9cc316632c"} 18:05:43 [18:05:43.997] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:51001c9cc316632c type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:51001c9cc316632c"} 18:05:44 [18:05:44.003] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1b61eb8f6f6dedc8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1b61eb8f6f6dedc8"} 18:05:44 [18:05:44.010] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:51001c9cc316632c has already been completed {"provingJobId":"1:BASE_PARITY:51001c9cc316632c"} 18:05:44 [18:05:44.017] INFO: prover-client:proving-broker Proving Broker started 18:05:44 [18:05:44.018] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2c5347d9305b2368 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2c5347d9305b2368"} 18:05:44 [18:05:44.019] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3sHKBV/1 with map size 1073741824 18:05:44 [18:05:44.023] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:44 [18:05:44.048] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:64a5aa001384fa24 epochNumber=2 {"provingJobId":"1:BASE_PARITY:64a5aa001384fa24"} 18:05:44 [18:05:44.049] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test3sHKBV/2 with map size 1073741824 18:05:44 [18:05:44.051] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:45 [18:05:45.120] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.130] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2c5347d9305b2368 settled=false {"provingJobId":"1:BASE_PARITY:2c5347d9305b2368","status":"pending"} 18:05:45 [18:05:45.134] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:64a5aa001384fa24 settled=false {"provingJobId":"1:BASE_PARITY:64a5aa001384fa24","status":"pending"} 18:05:45 [18:05:45.135] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:2c5347d9305b2368 type=BASE_PARITY startedAt=1749837944068 older agent has taken job {"provingJobId":"1:BASE_PARITY:2c5347d9305b2368"} 18:05:45 [18:05:45.136] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2c5347d9305b2368 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:2c5347d9305b2368"} 18:05:45 [18:05:45.139] INFO: prover-client:proving-broker Proving Broker started 18:05:45 [18:05:45.140] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b775211f057f56ea epochNumber=1 {"provingJobId":"1:BASE_PARITY:b775211f057f56ea"} 18:05:45 [18:05:45.142] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6e1CGi/1 with map size 1073741824 18:05:45 [18:05:45.144] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:45 [18:05:45.165] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa3803bc0f0f86aa epochNumber=2 {"provingJobId":"1:BASE_PARITY:aa3803bc0f0f86aa"} 18:05:45 [18:05:45.166] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test6e1CGi/2 with map size 1073741824 18:05:45 [18:05:45.169] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:46 [18:05:46.186] INFO: prover-client:proving-broker Proving Broker started 18:05:46 [18:05:46.189] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b775211f057f56ea settled=false {"provingJobId":"1:BASE_PARITY:b775211f057f56ea","status":"pending"} 18:05:46 [18:05:46.191] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:aa3803bc0f0f86aa settled=false {"provingJobId":"1:BASE_PARITY:aa3803bc0f0f86aa","status":"pending"} 18:05:46 [18:05:46.192] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b775211f057f56ea type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:b775211f057f56ea"} 18:05:46 [18:05:46.198] INFO: prover-client:proving-broker Proving Broker started 18:05:46 [18:05:46.199] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:444887b289c1876d epochNumber=1 {"provingJobId":"1:BASE_PARITY:444887b289c1876d"} 18:05:46 [18:05:46.200] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testJIiOWV/1 with map size 1073741824 18:05:46 [18:05:46.208] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:46 [18:05:46.228] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4ff730e5e7921068 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4ff730e5e7921068"} 18:05:46 [18:05:46.230] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testJIiOWV/2 with map size 1073741824 18:05:46 [18:05:46.233] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.252] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.255] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:444887b289c1876d settled=false {"provingJobId":"1:BASE_PARITY:444887b289c1876d","status":"pending"} 18:05:47 [18:05:47.257] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:4ff730e5e7921068 settled=false {"provingJobId":"1:BASE_PARITY:4ff730e5e7921068","status":"pending"} 18:05:47 [18:05:47.261] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:444887b289c1876d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:444887b289c1876d"} 18:05:47 [18:05:47.262] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:444887b289c1876d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:444887b289c1876d"} 18:05:47 [18:05:47.273] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.274] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1361e77c7975c036 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1361e77c7975c036"} 18:05:47 [18:05:47.275] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXIZtNH/1 with map size 1073741824 18:05:47 [18:05:47.278] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.304] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:72722590ad22e35d epochNumber=2 {"provingJobId":"1:BASE_PARITY:72722590ad22e35d"} 18:05:47 [18:05:47.306] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testXIZtNH/2 with map size 1073741824 18:05:47 [18:05:47.320] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.344] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1361e77c7975c036 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1361e77c7975c036"} 18:05:47 [18:05:47.351] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:72722590ad22e35d type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:72722590ad22e35d"} 18:05:47 [18:05:47.364] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.365] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:98242f65c53b089e epochNumber=1 {"provingJobId":"1:BASE_PARITY:98242f65c53b089e"} 18:05:47 [18:05:47.366] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSSIjiM/1 with map size 1073741824 18:05:47 [18:05:47.368] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.391] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:93a03ea0a3d03a5a epochNumber=2 {"provingJobId":"1:BASE_PARITY:93a03ea0a3d03a5a"} 18:05:47 [18:05:47.392] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSSIjiM/2 with map size 1073741824 18:05:47 [18:05:47.394] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.414] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:98242f65c53b089e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:98242f65c53b089e"} 18:05:47 [18:05:47.415] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:98242f65c53b089e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:98242f65c53b089e"} 18:05:47 [18:05:47.431] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:93a03ea0a3d03a5a type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:93a03ea0a3d03a5a"} 18:05:47 [18:05:47.446] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.447] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:f33dcc86255c5094 err=test error {"provingJoId":"1:BASE_PARITY:f33dcc86255c5094"} 18:05:47 [18:05:47.451] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.452] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d87e9e9f8c23e828 not found {"provingJobId":"1:BASE_PARITY:d87e9e9f8c23e828"} 18:05:47 [18:05:47.455] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.456] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:61b8fcda3746225a epochNumber=1 {"provingJobId":"1:BASE_PARITY:61b8fcda3746225a"} 18:05:47 [18:05:47.457] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testj0Dop9/1 with map size 1073741824 18:05:47 [18:05:47.460] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.490] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.502] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8c763df8dcf33d83 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8c763df8dcf33d83"} 18:05:47 [18:05:47.504] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testl7pYSR/1 with map size 1073741824 18:05:47 [18:05:47.509] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.654] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8c763df8dcf33d83 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:8c763df8dcf33d83"} 18:05:47 [18:05:47.668] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.670] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d09e516b03a7eb68 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d09e516b03a7eb68"} 18:05:47 [18:05:47.671] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testrPnrJo/1 with map size 1073741824 18:05:47 [18:05:47.675] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.795] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d09e516b03a7eb68 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d09e516b03a7eb68"} 18:05:47 [18:05:47.802] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2128ae3e69c1fbd6 epochNumber=10 {"provingJobId":"1:BASE_PARITY:2128ae3e69c1fbd6"} 18:05:47 [18:05:47.807] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testrPnrJo/10 with map size 1073741824 18:05:47 [18:05:47.810] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:47 [18:05:47.847] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:47 [18:05:47.848] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:47 [18:05:47.971] INFO: prover-client:proving-broker Proving Broker started 18:05:47 [18:05:47.972] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d73d420a9ba87f75 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d73d420a9ba87f75"} 18:05:47 [18:05:47.973] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsaA6qM/1 with map size 1073741824 18:05:47 [18:05:47.977] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.152] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d73d420a9ba87f75 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d73d420a9ba87f75"} 18:05:48 [18:05:48.207] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.208] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9d94fa767c55f69e epochNumber=1 {"provingJobId":"1:BASE_PARITY:9d94fa767c55f69e"} 18:05:48 [18:05:48.210] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM2ga5y/1 with map size 1073741824 18:05:48 [18:05:48.222] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.249] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:9d94fa767c55f69e type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:9d94fa767c55f69e"} 18:05:48 [18:05:48.254] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.255] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e771d2e603f3fa6d epochNumber=1 {"provingJobId":"1:BASE_PARITY:e771d2e603f3fa6d"} 18:05:48 [18:05:48.255] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLEju9r/1 with map size 1073741824 18:05:48 [18:05:48.257] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.278] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e771d2e603f3fa6d type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:e771d2e603f3fa6d"} 18:05:48 [18:05:48.280] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e771d2e603f3fa6d type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:e771d2e603f3fa6d"} 18:05:48 [18:05:48.294] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.295] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:549fea4a7072ffbb epochNumber=1 {"provingJobId":"1:BASE_PARITY:549fea4a7072ffbb"} 18:05:48 [18:05:48.298] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testUKiVyf/1 with map size 1073741824 18:05:48 [18:05:48.301] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.333] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:549fea4a7072ffbb type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:549fea4a7072ffbb"} 18:05:48 [18:05:48.345] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.347] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c4fe3be4a5e5cd05 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c4fe3be4a5e5cd05"} 18:05:48 [18:05:48.348] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBNWAPM/1 with map size 1073741824 18:05:48 [18:05:48.350] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.377] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:c4fe3be4a5e5cd05 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:c4fe3be4a5e5cd05"} 18:05:48 [18:05:48.378] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b9ec6d0d4cc8c0fb epochNumber=3 {"provingJobId":"1:BASE_PARITY:b9ec6d0d4cc8c0fb"} 18:05:48 [18:05:48.380] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testBNWAPM/3 with map size 1073741824 18:05:48 [18:05:48.384] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.422] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:48 [18:05:48.423] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:48 [18:05:48.425] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:c4fe3be4a5e5cd05 err=test error {"provingJoId":"1:BASE_PARITY:c4fe3be4a5e5cd05"} 18:05:48 [18:05:48.432] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testKytyaS/1 with map size 1073741824 18:05:48 [18:05:48.434] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.458] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testKytyaS/2 with map size 1073741824 18:05:48 [18:05:48.465] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.486] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.489] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bca4929b0e86338b settled=false {"provingJobId":"1:BASE_PARITY:bca4929b0e86338b","status":"pending"} 18:05:48 [18:05:48.490] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:3df5b396067ca339 settled=false {"provingJobId":"1:BASE_PARITY:3df5b396067ca339","status":"pending"} 18:05:48 [18:05:48.496] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testDDpOXZ/1 with map size 1073741824 18:05:48 [18:05:48.501] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.527] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testDDpOXZ/2 with map size 1073741824 18:05:48 [18:05:48.532] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.581] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.584] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:de779d73c5ba7680 settled=true {"provingJobId":"1:BASE_PARITY:de779d73c5ba7680","status":"fulfilled"} 18:05:48 [18:05:48.588] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8fe46cd87ba7138b settled=true {"provingJobId":"2:BASE_PARITY:8fe46cd87ba7138b","status":"fulfilled"} 18:05:48 [18:05:48.594] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testVV1ewx/1 with map size 1073741824 18:05:48 [18:05:48.600] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.652] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testVV1ewx/2 with map size 1073741824 18:05:48 [18:05:48.657] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.683] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.687] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:bbf3b751c0d11b86 settled=true {"provingJobId":"1:BASE_PARITY:bbf3b751c0d11b86","status":"fulfilled"} 18:05:48 [18:05:48.689] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:353e4c96aab164be settled=false {"provingJobId":"1:BASE_PARITY:353e4c96aab164be","status":"pending"} 18:05:48 [18:05:48.700] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.701] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b279551c1a82287e epochNumber=1 {"provingJobId":"1:BASE_PARITY:b279551c1a82287e"} 18:05:48 [18:05:48.701] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testq9kj43/1 with map size 1073741824 18:05:48 [18:05:48.707] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.738] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.739] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:23435e67abdfbed6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:23435e67abdfbed6"} 18:05:48 [18:05:48.740] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:23435e67abdfbed6: Error: db error: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:23435e67abdfbed6"} 18:05:48 [18:05:48.748] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.750] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:252786e59d096853 epochNumber=1 {"provingJobId":"1:BASE_PARITY:252786e59d096853"} 18:05:48 [18:05:48.751] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testsj2CBE/1 with map size 1073741824 18:05:48 [18:05:48.756] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.773] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:252786e59d096853 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:252786e59d096853"} 18:05:48 [18:05:48.773] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:252786e59d096853 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:252786e59d096853"} 18:05:48 [18:05:48.786] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.786] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e6a1f40b5ffea32a epochNumber=1 {"provingJobId":"1:BASE_PARITY:e6a1f40b5ffea32a"} 18:05:48 [18:05:48.789] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testT8LK9G/1 with map size 1073741824 18:05:48 [18:05:48.794] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.822] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e6a1f40b5ffea32a type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e6a1f40b5ffea32a"} 18:05:48 [18:05:48.823] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:e6a1f40b5ffea32a type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:e6a1f40b5ffea32a"} 18:05:48 [18:05:48.824] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:e6a1f40b5ffea32a: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:e6a1f40b5ffea32a"} 18:05:48 [18:05:48.831] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.832] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2695a04232cc9e56 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2695a04232cc9e56"} 18:05:48 [18:05:48.834] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testy4usDr/1 with map size 1073741824 18:05:48 [18:05:48.839] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.859] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2695a04232cc9e56 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2695a04232cc9e56"} 18:05:48 [18:05:48.859] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2695a04232cc9e56 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2695a04232cc9e56"} 18:05:48 [18:05:48.871] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.871] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e360f3c0fefb2cc5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e360f3c0fefb2cc5"} 18:05:48 [18:05:48.872] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8jfDSt/1 with map size 1073741824 18:05:48 [18:05:48.877] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.903] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e360f3c0fefb2cc5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:e360f3c0fefb2cc5"} 18:05:48 [18:05:48.903] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e360f3c0fefb2cc5 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:e360f3c0fefb2cc5"} 18:05:48 [18:05:48.904] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:e360f3c0fefb2cc5 jobErr=test error: Error: db error 18:05:48 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:e360f3c0fefb2cc5"} 18:05:48 [18:05:48.916] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.917] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4b0ef8b80780f83d not found {"provingJobId":"1:BASE_PARITY:4b0ef8b80780f83d"} 18:05:48 [18:05:48.930] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.931] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:dcfe5646a63f2166 err=test error {"provingJoId":"1:BASE_PARITY:dcfe5646a63f2166"} 18:05:48 [18:05:48.936] INFO: prover-client:proving-broker Proving Broker started 18:05:48 [18:05:48.948] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e28d384e842eea4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e28d384e842eea4"} 18:05:48 [18:05:48.950] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testFeMOCS/1 with map size 1073741824 18:05:48 [18:05:48.956] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:48 [18:05:48.978] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7e28d384e842eea4 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7e28d384e842eea4"} 18:05:48 [18:05:48.978] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7e28d384e842eea4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7e28d384e842eea4"} 18:05:49 [18:05:49.001] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:6337e3ee28936f40 epochNumber=2 {"provingJobId":"2:BASE_PARITY:6337e3ee28936f40"} 18:05:49 [18:05:49.004] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testFeMOCS/2 with map size 1073741824 18:05:49 [18:05:49.008] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.043] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:6337e3ee28936f40 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:6337e3ee28936f40"} 18:05:49 [18:05:49.044] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:6337e3ee28936f40 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:6337e3ee28936f40"} 18:05:49 [18:05:49.120] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:6adb3e839a444fdc epochNumber=3 {"provingJobId":"3:BASE_PARITY:6adb3e839a444fdc"} 18:05:49 [18:05:49.126] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testFeMOCS/3 with map size 1073741824 18:05:49 [18:05:49.132] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.174] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:49 [18:05:49.175] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:05:49 [18:05:49.214] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:3e06055f9933514c epochNumber=4 {"provingJobId":"4:BASE_PARITY:3e06055f9933514c"} 18:05:49 [18:05:49.217] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testFeMOCS/4 with map size 1073741824 18:05:49 [18:05:49.225] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.254] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:49 [18:05:49.255] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 18:05:49 [18:05:49.308] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:6ccb994d7161578c epochNumber=5 {"provingJobId":"5:BASE_PARITY:6ccb994d7161578c"} 18:05:49 [18:05:49.309] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testFeMOCS/5 with map size 1073741824 18:05:49 [18:05:49.315] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:05:49 [18:05:49.358] INFO: prover-client:proving-broker Cleaned up jobs=1 18:05:49 [18:05:49.359] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 18:05:49 PASS src/proving_broker/proving_broker.test.ts (12.946 s) 18:05:49 ProvingBroker 18:05:49 Producer API 18:05:49 ✓ refuses stale jobs (46 ms) 18:05:49 ✓ enqueues jobs (3 ms) 18:05:49 ✓ ignores duplicate jobs (4 ms) 18:05:49 ✓ reports correct status when enqueuing repeat jobs (3 ms) 18:05:49 ✓ throws an error in case of duplicate job IDs (18 ms) 18:05:49 ✓ returns not-found status for non-existing jobs (1 ms) 18:05:49 ✓ cancels jobs in queue (3 ms) 18:05:49 ✓ cancels jobs in-progress (2 ms) 18:05:49 ✓ returns job result if successful (3 ms) 18:05:49 ✓ returns job error if failed (2 ms) 18:05:49 ✓ correctly returns job status for concurrent writes (2 ms) 18:05:49 ✓ refuses stale jobs (41 ms) 18:05:49 ✓ enqueues jobs (35 ms) 18:05:49 ✓ ignores duplicate jobs (29 ms) 18:05:49 ✓ reports correct status when enqueuing repeat jobs (44 ms) 18:05:49 ✓ throws an error in case of duplicate job IDs (30 ms) 18:05:49 ✓ returns not-found status for non-existing jobs (5 ms) 18:05:49 ✓ cancels jobs in queue (44 ms) 18:05:49 ✓ cancels jobs in-progress (38 ms) 18:05:49 ✓ returns job result if successful (32 ms) 18:05:49 ✓ returns job error if failed (34 ms) 18:05:49 ✓ correctly returns job status for concurrent writes (48 ms) 18:05:49 Consumer API 18:05:49 ✓ returns undefined if no jobs are available (1 ms) 18:05:49 ✓ returns jobs in priority order (1 ms) 18:05:49 ✓ returns undefined if no jobs are available for the given allowList 18:05:49 ✓ returns a job if it is in the allowList (2 ms) 18:05:49 ✓ returns the most important job if it is in the allowList (2 ms) 18:05:49 ✓ returns any job if filter is empty (9 ms) 18:05:49 ✓ returns a new job when reporting job success (2 ms) 18:05:49 ✓ returns a new job when reporting permanent error (2 ms) 18:05:49 ✓ returns a new job when reporting retry-able error (1 ms) 18:05:49 ✓ returns a new job when reporting progress if current one is cancelled (2 ms) 18:05:49 ✓ returns a new job if job is already in progress elsewhere (1055 ms) 18:05:49 ✓ avoids sending the same job to a new agent after a restart (1004 ms) 18:05:49 ✓ avoids sending a completed job to a new agent after a restart (1005 ms) 18:05:49 ✓ tracks job result if in progress (3 ms) 18:05:49 ✓ tracks job result even if job is in queue (3 ms) 18:05:49 ✓ ignores reported job error if unknown job (1 ms) 18:05:49 ✓ ignores job result if unknown job (1 ms) 18:05:49 ✓ returns undefined if no jobs are available (2 ms) 18:05:49 ✓ returns jobs in priority order (47 ms) 18:05:49 ✓ returns undefined if no jobs are available for the given allowList (32 ms) 18:05:49 ✓ returns a job if it is in the allowList (69 ms) 18:05:49 ✓ returns the most important job if it is in the allowList (65 ms) 18:05:49 ✓ returns any job if filter is empty (71 ms) 18:05:49 ✓ returns a new job when reporting job success (41 ms) 18:05:49 ✓ returns a new job when reporting permanent error (53 ms) 18:05:49 ✓ returns a new job when reporting retry-able error (34 ms) 18:05:49 ✓ returns a new job when reporting progress if current one is cancelled (43 ms) 18:05:49 ✓ returns a new job if job is already in progress elsewhere (1124 ms) 18:05:49 ✓ avoids sending the same job to a new agent after a restart (1057 ms) 18:05:49 ✓ avoids sending a completed job to a new agent after a restart (1075 ms) 18:05:49 ✓ tracks job result if in progress (90 ms) 18:05:49 ✓ tracks job result even if job is in queue (80 ms) 18:05:49 ✓ ignores reported job error if unknown job (7 ms) 18:05:49 ✓ ignores job result if unknown job (4 ms) 18:05:49 Timeouts 18:05:49 ✓ tracks in progress jobs (1 ms) 18:05:49 ✓ re-enqueues jobs that time out (109 ms) 18:05:49 ✓ cancel stale jobs that time out (229 ms) 18:05:49 ✓ keeps the jobs in progress while it is alive (214 ms) 18:05:49 ✓ tracks in progress jobs (34 ms) 18:05:49 ✓ re-enqueues jobs that time out (174 ms) 18:05:49 ✓ cancel stale jobs that time out (302 ms) 18:05:49 ✓ keeps the jobs in progress while it is alive (236 ms) 18:05:49 Retries 18:05:49 ✓ retries jobs (3 ms) 18:05:49 ✓ retries up to a maximum number of times (3 ms) 18:05:49 ✓ passing retry=false does not retry (2 ms) 18:05:49 ✓ does not retry if job is stale (28 ms) 18:05:49 ✓ retries jobs (47 ms) 18:05:49 ✓ retries up to a maximum number of times (38 ms) 18:05:49 ✓ passing retry=false does not retry (51 ms) 18:05:49 ✓ does not retry if job is stale (83 ms) 18:05:49 Database management 18:05:49 ✓ re-enqueues proof requests on start (4 ms) 18:05:49 ✓ restores proof results on start (2 ms) 18:05:49 ✓ only re-enqueues unfinished jobs (2 ms) 18:05:49 ✓ saves job when enqueued (5 ms) 18:05:49 ✓ does not retain job if database fails to save (3 ms) 18:05:49 ✓ saves job result (2 ms) 18:05:49 ✓ saves result even if database fails to save (6 ms) 18:05:49 ✓ saves job error (3 ms) 18:05:49 ✓ saves job error even if database fails to save (2 ms) 18:05:49 ✓ does not save job result if job is unknown (2 ms) 18:05:49 ✓ does not save job error if job is unknown (2 ms) 18:05:49 ✓ cleans up old jobs periodically (262 ms) 18:05:49 ✓ re-enqueues proof requests on start (66 ms) 18:05:49 ✓ restores proof results on start (99 ms) 18:05:49 ✓ only re-enqueues unfinished jobs (101 ms) 18:05:49 ✓ saves job when enqueued (41 ms) 18:05:49 ✓ does not retain job if database fails to save (7 ms) 18:05:49 ✓ saves job result (39 ms) 18:05:49 ✓ saves result even if database fails to save (44 ms) 18:05:49 ✓ saves job error (39 ms) 18:05:49 ✓ saves job error even if database fails to save (39 ms) 18:05:49 ✓ does not save job result if job is unknown (13 ms) 18:05:49 ✓ does not save job error if job is unknown (9 ms) 18:05:49 ✓ cleans up old jobs periodically (459 ms) 18:05:49 18:05:49 Test Suites: 1 passed, 1 total 18:05:49 Tests: 96 passed, 96 total 18:05:49 Snapshots: 0 total 18:05:49 Time: 13.04 s 18:05:49 Ran all test suites matching /prover-client\/src\/proving_broker\/proving_broker.test.ts/i. 18:05:49 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?