Parent Log: http://ci.aztec-labs.com/6bcaf8b705a8c8ed Command: a222d7f5a623064c yarn-project/scripts/run_test.sh prover-client/src/proving_broker/proving_broker.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/a37c3cdcd562df234bc44cc52cdce68d105ff2f2 Env: REF_NAME=gh-readonly-queue/next/pr-15135-df0c3e8efd6211cf15eedf67d2a591cc54099ac8 CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Wed Jun 18 18:51:22 UTC 2025 System: ARCH=arm64 CPUS=64 MEM=247Gi HOSTNAME=pr-15135_arm64_a1-fast Resources: CPU_LIST=0-63 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_b95296ef06707e5f_next 18:51:24 [18:51:24.906] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.911] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8f20c98a17469a08 epochNumber=42 {"provingJobId":"1:BASE_PARITY:8f20c98a17469a08"} 18:51:24 [18:51:24.914] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:a32c8aaf00ca42b1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a32c8aaf00ca42b1"} 18:51:24 [18:51:24.939] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.940] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3564ed96ba17d3e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3564ed96ba17d3e8"} 18:51:24 [18:51:24.941] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1414e4ad785a6c77 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1414e4ad785a6c77"} 18:51:24 [18:51:24.943] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.944] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:870d7aa7b2991ed2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:870d7aa7b2991ed2"} 18:51:24 [18:51:24.945] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:870d7aa7b2991ed2 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:870d7aa7b2991ed2"} 18:51:24 [18:51:24.947] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.948] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2a386d337a8c47a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2a386d337a8c47a4"} 18:51:24 [18:51:24.949] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2a386d337a8c47a4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2a386d337a8c47a4"} 18:51:24 [18:51:24.949] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2a386d337a8c47a4 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2a386d337a8c47a4"} 18:51:24 [18:51:24.950] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:2a386d337a8c47a4 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:2a386d337a8c47a4"} 18:51:24 [18:51:24.951] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.952] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f128c880597f9a94 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f128c880597f9a94"} 18:51:24 [18:51:24.970] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.971] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.972] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08398c10db0807ce epochNumber=1 {"provingJobId":"1:BASE_PARITY:08398c10db0807ce"} 18:51:24 [18:51:24.973] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:08398c10db0807ce {"provingJobId":"1:BASE_PARITY:08398c10db0807ce"} 18:51:24 [18:51:24.973] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:08398c10db0807ce type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:08398c10db0807ce"} 18:51:24 [18:51:24.973] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:08398c10db0807ce type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:08398c10db0807ce"} 18:51:24 [18:51:24.975] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.975] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d779a07d5916445d epochNumber=1 {"provingJobId":"1:BASE_PARITY:d779a07d5916445d"} 18:51:24 [18:51:24.976] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:d779a07d5916445d {"provingJobId":"1:BASE_PARITY:d779a07d5916445d"} 18:51:24 [18:51:24.976] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:d779a07d5916445d type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:d779a07d5916445d"} 18:51:24 [18:51:24.978] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.978] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ad492f2177fcc540 epochNumber=1 {"provingJobId":"1:BASE_PARITY:ad492f2177fcc540"} 18:51:24 [18:51:24.979] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:ad492f2177fcc540 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:ad492f2177fcc540"} 18:51:24 [18:51:24.979] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:ad492f2177fcc540 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:ad492f2177fcc540"} 18:51:24 [18:51:24.980] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.981] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3311ecd3f602fc60 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3311ecd3f602fc60"} 18:51:24 [18:51:24.981] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3311ecd3f602fc60 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3311ecd3f602fc60"} 18:51:24 [18:51:24.982] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3311ecd3f602fc60 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3311ecd3f602fc60"} 18:51:24 [18:51:24.983] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.983] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:593b9e9a50ae2f43 epochNumber=0 {"provingJobId":"1:BASE_PARITY:593b9e9a50ae2f43"} 18:51:24 [18:51:24.983] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:593b9e9a50ae2f43 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:593b9e9a50ae2f43"} 18:51:24 [18:51:24.984] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:508e07f11d082c32 epochNumber=0 {"provingJobId":"1:BASE_PARITY:508e07f11d082c32"} 18:51:24 [18:51:24.984] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:593b9e9a50ae2f43 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:593b9e9a50ae2f43"} 18:51:24 [18:51:24.984] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c8ee50fa4a4a619c epochNumber=0 {"provingJobId":"1:BASE_PARITY:c8ee50fa4a4a619c"} 18:51:24 [18:51:24.985] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.986] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.987] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:31128f6bbcec8031 epochNumber=2 {"provingJobId":"1:BASE_PARITY:31128f6bbcec8031"} 18:51:24 [18:51:24.987] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2c164bbb10c5b19 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2c164bbb10c5b19"} 18:51:24 [18:51:24.988] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.988] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7b6b3873846647f1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7b6b3873846647f1"} 18:51:24 [18:51:24.994] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.994] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0876780469d4331e epochNumber=1 {"provingJobId":"1:BASE_PARITY:0876780469d4331e"} 18:51:24 [18:51:24.995] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9517cbfb0193f34d epochNumber=1 {"provingJobId":"1:BASE_PARITY:9517cbfb0193f34d"} 18:51:24 [18:51:24.995] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:31745c47dbeb9d1c epochNumber=2 {"provingJobId":"1:BASE_PARITY:31745c47dbeb9d1c"} 18:51:24 [18:51:24.995] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:694612f2a0d64145 epochNumber=1 {"provingJobId":"1:BASE_PARITY:694612f2a0d64145"} 18:51:24 [18:51:24.996] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2f82f6774c6f2fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2f82f6774c6f2fb"} 18:51:24 [18:51:24.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:30638e649524cf60 epochNumber=1 {"provingJobId":"1:BASE_PARITY:30638e649524cf60"} 18:51:24 [18:51:24.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5e4f9be2e8a0598f epochNumber=2 {"provingJobId":"1:BASE_PARITY:5e4f9be2e8a0598f"} 18:51:24 [18:51:24.997] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:678ccd3150a706aa epochNumber=1 {"provingJobId":"1:BASE_PARITY:678ccd3150a706aa"} 18:51:24 [18:51:24.998] INFO: prover-client:proving-broker Proving Broker started 18:51:24 [18:51:24.999] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d8677084e243855b epochNumber=1 {"provingJobId":"1:BASE_PARITY:d8677084e243855b"} 18:51:24 [18:51:24.999] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3efbc9682b72625f epochNumber=1 {"provingJobId":"1:BASE_PARITY:3efbc9682b72625f"} 18:51:24 [18:51:24.999] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0b1d3baf2c25c880 epochNumber=2 {"provingJobId":"1:BASE_PARITY:0b1d3baf2c25c880"} 18:51:24 [18:51:25.000] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:416252efc3b3d935 epochNumber=1 {"provingJobId":"1:BASE_PARITY:416252efc3b3d935"} 18:51:25 [18:51:25.001] INFO: prover-client:proving-broker Proving Broker started 18:51:25 [18:51:25.001] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1978c0e4254b7bd6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:1978c0e4254b7bd6"} 18:51:25 [18:51:25.002] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8f5fbfffc245785 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a8f5fbfffc245785"} 18:51:25 [18:51:25.002] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:1978c0e4254b7bd6 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:1978c0e4254b7bd6"} 18:51:25 [18:51:25.003] INFO: prover-client:proving-broker Proving Broker started 18:51:25 [18:51:25.003] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:52df05a8beaf9965 epochNumber=1 {"provingJobId":"1:BASE_PARITY:52df05a8beaf9965"} 18:51:25 [18:51:25.004] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a4da139941fce65e epochNumber=1 {"provingJobId":"1:BASE_PARITY:a4da139941fce65e"} 18:51:25 [18:51:25.004] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:52df05a8beaf9965 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:52df05a8beaf9965"} 18:51:25 [18:51:25.005] INFO: prover-client:proving-broker Proving Broker started 18:51:25 [18:51:25.005] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:67c5fc15baee6edb epochNumber=1 {"provingJobId":"1:BASE_PARITY:67c5fc15baee6edb"} 18:51:25 [18:51:25.006] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:03db99724e62f543 epochNumber=1 {"provingJobId":"1:BASE_PARITY:03db99724e62f543"} 18:51:25 [18:51:25.006] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:67c5fc15baee6edb type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:67c5fc15baee6edb"} 18:51:25 [18:51:25.007] INFO: prover-client:proving-broker Proving Broker started 18:51:25 [18:51:25.007] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e666d443d6e73ee6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:e666d443d6e73ee6"} 18:51:25 [18:51:25.007] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:e666d443d6e73ee6 {"provingJobId":"1:BASE_PARITY:e666d443d6e73ee6"} 18:51:25 [18:51:25.008] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e666d443d6e73ee6 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:e666d443d6e73ee6"} 18:51:25 [18:51:25.008] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f5aeac03ed6ee988 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f5aeac03ed6ee988"} 18:51:25 [18:51:25.008] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:e666d443d6e73ee6 has already been completed {"provingJobId":"1:BASE_PARITY:e666d443d6e73ee6"} 18:51:25 [18:51:25.009] INFO: prover-client:proving-broker Proving Broker started 18:51:25 [18:51:25.010] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b112ce891f0ed8d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b112ce891f0ed8d5"} 18:51:25 [18:51:25.010] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:27627fb2e7b5616b epochNumber=2 {"provingJobId":"1:BASE_PARITY:27627fb2e7b5616b"} 18:51:26 [18:51:26.061] INFO: prover-client:proving-broker Proving Broker started 18:51:26 [18:51:26.062] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b112ce891f0ed8d5 settled=false {"provingJobId":"1:BASE_PARITY:b112ce891f0ed8d5","status":"pending"} 18:51:26 [18:51:26.062] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:27627fb2e7b5616b settled=false {"provingJobId":"1:BASE_PARITY:27627fb2e7b5616b","status":"pending"} 18:51:26 [18:51:26.063] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:b112ce891f0ed8d5 type=BASE_PARITY startedAt=1750272685010 older agent has taken job {"provingJobId":"1:BASE_PARITY:b112ce891f0ed8d5"} 18:51:26 [18:51:26.063] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b112ce891f0ed8d5 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:b112ce891f0ed8d5"} 18:51:26 [18:51:26.064] INFO: prover-client:proving-broker Proving Broker started 18:51:26 [18:51:26.064] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b583672f257d06bf epochNumber=1 {"provingJobId":"1:BASE_PARITY:b583672f257d06bf"} 18:51:26 [18:51:26.065] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:08d74860f9bcbf23 epochNumber=2 {"provingJobId":"1:BASE_PARITY:08d74860f9bcbf23"} 18:51:27 [18:51:27.066] INFO: prover-client:proving-broker Proving Broker started 18:51:27 [18:51:27.066] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b583672f257d06bf settled=false {"provingJobId":"1:BASE_PARITY:b583672f257d06bf","status":"pending"} 18:51:27 [18:51:27.066] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:08d74860f9bcbf23 settled=false {"provingJobId":"1:BASE_PARITY:08d74860f9bcbf23","status":"pending"} 18:51:27 [18:51:27.067] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:b583672f257d06bf type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:b583672f257d06bf"} 18:51:27 [18:51:27.068] INFO: prover-client:proving-broker Proving Broker started 18:51:27 [18:51:27.068] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:29224042808bc360 epochNumber=1 {"provingJobId":"1:BASE_PARITY:29224042808bc360"} 18:51:27 [18:51:27.069] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:1f28ce6925252b55 epochNumber=2 {"provingJobId":"1:BASE_PARITY:1f28ce6925252b55"} 18:51:28 [18:51:28.070] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.070] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:29224042808bc360 settled=false {"provingJobId":"1:BASE_PARITY:29224042808bc360","status":"pending"} 18:51:28 [18:51:28.070] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:1f28ce6925252b55 settled=false {"provingJobId":"1:BASE_PARITY:1f28ce6925252b55","status":"pending"} 18:51:28 [18:51:28.071] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:29224042808bc360 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:29224042808bc360"} 18:51:28 [18:51:28.071] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:29224042808bc360 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:29224042808bc360"} 18:51:28 [18:51:28.072] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.073] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9da86e82342886b2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9da86e82342886b2"} 18:51:28 [18:51:28.073] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2f8809625088edcb epochNumber=2 {"provingJobId":"1:BASE_PARITY:2f8809625088edcb"} 18:51:28 [18:51:28.074] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:9da86e82342886b2 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:9da86e82342886b2"} 18:51:28 [18:51:28.074] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:2f8809625088edcb type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:2f8809625088edcb"} 18:51:28 [18:51:28.075] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5301d9cd8d02491d epochNumber=1 {"provingJobId":"1:BASE_PARITY:5301d9cd8d02491d"} 18:51:28 [18:51:28.076] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:11f226f0c95f1372 epochNumber=2 {"provingJobId":"1:BASE_PARITY:11f226f0c95f1372"} 18:51:28 [18:51:28.076] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:5301d9cd8d02491d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:5301d9cd8d02491d"} 18:51:28 [18:51:28.076] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:5301d9cd8d02491d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:5301d9cd8d02491d"} 18:51:28 [18:51:28.077] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:11f226f0c95f1372 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:11f226f0c95f1372"} 18:51:28 [18:51:28.078] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.078] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:22384deede57cff7 err=test error {"provingJoId":"1:BASE_PARITY:22384deede57cff7"} 18:51:28 [18:51:28.079] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.080] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:813e0d58ee1c1097 not found {"provingJobId":"1:BASE_PARITY:813e0d58ee1c1097"} 18:51:28 [18:51:28.081] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.081] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b63eca66a02e2b6 epochNumber=1 {"provingJobId":"1:BASE_PARITY:9b63eca66a02e2b6"} 18:51:28 [18:51:28.083] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.083] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:66e09b835ea4fc25 epochNumber=1 {"provingJobId":"1:BASE_PARITY:66e09b835ea4fc25"} 18:51:28 [18:51:28.183] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:66e09b835ea4fc25 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:66e09b835ea4fc25"} 18:51:28 [18:51:28.184] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.185] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c93733ebe116f97a epochNumber=1 {"provingJobId":"1:BASE_PARITY:c93733ebe116f97a"} 18:51:28 [18:51:28.286] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c93733ebe116f97a timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:c93733ebe116f97a"} 18:51:28 [18:51:28.293] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:ee2e9bd328011de3 epochNumber=10 {"provingJobId":"1:BASE_PARITY:ee2e9bd328011de3"} 18:51:28 [18:51:28.312] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:28 [18:51:28.419] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.420] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f017e0f026cffbd0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:f017e0f026cffbd0"} 18:51:28 [18:51:28.570] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:f017e0f026cffbd0 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:f017e0f026cffbd0"} 18:51:28 [18:51:28.621] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.622] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6ade0dd0fdba928e epochNumber=1 {"provingJobId":"1:BASE_PARITY:6ade0dd0fdba928e"} 18:51:28 [18:51:28.622] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:6ade0dd0fdba928e type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:6ade0dd0fdba928e"} 18:51:28 [18:51:28.623] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.624] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8a0017f33423d771 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8a0017f33423d771"} 18:51:28 [18:51:28.625] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8a0017f33423d771 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:8a0017f33423d771"} 18:51:28 [18:51:28.625] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8a0017f33423d771 type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:8a0017f33423d771"} 18:51:28 [18:51:28.626] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.626] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:26594502f33fce9b epochNumber=1 {"provingJobId":"1:BASE_PARITY:26594502f33fce9b"} 18:51:28 [18:51:28.627] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:26594502f33fce9b type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:26594502f33fce9b"} 18:51:28 [18:51:28.628] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.628] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4de3b643c6a51eb3 epochNumber=1 {"provingJobId":"1:BASE_PARITY:4de3b643c6a51eb3"} 18:51:28 [18:51:28.629] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:4de3b643c6a51eb3 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:4de3b643c6a51eb3"} 18:51:28 [18:51:28.629] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:282b3e2ee44e171b epochNumber=3 {"provingJobId":"1:BASE_PARITY:282b3e2ee44e171b"} 18:51:28 [18:51:28.653] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:28 [18:51:28.653] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:4de3b643c6a51eb3 err=test error {"provingJoId":"1:BASE_PARITY:4de3b643c6a51eb3"} 18:51:28 [18:51:28.655] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.655] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:78f8060119b0327b settled=false {"provingJobId":"1:BASE_PARITY:78f8060119b0327b","status":"pending"} 18:51:28 [18:51:28.655] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:afed5614181799dd settled=false {"provingJobId":"1:BASE_PARITY:afed5614181799dd","status":"pending"} 18:51:28 [18:51:28.657] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.658] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:ddb5179282bff48a settled=true {"provingJobId":"1:BASE_PARITY:ddb5179282bff48a","status":"fulfilled"} 18:51:28 [18:51:28.658] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:baf348cc11ab8ec7 settled=true {"provingJobId":"2:BASE_PARITY:baf348cc11ab8ec7","status":"fulfilled"} 18:51:28 [18:51:28.659] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.659] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f7ab4cde78356265 settled=true {"provingJobId":"1:BASE_PARITY:f7ab4cde78356265","status":"fulfilled"} 18:51:28 [18:51:28.659] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:99c659d7037da99d settled=false {"provingJobId":"1:BASE_PARITY:99c659d7037da99d","status":"pending"} 18:51:28 [18:51:28.660] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.661] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7057de0d7408fc53 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7057de0d7408fc53"} 18:51:28 [18:51:28.663] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.663] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aa4a61abdae5f728 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aa4a61abdae5f728"} 18:51:28 [18:51:28.664] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:aa4a61abdae5f728: Error: db error: Error: db error 18:51:28 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:aa4a61abdae5f728"} 18:51:28 [18:51:28.665] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.665] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:2cd1c2e9388108d8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:2cd1c2e9388108d8"} 18:51:28 [18:51:28.666] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:2cd1c2e9388108d8 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:2cd1c2e9388108d8"} 18:51:28 [18:51:28.666] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:2cd1c2e9388108d8 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:2cd1c2e9388108d8"} 18:51:28 [18:51:28.667] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.667] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c6b0acdf4562458d epochNumber=1 {"provingJobId":"1:BASE_PARITY:c6b0acdf4562458d"} 18:51:28 [18:51:28.668] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c6b0acdf4562458d type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c6b0acdf4562458d"} 18:51:28 [18:51:28.668] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:c6b0acdf4562458d type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:c6b0acdf4562458d"} 18:51:28 [18:51:28.668] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:c6b0acdf4562458d: Error: db error 18:51:28 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:c6b0acdf4562458d"} 18:51:28 [18:51:28.669] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.670] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c2474e3cd170c838 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c2474e3cd170c838"} 18:51:28 [18:51:28.670] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c2474e3cd170c838 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:c2474e3cd170c838"} 18:51:28 [18:51:28.670] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:c2474e3cd170c838 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:c2474e3cd170c838"} 18:51:28 [18:51:28.672] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.672] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ea548fb96318917 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8ea548fb96318917"} 18:51:28 [18:51:28.672] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8ea548fb96318917 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8ea548fb96318917"} 18:51:28 [18:51:28.672] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:8ea548fb96318917 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:8ea548fb96318917"} 18:51:28 [18:51:28.673] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:8ea548fb96318917 jobErr=test error: Error: db error 18:51:28 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:8ea548fb96318917"} 18:51:28 [18:51:28.674] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.675] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:c4beb1c59388043d not found {"provingJobId":"1:BASE_PARITY:c4beb1c59388043d"} 18:51:28 [18:51:28.678] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.679] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:1e8557e50a1c378d err=test error {"provingJoId":"1:BASE_PARITY:1e8557e50a1c378d"} 18:51:28 [18:51:28.680] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.691] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f1182da9fd4cc49 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f1182da9fd4cc49"} 18:51:28 [18:51:28.692] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:3f1182da9fd4cc49 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:3f1182da9fd4cc49"} 18:51:28 [18:51:28.692] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:3f1182da9fd4cc49 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:3f1182da9fd4cc49"} 18:51:28 [18:51:28.702] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:71aa191285737e22 epochNumber=2 {"provingJobId":"2:BASE_PARITY:71aa191285737e22"} 18:51:28 [18:51:28.702] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:71aa191285737e22 type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:71aa191285737e22"} 18:51:28 [18:51:28.702] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:71aa191285737e22 type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:71aa191285737e22"} 18:51:28 [18:51:28.763] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:263f2cf00a35e906 epochNumber=3 {"provingJobId":"3:BASE_PARITY:263f2cf00a35e906"} 18:51:28 [18:51:28.781] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:28 [18:51:28.825] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:93110a57a44038d1 epochNumber=4 {"provingJobId":"4:BASE_PARITY:93110a57a44038d1"} 18:51:28 [18:51:28.831] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:28 [18:51:28.886] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:f75f8bb36edda4f9 epochNumber=5 {"provingJobId":"5:BASE_PARITY:f75f8bb36edda4f9"} 18:51:28 [18:51:28.907] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:28 [18:51:28.939] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.939] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5ee95c2d6e0b0d94 epochNumber=42 {"provingJobId":"1:BASE_PARITY:5ee95c2d6e0b0d94"} 18:51:28 [18:51:28.940] INFO: prover-client:proving-broker-database Creating broker database for epoch 42 at /tmp/proving-broker-testYA5kOk/42 with map size 1073741824 18:51:28 [18:51:28.945] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:28 [18:51:28.964] WARN: prover-client:proving-broker Tried enqueueing stale proving job id=1:BASE_PARITY:b73e5319c5110522 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b73e5319c5110522"} 18:51:28 [18:51:28.968] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.968] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:5709fd39cd948650 epochNumber=1 {"provingJobId":"1:BASE_PARITY:5709fd39cd948650"} 18:51:28 [18:51:28.969] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzRB1JU/1 with map size 1073741824 18:51:28 [18:51:28.970] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:28 [18:51:28.986] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:77330c6510213983 epochNumber=1 {"provingJobId":"1:BASE_PARITY:77330c6510213983"} 18:51:28 [18:51:28.993] INFO: prover-client:proving-broker Proving Broker started 18:51:28 [18:51:28.994] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:46885f29549618b9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:46885f29549618b9"} 18:51:28 [18:51:28.994] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCkvB3F/1 with map size 1073741824 18:51:28 [18:51:28.995] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.010] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:46885f29549618b9 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:46885f29549618b9"} 18:51:29 [18:51:29.014] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.014] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6830f307ea284ef1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:6830f307ea284ef1"} 18:51:29 [18:51:29.014] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testM7T3eB/1 with map size 1073741824 18:51:29 [18:51:29.016] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.032] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6830f307ea284ef1 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:6830f307ea284ef1"} 18:51:29 [18:51:29.032] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6830f307ea284ef1 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6830f307ea284ef1"} 18:51:29 [18:51:29.038] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:6830f307ea284ef1 epochNumber=1. Not enqueuing again {"provingJobId":"1:BASE_PARITY:6830f307ea284ef1"} 18:51:29 [18:51:29.040] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.041] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:58c6c82fb9e710ab epochNumber=1 {"provingJobId":"1:BASE_PARITY:58c6c82fb9e710ab"} 18:51:29 [18:51:29.043] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test6NZF13/1 with map size 1073741824 18:51:29 [18:51:29.044] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.063] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.065] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.066] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:411edde9db62934f epochNumber=1 {"provingJobId":"1:BASE_PARITY:411edde9db62934f"} 18:51:29 [18:51:29.066] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testARz0Xv/1 with map size 1073741824 18:51:29 [18:51:29.068] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.082] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:411edde9db62934f {"provingJobId":"1:BASE_PARITY:411edde9db62934f"} 18:51:29 [18:51:29.082] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:411edde9db62934f type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:411edde9db62934f"} 18:51:29 [18:51:29.083] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:411edde9db62934f type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:411edde9db62934f"} 18:51:29 [18:51:29.090] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.091] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b5fdcdb20a048966 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b5fdcdb20a048966"} 18:51:29 [18:51:29.091] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testiFohBL/1 with map size 1073741824 18:51:29 [18:51:29.092] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.106] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:b5fdcdb20a048966 {"provingJobId":"1:BASE_PARITY:b5fdcdb20a048966"} 18:51:29 [18:51:29.107] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b5fdcdb20a048966 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:b5fdcdb20a048966"} 18:51:29 [18:51:29.114] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.115] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:6d03194b05c4eeca epochNumber=1 {"provingJobId":"1:BASE_PARITY:6d03194b05c4eeca"} 18:51:29 [18:51:29.115] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testF19QQp/1 with map size 1073741824 18:51:29 [18:51:29.117] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.131] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:6d03194b05c4eeca type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:6d03194b05c4eeca"} 18:51:29 [18:51:29.131] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:6d03194b05c4eeca type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:6d03194b05c4eeca"} 18:51:29 [18:51:29.138] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.139] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4278ceeba6f6978e epochNumber=1 {"provingJobId":"1:BASE_PARITY:4278ceeba6f6978e"} 18:51:29 [18:51:29.139] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzXW4ay/1 with map size 1073741824 18:51:29 [18:51:29.141] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.156] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:4278ceeba6f6978e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:4278ceeba6f6978e"} 18:51:29 [18:51:29.156] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4278ceeba6f6978e type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4278ceeba6f6978e"} 18:51:29 [18:51:29.163] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.164] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e130210952c6fc7 epochNumber=0 {"provingJobId":"1:BASE_PARITY:7e130210952c6fc7"} 18:51:29 [18:51:29.164] INFO: prover-client:proving-broker-database Creating broker database for epoch 0 at /tmp/proving-broker-testd3FHKW/0 with map size 1073741824 18:51:29 [18:51:29.165] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.180] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7e130210952c6fc7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7e130210952c6fc7"} 18:51:29 [18:51:29.180] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8d8a62c55e1de2d9 epochNumber=0 {"provingJobId":"1:BASE_PARITY:8d8a62c55e1de2d9"} 18:51:29 [18:51:29.181] WARN: prover-client:proving-broker Cached proving job id=1:BASE_PARITY:7e130210952c6fc7 epochNumber=0. Not enqueuing again {"provingJobId":"1:BASE_PARITY:7e130210952c6fc7"} 18:51:29 [18:51:29.181] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:cb629739ebb4109b epochNumber=0 {"provingJobId":"1:BASE_PARITY:cb629739ebb4109b"} 18:51:29 [18:51:29.192] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.194] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.194] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:9b486b59149585f3 epochNumber=2 {"provingJobId":"1:BASE_PARITY:9b486b59149585f3"} 18:51:29 [18:51:29.195] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testX86IrO/2 with map size 1073741824 18:51:29 [18:51:29.196] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.210] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a45f8224dd52a0da epochNumber=1 {"provingJobId":"1:BASE_PARITY:a45f8224dd52a0da"} 18:51:29 [18:51:29.210] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testX86IrO/1 with map size 1073741824 18:51:29 [18:51:29.212] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.228] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.228] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bb6d42c87bfb52a4 epochNumber=1 {"provingJobId":"1:BASE_PARITY:bb6d42c87bfb52a4"} 18:51:29 [18:51:29.229] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnXYnt7/1 with map size 1073741824 18:51:29 [18:51:29.230] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.246] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.246] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7726166d42ccd9b5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7726166d42ccd9b5"} 18:51:29 [18:51:29.247] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testb6yewc/1 with map size 1073741824 18:51:29 [18:51:29.248] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.262] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f452edccb42426d epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f452edccb42426d"} 18:51:29 [18:51:29.266] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fb9d75aa73f81cb8 epochNumber=2 {"provingJobId":"1:BASE_PARITY:fb9d75aa73f81cb8"} 18:51:29 [18:51:29.267] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testb6yewc/2 with map size 1073741824 18:51:29 [18:51:29.268] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.284] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:84069a841f1e73c0 epochNumber=1 {"provingJobId":"1:BASE_PARITY:84069a841f1e73c0"} 18:51:29 [18:51:29.291] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.292] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8da7947370c45140 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8da7947370c45140"} 18:51:29 [18:51:29.292] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testzSCDPi/1 with map size 1073741824 18:51:29 [18:51:29.293] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.307] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:27130316a69583fb epochNumber=1 {"provingJobId":"1:BASE_PARITY:27130316a69583fb"} 18:51:29 [18:51:29.312] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:741399b02280565a epochNumber=2 {"provingJobId":"1:BASE_PARITY:741399b02280565a"} 18:51:29 [18:51:29.313] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testzSCDPi/2 with map size 1073741824 18:51:29 [18:51:29.314] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.328] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:aaa2cef84d3ed641 epochNumber=1 {"provingJobId":"1:BASE_PARITY:aaa2cef84d3ed641"} 18:51:29 [18:51:29.335] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.335] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a85901d7f122585e epochNumber=1 {"provingJobId":"1:BASE_PARITY:a85901d7f122585e"} 18:51:29 [18:51:29.336] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testdgDcFf/1 with map size 1073741824 18:51:29 [18:51:29.337] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.352] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:376e1975fb473533 epochNumber=1 {"provingJobId":"1:BASE_PARITY:376e1975fb473533"} 18:51:29 [18:51:29.357] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:15790002faf0df92 epochNumber=2 {"provingJobId":"1:BASE_PARITY:15790002faf0df92"} 18:51:29 [18:51:29.357] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testdgDcFf/2 with map size 1073741824 18:51:29 [18:51:29.359] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.372] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:470e8f51f763d85d epochNumber=1 {"provingJobId":"1:BASE_PARITY:470e8f51f763d85d"} 18:51:29 [18:51:29.379] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.380] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51e5f75c465c9456 epochNumber=1 {"provingJobId":"1:BASE_PARITY:51e5f75c465c9456"} 18:51:29 [18:51:29.381] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testXJ7fhe/1 with map size 1073741824 18:51:29 [18:51:29.382] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.397] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:f3c0f1104d8cacef epochNumber=1 {"provingJobId":"1:BASE_PARITY:f3c0f1104d8cacef"} 18:51:29 [18:51:29.402] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:51e5f75c465c9456 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:51e5f75c465c9456"} 18:51:29 [18:51:29.409] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.410] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b33c0b687f850622 epochNumber=1 {"provingJobId":"1:BASE_PARITY:b33c0b687f850622"} 18:51:29 [18:51:29.410] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQ3UU5B/1 with map size 1073741824 18:51:29 [18:51:29.412] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.426] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:c23d817f927623e8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:c23d817f927623e8"} 18:51:29 [18:51:29.430] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:b33c0b687f850622 type=BASE_PARITY totalAttempts=1 err=result {"provingJobId":"1:BASE_PARITY:b33c0b687f850622"} 18:51:29 [18:51:29.438] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.439] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:87b2f07c7b457187 epochNumber=1 {"provingJobId":"1:BASE_PARITY:87b2f07c7b457187"} 18:51:29 [18:51:29.439] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLJy5XH/1 with map size 1073741824 18:51:29 [18:51:29.441] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.456] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:fe86be52d7eebfb1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:fe86be52d7eebfb1"} 18:51:29 [18:51:29.461] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:87b2f07c7b457187 type=BASE_PARITY retry=1 err=result {"provingJobId":"1:BASE_PARITY:87b2f07c7b457187"} 18:51:29 [18:51:29.464] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.464] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:af76a43616c7b269 epochNumber=1 {"provingJobId":"1:BASE_PARITY:af76a43616c7b269"} 18:51:29 [18:51:29.465] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testCcPbv4/1 with map size 1073741824 18:51:29 [18:51:29.466] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.480] INFO: prover-client:proving-broker Cancelling job id=1:BASE_PARITY:af76a43616c7b269 {"provingJobId":"1:BASE_PARITY:af76a43616c7b269"} 18:51:29 [18:51:29.480] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:af76a43616c7b269 type=BASE_PARITY totalAttempts=1 err=Aborted {"provingJobId":"1:BASE_PARITY:af76a43616c7b269"} 18:51:29 [18:51:29.485] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3810a580970bf49a epochNumber=1 {"provingJobId":"1:BASE_PARITY:3810a580970bf49a"} 18:51:29 [18:51:29.490] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:af76a43616c7b269 has already been completed {"provingJobId":"1:BASE_PARITY:af76a43616c7b269"} 18:51:29 [18:51:29.493] INFO: prover-client:proving-broker Proving Broker started 18:51:29 [18:51:29.493] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:440c3d05f16b78c1 epochNumber=1 {"provingJobId":"1:BASE_PARITY:440c3d05f16b78c1"} 18:51:29 [18:51:29.494] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testTgmbrY/1 with map size 1073741824 18:51:29 [18:51:29.495] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:29 [18:51:29.508] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:09872607dca1cd59 epochNumber=2 {"provingJobId":"1:BASE_PARITY:09872607dca1cd59"} 18:51:29 [18:51:29.509] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testTgmbrY/2 with map size 1073741824 18:51:29 [18:51:29.510] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:30 [18:51:30.576] INFO: prover-client:proving-broker Proving Broker started 18:51:30 [18:51:30.582] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:440c3d05f16b78c1 settled=false {"provingJobId":"1:BASE_PARITY:440c3d05f16b78c1","status":"pending"} 18:51:30 [18:51:30.584] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:09872607dca1cd59 settled=false {"provingJobId":"1:BASE_PARITY:09872607dca1cd59","status":"pending"} 18:51:30 [18:51:30.585] INFO: prover-client:proving-broker Proving job id=1:BASE_PARITY:440c3d05f16b78c1 type=BASE_PARITY startedAt=1750272689523 older agent has taken job {"provingJobId":"1:BASE_PARITY:440c3d05f16b78c1"} 18:51:30 [18:51:30.585] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:440c3d05f16b78c1 type=BASE_PARITY already being worked on by another agent. Sending new one {"provingJobId":"1:BASE_PARITY:440c3d05f16b78c1"} 18:51:30 [18:51:30.588] INFO: prover-client:proving-broker Proving Broker started 18:51:30 [18:51:30.589] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7e8cc74ea97ab3a2 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7e8cc74ea97ab3a2"} 18:51:30 [18:51:30.589] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBp1lJU/1 with map size 1073741824 18:51:30 [18:51:30.590] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:30 [18:51:30.607] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:139144e36b0342ea epochNumber=2 {"provingJobId":"1:BASE_PARITY:139144e36b0342ea"} 18:51:30 [18:51:30.608] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testBp1lJU/2 with map size 1073741824 18:51:30 [18:51:30.609] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:31 [18:51:31.630] INFO: prover-client:proving-broker Proving Broker started 18:51:31 [18:51:31.632] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:7e8cc74ea97ab3a2 settled=false {"provingJobId":"1:BASE_PARITY:7e8cc74ea97ab3a2","status":"pending"} 18:51:31 [18:51:31.633] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:139144e36b0342ea settled=false {"provingJobId":"1:BASE_PARITY:139144e36b0342ea","status":"pending"} 18:51:31 [18:51:31.634] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7e8cc74ea97ab3a2 type=BASE_PARITY not found in the in-progress cache, adding it {"provingJobId":"1:BASE_PARITY:7e8cc74ea97ab3a2"} 18:51:31 [18:51:31.637] INFO: prover-client:proving-broker Proving Broker started 18:51:31 [18:51:31.637] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:58434cfdc922a7b9 epochNumber=1 {"provingJobId":"1:BASE_PARITY:58434cfdc922a7b9"} 18:51:31 [18:51:31.638] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test09adBl/1 with map size 1073741824 18:51:31 [18:51:31.639] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:31 [18:51:31.657] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:b0af7c30d24de9f0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:b0af7c30d24de9f0"} 18:51:31 [18:51:31.657] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-test09adBl/2 with map size 1073741824 18:51:31 [18:51:31.659] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.673] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.675] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:58434cfdc922a7b9 settled=false {"provingJobId":"1:BASE_PARITY:58434cfdc922a7b9","status":"pending"} 18:51:32 [18:51:32.677] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:b0af7c30d24de9f0 settled=false {"provingJobId":"1:BASE_PARITY:b0af7c30d24de9f0","status":"pending"} 18:51:32 [18:51:32.678] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:58434cfdc922a7b9 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:58434cfdc922a7b9"} 18:51:32 [18:51:32.678] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:58434cfdc922a7b9 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:58434cfdc922a7b9"} 18:51:32 [18:51:32.688] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.689] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:636cfa99dfe05696 epochNumber=1 {"provingJobId":"1:BASE_PARITY:636cfa99dfe05696"} 18:51:32 [18:51:32.689] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgoSavG/1 with map size 1073741824 18:51:32 [18:51:32.690] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.706] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:4364bbbbf0e7c989 epochNumber=2 {"provingJobId":"1:BASE_PARITY:4364bbbbf0e7c989"} 18:51:32 [18:51:32.706] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testgoSavG/2 with map size 1073741824 18:51:32 [18:51:32.707] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.721] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:636cfa99dfe05696 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:636cfa99dfe05696"} 18:51:32 [18:51:32.727] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:4364bbbbf0e7c989 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:4364bbbbf0e7c989"} 18:51:32 [18:51:32.735] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.736] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a3d12cbe65f3b313 epochNumber=1 {"provingJobId":"1:BASE_PARITY:a3d12cbe65f3b313"} 18:51:32 [18:51:32.736] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testQnusK6/1 with map size 1073741824 18:51:32 [18:51:32.739] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.753] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:754ece310b7304f0 epochNumber=2 {"provingJobId":"1:BASE_PARITY:754ece310b7304f0"} 18:51:32 [18:51:32.754] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testQnusK6/2 with map size 1073741824 18:51:32 [18:51:32.755] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.770] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a3d12cbe65f3b313 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a3d12cbe65f3b313"} 18:51:32 [18:51:32.770] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a3d12cbe65f3b313 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a3d12cbe65f3b313"} 18:51:32 [18:51:32.776] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:754ece310b7304f0 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:754ece310b7304f0"} 18:51:32 [18:51:32.783] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.784] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:ab5a25896ecf3976 err=test error {"provingJoId":"1:BASE_PARITY:ab5a25896ecf3976"} 18:51:32 [18:51:32.787] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.788] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:9c735026c083453b not found {"provingJobId":"1:BASE_PARITY:9c735026c083453b"} 18:51:32 [18:51:32.791] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.791] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7375f081cdf5c727 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7375f081cdf5c727"} 18:51:32 [18:51:32.792] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testw9WpA9/1 with map size 1073741824 18:51:32 [18:51:32.793] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.809] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.810] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:48205ca55ff4b355 epochNumber=1 {"provingJobId":"1:BASE_PARITY:48205ca55ff4b355"} 18:51:32 [18:51:32.811] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testBjEHLz/1 with map size 1073741824 18:51:32 [18:51:32.812] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:32 [18:51:32.935] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:48205ca55ff4b355 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:48205ca55ff4b355"} 18:51:32 [18:51:32.941] INFO: prover-client:proving-broker Proving Broker started 18:51:32 [18:51:32.941] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d3774348dec283a8 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d3774348dec283a8"} 18:51:32 [18:51:32.942] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testHLhq5g/1 with map size 1073741824 18:51:32 [18:51:32.943] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.069] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d3774348dec283a8 timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d3774348dec283a8"} 18:51:33 [18:51:33.073] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:044b9bf01dbd7906 epochNumber=10 {"provingJobId":"1:BASE_PARITY:044b9bf01dbd7906"} 18:51:33 [18:51:33.073] INFO: prover-client:proving-broker-database Creating broker database for epoch 10 at /tmp/proving-broker-testHLhq5g/10 with map size 1073741824 18:51:33 [18:51:33.074] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.095] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:33 [18:51:33.096] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:51:33 [18:51:33.217] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.218] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d47e154c09d7d05e epochNumber=1 {"provingJobId":"1:BASE_PARITY:d47e154c09d7d05e"} 18:51:33 [18:51:33.218] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test3ZpxAh/1 with map size 1073741824 18:51:33 [18:51:33.219] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.394] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:d47e154c09d7d05e timed out. Adding it back to the queue. {"provingJobId":"1:BASE_PARITY:d47e154c09d7d05e"} 18:51:33 [18:51:33.438] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.439] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:d08d6a5a45456673 epochNumber=1 {"provingJobId":"1:BASE_PARITY:d08d6a5a45456673"} 18:51:33 [18:51:33.439] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testx0hnl0/1 with map size 1073741824 18:51:33 [18:51:33.440] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.457] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:d08d6a5a45456673 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:d08d6a5a45456673"} 18:51:33 [18:51:33.460] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.460] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:e0968cdd34b1ad6f epochNumber=1 {"provingJobId":"1:BASE_PARITY:e0968cdd34b1ad6f"} 18:51:33 [18:51:33.461] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testu4kwIU/1 with map size 1073741824 18:51:33 [18:51:33.462] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.477] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:e0968cdd34b1ad6f type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:e0968cdd34b1ad6f"} 18:51:33 [18:51:33.477] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:e0968cdd34b1ad6f type=BASE_PARITY totalAttempts=2 err=test error {"provingJobId":"1:BASE_PARITY:e0968cdd34b1ad6f"} 18:51:33 [18:51:33.485] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.485] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:3f268a57c5117e72 epochNumber=1 {"provingJobId":"1:BASE_PARITY:3f268a57c5117e72"} 18:51:33 [18:51:33.486] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testLi0X2M/1 with map size 1073741824 18:51:33 [18:51:33.488] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.504] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:3f268a57c5117e72 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:3f268a57c5117e72"} 18:51:33 [18:51:33.511] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.512] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8f8191315e13c6d5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:8f8191315e13c6d5"} 18:51:33 [18:51:33.512] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testgLCqjV/1 with map size 1073741824 18:51:33 [18:51:33.514] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.528] INFO: prover-client:proving-broker Retrying proving job id=1:BASE_PARITY:8f8191315e13c6d5 type=BASE_PARITY retry=1 err=test error {"provingJobId":"1:BASE_PARITY:8f8191315e13c6d5"} 18:51:33 [18:51:33.529] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:bcf80f794e5ae9f0 epochNumber=3 {"provingJobId":"1:BASE_PARITY:bcf80f794e5ae9f0"} 18:51:33 [18:51:33.529] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testgLCqjV/3 with map size 1073741824 18:51:33 [18:51:33.531] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.562] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:33 [18:51:33.562] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:51:33 [18:51:33.570] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:8f8191315e13c6d5 err=test error {"provingJoId":"1:BASE_PARITY:8f8191315e13c6d5"} 18:51:33 [18:51:33.574] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testkDxMnb/1 with map size 1073741824 18:51:33 [18:51:33.575] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.590] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testkDxMnb/2 with map size 1073741824 18:51:33 [18:51:33.592] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.605] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.607] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:2f5678ba7a4bb9dc settled=false {"provingJobId":"1:BASE_PARITY:2f5678ba7a4bb9dc","status":"pending"} 18:51:33 [18:51:33.609] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:f05f881a13d2e05d settled=false {"provingJobId":"1:BASE_PARITY:f05f881a13d2e05d","status":"pending"} 18:51:33 [18:51:33.613] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testjSt1KW/1 with map size 1073741824 18:51:33 [18:51:33.615] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.629] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testjSt1KW/2 with map size 1073741824 18:51:33 [18:51:33.631] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.655] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.657] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:cea5aa98b23fdaa3 settled=true {"provingJobId":"1:BASE_PARITY:cea5aa98b23fdaa3","status":"fulfilled"} 18:51:33 [18:51:33.658] INFO: prover-client:proving-broker Restoring proving job id=2:BASE_PARITY:8c1ffc41f44985a8 settled=true {"provingJobId":"2:BASE_PARITY:8c1ffc41f44985a8","status":"fulfilled"} 18:51:33 [18:51:33.662] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testSL2vLE/1 with map size 1073741824 18:51:33 [18:51:33.664] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.683] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testSL2vLE/2 with map size 1073741824 18:51:33 [18:51:33.684] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.698] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.699] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:426e5798a49afd9a settled=true {"provingJobId":"1:BASE_PARITY:426e5798a49afd9a","status":"fulfilled"} 18:51:33 [18:51:33.701] INFO: prover-client:proving-broker Restoring proving job id=1:BASE_PARITY:19f7aacb214ca6f8 settled=false {"provingJobId":"1:BASE_PARITY:19f7aacb214ca6f8","status":"pending"} 18:51:33 [18:51:33.705] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.706] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:0d05f19908772280 epochNumber=1 {"provingJobId":"1:BASE_PARITY:0d05f19908772280"} 18:51:33 [18:51:33.706] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test8s4TuI/1 with map size 1073741824 18:51:33 [18:51:33.708] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.725] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.726] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:52fbd44fd30796d7 epochNumber=1 {"provingJobId":"1:BASE_PARITY:52fbd44fd30796d7"} 18:51:33 [18:51:33.726] ERROR: prover-client:proving-broker Failed to save proving job id=1:BASE_PARITY:52fbd44fd30796d7: Error: db error: Error: db error 18:51:33 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1126:63) {"provingJobId":"1:BASE_PARITY:52fbd44fd30796d7"} 18:51:33 [18:51:33.730] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.731] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:8ad211127be1aa7e epochNumber=1 {"provingJobId":"1:BASE_PARITY:8ad211127be1aa7e"} 18:51:33 [18:51:33.731] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testY3k2ro/1 with map size 1073741824 18:51:33 [18:51:33.733] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.747] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:8ad211127be1aa7e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:8ad211127be1aa7e"} 18:51:33 [18:51:33.747] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:8ad211127be1aa7e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:8ad211127be1aa7e"} 18:51:33 [18:51:33.755] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.756] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:7787141e302e34c5 epochNumber=1 {"provingJobId":"1:BASE_PARITY:7787141e302e34c5"} 18:51:33 [18:51:33.757] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testlIiroI/1 with map size 1073741824 18:51:33 [18:51:33.758] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.772] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:7787141e302e34c5 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:7787141e302e34c5"} 18:51:33 [18:51:33.772] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:7787141e302e34c5 type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:7787141e302e34c5"} 18:51:33 [18:51:33.773] ERROR: prover-client:proving-broker Failed to save proving job result id=1:BASE_PARITY:7787141e302e34c5: Error: db error 18:51:33 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1159:69) {"provingJobId":"1:BASE_PARITY:7787141e302e34c5"} 18:51:33 [18:51:33.776] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.777] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:51e4ab1d87e3d178 epochNumber=1 {"provingJobId":"1:BASE_PARITY:51e4ab1d87e3d178"} 18:51:33 [18:51:33.777] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test0LUI6z/1 with map size 1073741824 18:51:33 [18:51:33.778] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.792] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:51e4ab1d87e3d178 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:51e4ab1d87e3d178"} 18:51:33 [18:51:33.792] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:51e4ab1d87e3d178 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:51e4ab1d87e3d178"} 18:51:33 [18:51:33.801] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.802] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:257b44f298910f35 epochNumber=1 {"provingJobId":"1:BASE_PARITY:257b44f298910f35"} 18:51:33 [18:51:33.802] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-test2bXkUg/1 with map size 1073741824 18:51:33 [18:51:33.804] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.818] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:257b44f298910f35 type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:257b44f298910f35"} 18:51:33 [18:51:33.818] INFO: prover-client:proving-broker Marking proving job as failed id=1:BASE_PARITY:257b44f298910f35 type=BASE_PARITY totalAttempts=1 err=test error {"provingJobId":"1:BASE_PARITY:257b44f298910f35"} 18:51:33 [18:51:33.819] ERROR: prover-client:proving-broker Failed to save proving job error status id=1:BASE_PARITY:257b44f298910f35 jobErr=test error: Error: db error 18:51:33 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/proving_broker.test.ts:1192:68) {"provingJobId":"1:BASE_PARITY:257b44f298910f35"} 18:51:33 [18:51:33.822] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.823] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:47e46c77e4476fd9 not found {"provingJobId":"1:BASE_PARITY:47e46c77e4476fd9"} 18:51:33 [18:51:33.826] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.826] WARN: prover-client:proving-broker Can't set error on unknown proving job id=1:BASE_PARITY:3336ddfb52214d2d err=test error {"provingJoId":"1:BASE_PARITY:3336ddfb52214d2d"} 18:51:33 [18:51:33.829] INFO: prover-client:proving-broker Proving Broker started 18:51:33 [18:51:33.840] INFO: prover-client:proving-broker New proving job id=1:BASE_PARITY:a8408d9deea6329e epochNumber=1 {"provingJobId":"1:BASE_PARITY:a8408d9deea6329e"} 18:51:33 [18:51:33.840] INFO: prover-client:proving-broker-database Creating broker database for epoch 1 at /tmp/proving-broker-testnT2Th7/1 with map size 1073741824 18:51:33 [18:51:33.843] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.857] WARN: prover-client:proving-broker Proving job id=1:BASE_PARITY:a8408d9deea6329e type=BASE_PARITY not in the in-progress set {"provingJobId":"1:BASE_PARITY:a8408d9deea6329e"} 18:51:33 [18:51:33.857] INFO: prover-client:proving-broker Proving job complete id=1:BASE_PARITY:a8408d9deea6329e type=BASE_PARITY totalAttempts=1 {"provingJobId":"1:BASE_PARITY:a8408d9deea6329e"} 18:51:33 [18:51:33.872] INFO: prover-client:proving-broker New proving job id=2:BASE_PARITY:d1beafa5f58cc4ae epochNumber=2 {"provingJobId":"2:BASE_PARITY:d1beafa5f58cc4ae"} 18:51:33 [18:51:33.873] INFO: prover-client:proving-broker-database Creating broker database for epoch 2 at /tmp/proving-broker-testnT2Th7/2 with map size 1073741824 18:51:33 [18:51:33.874] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.888] WARN: prover-client:proving-broker Proving job id=2:BASE_PARITY:d1beafa5f58cc4ae type=BASE_PARITY not in the in-progress set {"provingJobId":"2:BASE_PARITY:d1beafa5f58cc4ae"} 18:51:33 [18:51:33.889] INFO: prover-client:proving-broker Proving job complete id=2:BASE_PARITY:d1beafa5f58cc4ae type=BASE_PARITY totalAttempts=1 {"provingJobId":"2:BASE_PARITY:d1beafa5f58cc4ae"} 18:51:33 [18:51:33.954] INFO: prover-client:proving-broker New proving job id=3:BASE_PARITY:a9f1d295c9007a30 epochNumber=3 {"provingJobId":"3:BASE_PARITY:a9f1d295c9007a30"} 18:51:33 [18:51:33.955] INFO: prover-client:proving-broker-database Creating broker database for epoch 3 at /tmp/proving-broker-testnT2Th7/3 with map size 1073741824 18:51:33 [18:51:33.956] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:33 [18:51:33.980] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:33 [18:51:33.980] INFO: prover-client:proving-broker-database Deleting broker database for epoch 1 18:51:34 [18:51:34.030] INFO: prover-client:proving-broker New proving job id=4:BASE_PARITY:a248f4eaa2c998d7 epochNumber=4 {"provingJobId":"4:BASE_PARITY:a248f4eaa2c998d7"} 18:51:34 [18:51:34.031] INFO: prover-client:proving-broker-database Creating broker database for epoch 4 at /tmp/proving-broker-testnT2Th7/4 with map size 1073741824 18:51:34 [18:51:34.032] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:34 [18:51:34.056] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:34 [18:51:34.057] INFO: prover-client:proving-broker-database Deleting broker database for epoch 2 18:51:34 [18:51:34.107] INFO: prover-client:proving-broker New proving job id=5:BASE_PARITY:bf42943353b44a95 epochNumber=5 {"provingJobId":"5:BASE_PARITY:bf42943353b44a95"} 18:51:34 [18:51:34.108] INFO: prover-client:proving-broker-database Creating broker database for epoch 5 at /tmp/proving-broker-testnT2Th7/5 with map size 1073741824 18:51:34 [18:51:34.109] INFO: kv-store:lmdb-v2 Starting data store with maxReaders 16 18:51:34 [18:51:34.133] INFO: prover-client:proving-broker Cleaned up jobs=1 18:51:34 [18:51:34.133] INFO: prover-client:proving-broker-database Deleting broker database for epoch 3 18:51:34 PASS src/proving_broker/proving_broker.test.ts (11.163 s) 18:51:34 ProvingBroker 18:51:34 Producer API 18:51:34 ✓ refuses stale jobs (35 ms) 18:51:34 ✓ enqueues jobs (3 ms) 18:51:34 ✓ ignores duplicate jobs (4 ms) 18:51:34 ✓ reports correct status when enqueuing repeat jobs (4 ms) 18:51:34 ✓ throws an error in case of duplicate job IDs (18 ms) 18:51:34 ✓ returns not-found status for non-existing jobs (1 ms) 18:51:34 ✓ cancels jobs in queue (3 ms) 18:51:34 ✓ cancels jobs in-progress (2 ms) 18:51:34 ✓ returns job result if successful (3 ms) 18:51:34 ✓ returns job error if failed (2 ms) 18:51:34 ✓ correctly returns job status for concurrent writes (3 ms) 18:51:34 ✓ refuses stale jobs (29 ms) 18:51:34 ✓ enqueues jobs (25 ms) 18:51:34 ✓ ignores duplicate jobs (20 ms) 18:51:34 ✓ reports correct status when enqueuing repeat jobs (27 ms) 18:51:34 ✓ throws an error in case of duplicate job IDs (23 ms) 18:51:34 ✓ returns not-found status for non-existing jobs (2 ms) 18:51:34 ✓ cancels jobs in queue (25 ms) 18:51:34 ✓ cancels jobs in-progress (24 ms) 18:51:34 ✓ returns job result if successful (24 ms) 18:51:34 ✓ returns job error if failed (25 ms) 18:51:34 ✓ correctly returns job status for concurrent writes (29 ms) 18:51:34 Consumer API 18:51:34 ✓ returns undefined if no jobs are available (1 ms) 18:51:34 ✓ returns jobs in priority order (2 ms) 18:51:34 ✓ returns undefined if no jobs are available for the given allowList (5 ms) 18:51:34 ✓ returns a job if it is in the allowList (3 ms) 18:51:34 ✓ returns the most important job if it is in the allowList (2 ms) 18:51:34 ✓ returns any job if filter is empty (2 ms) 18:51:34 ✓ returns a new job when reporting job success (2 ms) 18:51:34 ✓ returns a new job when reporting permanent error (1 ms) 18:51:34 ✓ returns a new job when reporting retry-able error (1 ms) 18:51:34 ✓ returns a new job when reporting progress if current one is cancelled (3 ms) 18:51:34 ✓ returns a new job if job is already in progress elsewhere (1054 ms) 18:51:34 ✓ avoids sending the same job to a new agent after a restart (1003 ms) 18:51:34 ✓ avoids sending a completed job to a new agent after a restart (1004 ms) 18:51:34 ✓ tracks job result if in progress (3 ms) 18:51:34 ✓ tracks job result even if job is in queue (2 ms) 18:51:34 ✓ ignores reported job error if unknown job (2 ms) 18:51:34 ✓ ignores job result if unknown job (1 ms) 18:51:34 ✓ returns undefined if no jobs are available (2 ms) 18:51:34 ✓ returns jobs in priority order (33 ms) 18:51:34 ✓ returns undefined if no jobs are available for the given allowList (19 ms) 18:51:34 ✓ returns a job if it is in the allowList (45 ms) 18:51:34 ✓ returns the most important job if it is in the allowList (44 ms) 18:51:34 ✓ returns any job if filter is empty (44 ms) 18:51:34 ✓ returns a new job when reporting job success (30 ms) 18:51:34 ✓ returns a new job when reporting permanent error (29 ms) 18:51:34 ✓ returns a new job when reporting retry-able error (25 ms) 18:51:34 ✓ returns a new job when reporting progress if current one is cancelled (28 ms) 18:51:34 ✓ returns a new job if job is already in progress elsewhere (1095 ms) 18:51:34 ✓ avoids sending the same job to a new agent after a restart (1048 ms) 18:51:34 ✓ avoids sending a completed job to a new agent after a restart (1051 ms) 18:51:34 ✓ tracks job result if in progress (47 ms) 18:51:34 ✓ tracks job result even if job is in queue (48 ms) 18:51:34 ✓ ignores reported job error if unknown job (3 ms) 18:51:34 ✓ ignores job result if unknown job (4 ms) 18:51:34 Timeouts 18:51:34 ✓ tracks in progress jobs (1 ms) 18:51:34 ✓ re-enqueues jobs that time out (102 ms) 18:51:34 ✓ cancel stale jobs that time out (234 ms) 18:51:34 ✓ keeps the jobs in progress while it is alive (201 ms) 18:51:34 ✓ tracks in progress jobs (19 ms) 18:51:34 ✓ re-enqueues jobs that time out (131 ms) 18:51:34 ✓ cancel stale jobs that time out (276 ms) 18:51:34 ✓ keeps the jobs in progress while it is alive (220 ms) 18:51:34 Retries 18:51:34 ✓ retries jobs (2 ms) 18:51:34 ✓ retries up to a maximum number of times (2 ms) 18:51:34 ✓ passing retry=false does not retry (1 ms) 18:51:34 ✓ does not retry if job is stale (27 ms) 18:51:34 ✓ retries jobs (21 ms) 18:51:34 ✓ retries up to a maximum number of times (25 ms) 18:51:34 ✓ passing retry=false does not retry (26 ms) 18:51:34 ✓ does not retry if job is stale (62 ms) 18:51:34 Database management 18:51:34 ✓ re-enqueues proof requests on start (3 ms) 18:51:34 ✓ restores proof results on start (1 ms) 18:51:34 ✓ only re-enqueues unfinished jobs (2 ms) 18:51:34 ✓ saves job when enqueued (2 ms) 18:51:34 ✓ does not retain job if database fails to save (3 ms) 18:51:34 ✓ saves job result (1 ms) 18:51:34 ✓ saves result even if database fails to save (2 ms) 18:51:34 ✓ saves job error (2 ms) 18:51:34 ✓ saves job error even if database fails to save (2 ms) 18:51:34 ✓ does not save job result if job is unknown (4 ms) 18:51:34 ✓ does not save job error if job is unknown (1 ms) 18:51:34 ✓ cleans up old jobs periodically (257 ms) 18:51:34 ✓ re-enqueues proof requests on start (39 ms) 18:51:34 ✓ restores proof results on start (49 ms) 18:51:34 ✓ only re-enqueues unfinished jobs (43 ms) 18:51:34 ✓ saves job when enqueued (19 ms) 18:51:34 ✓ does not retain job if database fails to save (4 ms) 18:51:34 ✓ saves job result (26 ms) 18:51:34 ✓ saves result even if database fails to save (20 ms) 18:51:34 ✓ saves job error (24 ms) 18:51:34 ✓ saves job error even if database fails to save (20 ms) 18:51:34 ✓ does not save job result if job is unknown (3 ms) 18:51:34 ✓ does not save job error if job is unknown (3 ms) 18:51:34 ✓ cleans up old jobs periodically (348 ms) 18:51:34 18:51:34 Test Suites: 1 passed, 1 total 18:51:34 Tests: 96 passed, 96 total 18:51:34 Snapshots: 0 total 18:51:34 Time: 11.235 s 18:51:34 Ran all test suites matching prover-client/src/proving_broker/proving_broker.test.ts. 18:51:34 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?