Parent Log: http://ci.aztec-labs.com/4451244b7f022098 Command: 731d98bdb3377842:ISOLATE=1:NAME=prover-node/src/job/epoch-proving-job.test.ts yarn-project/scripts/run_test.sh prover-node/src/job/epoch-proving-job.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/4110b08ca823c6af4399e58e54e0483d6e3459ed Env: REF_NAME=gh-readonly-queue/next/pr-15064-daebab2c4bace59ef25ddb29040a040926f095e6 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Mon Jun 16 09:31:45 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15064_amd64_x2-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_4ec0c4254cabba91_next 09:31:45 +++ id -u 09:31:45 +++ id -g 09:31:45 ++ docker run -d --name prover-node_src_job_epoch-proving-job.test.ts --net=none --cpuset-cpus=0-127 --cpus=2 --memory=8g --user 1000:1000 -v/home/aztec-dev:/home/aztec-dev --mount type=tmpfs,target=/tmp,tmpfs-size=1g --workdir /home/aztec-dev/aztec-packages -e HOME -e VERBOSE -e GIT_CONFIG_GLOBAL=/home/aztec-dev/aztec-packages/build-images/src/home/.gitconfig -e FORCE_COLOR=true -e CPUS -e MEM aztecprotocol/build:3.0 /bin/bash -c 'timeout -v 600s bash -c '\''yarn-project/scripts/run_test.sh prover-node/src/job/epoch-proving-job.test.ts'\''' 09:31:45 + cid=27dc7d8932ec7ba7cdc24543f932e85d443cab8b47e316f75966e3db92a1a45d 09:31:45 + set +x 09:31:49 [09:31:49.167] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"7661d0b4-ca9a-4e4a-ad69-7be200236068"} 09:31:50 [09:31:50.371] INFO: prover-node:epoch-proving-job Finalised proof for epoch 1 {"epochNumber":1,"uuid":"7661d0b4-ca9a-4e4a-ad69-7be200236068","duration":1198.1156030000002} 09:31:50 [09:31:50.375] INFO: prover-node:epoch-proving-job Submitted proof for epoch 1 (blocks 1 to 3) {"epochNumber":1,"uuid":"7661d0b4-ca9a-4e4a-ad69-7be200236068"} 09:31:50 [09:31:50.496] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"069c9e24-661a-405a-a976-b16fbeba6df4"} 09:31:51 [09:31:51.625] ERROR: prover-node:epoch-proving-job Error running epoch 1 prover job: Error: Txs failed processing: 0x241fcd4d8a0e44284d33e75ff7118995163aa95dd73ad688b5a6016095d181b4 (Error: Failed to process tx), 0x0748cd67ac482a924cc206d8b43c0abc95e39c0d2cc4049face409bb013c360d (Error: Failed to process tx) 09:31:51 at EpochProvingJob.processTxs (/home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:320:13) 09:31:51 at runNextTicks (node:internal/process/task_queues:65:5) 09:31:51 at listOnTimeout (node:internal/timers:549:9) 09:31:51 at processTimers (node:internal/timers:523:7) 09:31:51 at /home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:151:27 {"uuid":"069c9e24-661a-405a-a976-b16fbeba6df4","epochNumber":1} 09:31:51 [09:31:51.773] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"ba98d56c-9981-4e4c-9afe-aa5474d44668"} 09:31:52 [09:31:52.973] ERROR: prover-node:epoch-proving-job Error running epoch 1 prover job: Error: Failed to process all txs: processed 0 out of 2 09:31:52 at EpochProvingJob.processTxs (/home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:328:13) 09:31:52 at runNextTicks (node:internal/process/task_queues:65:5) 09:31:52 at listOnTimeout (node:internal/timers:549:9) 09:31:52 at processTimers (node:internal/timers:523:7) 09:31:52 at /home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:151:27 {"uuid":"ba98d56c-9981-4e4c-9afe-aa5474d44668","epochNumber":1} 09:31:53 [09:31:53.118] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"df669768-5755-4ffc-b08f-88b3f398b145"} 09:31:54 [09:31:54.102] WARN: prover-node:epoch-proving-job Stopping job due to deadline hit {"uuid":"df669768-5755-4ffc-b08f-88b3f398b145","epochNumber":1} 09:31:54 [09:31:54.303] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"df669768-5755-4ffc-b08f-88b3f398b145","epochNumber":1,"details":"Halted execution due to state timed-out"} 09:31:54 [09:31:54.422] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"384eb340-8c7c-4b34-8df5-c04e4dce49de"} 09:31:55 [09:31:55.568] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"384eb340-8c7c-4b34-8df5-c04e4dce49de","epochNumber":1,"details":"Halted execution due to state stopped"} 09:31:55 [09:31:55.960] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"c8cbe115-7417-4553-a9c4-82a54bedf97a"} 09:31:56 [09:31:56.532] WARN: prover-node:epoch-proving-job Epoch blocks changed underfoot {"uuid":"c8cbe115-7417-4553-a9c4-82a54bedf97a","epochNumber":1,"oldBlockHashes":["0x1b34b8a5e0be24f9232544a30fbe0cc0eb1f40adb6aa920ab7d1958c80af6a6c","0x2e60a67b4f96054b2c8daaeb586f90d327748661a8b0255e00e462f9dc23e64c","0x19f8a69c79364893687a73b381f73ffdc7ea63f571ffb7b4cfa72dd1999b6e8c"],"newBlockHashes":["0x1b34b8a5e0be24f9232544a30fbe0cc0eb1f40adb6aa920ab7d1958c80af6a6c","0x2e60a67b4f96054b2c8daaeb586f90d327748661a8b0255e00e462f9dc23e64c","0x19f8a69c79364893687a73b381f73ffdc7ea63f571ffb7b4cfa72dd1999b6e8c","0x02908c8182d086d307ef894106f39df83cf7a7d74b03adeeca674369b53129a5"]} 09:31:56 [09:31:56.951] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"c8cbe115-7417-4553-a9c4-82a54bedf97a","epochNumber":1,"details":"Halted execution due to state reorg"} 09:31:56 PASS src/job/epoch-proving-job.test.ts (10.488 s) 09:31:56 epoch-proving-job 09:31:56 ✓ works (1655 ms) 09:31:56 ✓ fails if fails to process txs for a block (1247 ms) 09:31:56 ✓ fails if does not process all txs for a block (1347 ms) 09:31:56 ✓ times out if deadline is hit (1329 ms) 09:31:56 ✓ halts if stopped externally (1265 ms) 09:31:56 ✓ halts if a new block for the epoch is found (1383 ms) 09:31:56 09:31:56 Test Suites: 1 passed, 1 total 09:31:56 Tests: 6 passed, 6 total 09:31:56 Snapshots: 0 total 09:31:56 Time: 10.559 s 09:31:56 Ran all test suites matching /prover-node\/src\/job\/epoch-proving-job.test.ts/i. 09:31:56 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?