Parent Log: http://ci.aztec-labs.com/5341f5baef09d91c Command: a73e94f3b52b3d94:ISOLATE=1:NAME=prover-node/src/job/epoch-proving-job.test.ts yarn-project/scripts/run_test.sh prover-node/src/job/epoch-proving-job.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/aa12cea18cfe4e45e23b46cf1a086d8331837eb5 Env: REF_NAME=db/simple-bc-hash CURRENT_VERSION=0.87.6 CI_FULL=0 Date: Mon Jun 16 17:15:21 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=db_simple-bc-hash_amd64 Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_4ec0c4254cabba91_next 17:15:21 +++ id -u 17:15:21 +++ id -g 17:15:21 ++ docker run -d --name prover-node_src_job_epoch-proving-job.test.ts --net=none --cpuset-cpus=0-127 --cpus=2 --memory=8g --user 1000:1000 -v/home/aztec-dev:/home/aztec-dev --mount type=tmpfs,target=/tmp,tmpfs-size=1g --workdir /home/aztec-dev/aztec-packages -e HOME -e VERBOSE -e GIT_CONFIG_GLOBAL=/home/aztec-dev/aztec-packages/build-images/src/home/.gitconfig -e FORCE_COLOR=true -e CPUS -e MEM aztecprotocol/build:3.0 /bin/bash -c 'timeout -v 600s bash -c '\''yarn-project/scripts/run_test.sh prover-node/src/job/epoch-proving-job.test.ts'\''' 17:15:21 + cid=1d33bd44a3149911f088fe3b2f9866801987b261f981f44f08d83795779ce37b 17:15:21 + set +x 17:15:25 [17:15:25.723] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"ea5f22db-e628-4fdc-a844-6968e280af2c"} 17:15:27 [17:15:27.016] INFO: prover-node:epoch-proving-job Finalised proof for epoch 1 {"epochNumber":1,"uuid":"ea5f22db-e628-4fdc-a844-6968e280af2c","duration":1287.9599469999998} 17:15:27 [17:15:27.018] INFO: prover-node:epoch-proving-job Submitted proof for epoch 1 (blocks 1 to 3) {"epochNumber":1,"uuid":"ea5f22db-e628-4fdc-a844-6968e280af2c"} 17:15:27 [17:15:27.173] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"04b9edf3-5f9b-4e41-ab75-296a3fc2b28f"} 17:15:28 [17:15:28.318] ERROR: prover-node:epoch-proving-job Error running epoch 1 prover job: Error: Txs failed processing: 0x048a937dcdcfa376aeaa7fb2d26226853c1a2b5b7823742d55555034e8162548 (Error: Failed to process tx), 0x1b84f2d2fec6d6ba3c74e7a6255c90beb44be4dc799f6ef1287ade7c1c1e1b53 (Error: Failed to process tx) 17:15:28 at EpochProvingJob.processTxs (/home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:320:13) 17:15:28 at runNextTicks (node:internal/process/task_queues:65:5) 17:15:28 at listOnTimeout (node:internal/timers:549:9) 17:15:28 at processTimers (node:internal/timers:523:7) 17:15:28 at /home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:151:27 {"uuid":"04b9edf3-5f9b-4e41-ab75-296a3fc2b28f","epochNumber":1} 17:15:28 [17:15:28.435] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"69d01b60-474b-4147-b100-21a36dabfe75"} 17:15:29 [17:15:29.528] ERROR: prover-node:epoch-proving-job Error running epoch 1 prover job: Error: Failed to process all txs: processed 0 out of 2 17:15:29 at EpochProvingJob.processTxs (/home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:328:13) 17:15:29 at runNextTicks (node:internal/process/task_queues:65:5) 17:15:29 at listOnTimeout (node:internal/timers:549:9) 17:15:29 at processTimers (node:internal/timers:523:7) 17:15:29 at /home/aztec-dev/aztec-packages/yarn-project/prover-node/src/job/epoch-proving-job.ts:151:27 {"uuid":"69d01b60-474b-4147-b100-21a36dabfe75","epochNumber":1} 17:15:29 [17:15:29.634] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"f58b9e4a-f6a0-4261-bdb9-128a4ba94e33"} 17:15:30 [17:15:30.525] WARN: prover-node:epoch-proving-job Stopping job due to deadline hit {"uuid":"f58b9e4a-f6a0-4261-bdb9-128a4ba94e33","epochNumber":1} 17:15:30 [17:15:30.725] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"f58b9e4a-f6a0-4261-bdb9-128a4ba94e33","epochNumber":1,"details":"Halted execution due to state timed-out"} 17:15:30 [17:15:30.852] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"d753db65-53e0-4b9a-a220-ca5102804702"} 17:15:32 [17:15:32.012] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"d753db65-53e0-4b9a-a220-ca5102804702","epochNumber":1,"details":"Halted execution due to state stopped"} 17:15:32 [17:15:32.308] INFO: prover-node:epoch-proving-job Starting epoch 1 proving job with blocks 1 to 3 {"fromBlock":1,"toBlock":3,"epochSizeBlocks":3,"epochNumber":1,"uuid":"53437a72-4738-42b1-9bc2-4aae75033c77"} 17:15:32 [17:15:32.806] WARN: prover-node:epoch-proving-job Epoch blocks changed underfoot {"uuid":"53437a72-4738-42b1-9bc2-4aae75033c77","epochNumber":1,"oldBlockHashes":["0x1b34b8a5e0be24f9232544a30fbe0cc0eb1f40adb6aa920ab7d1958c80af6a6c","0x2e60a67b4f96054b2c8daaeb586f90d327748661a8b0255e00e462f9dc23e64c","0x19f8a69c79364893687a73b381f73ffdc7ea63f571ffb7b4cfa72dd1999b6e8c"],"newBlockHashes":["0x1b34b8a5e0be24f9232544a30fbe0cc0eb1f40adb6aa920ab7d1958c80af6a6c","0x2e60a67b4f96054b2c8daaeb586f90d327748661a8b0255e00e462f9dc23e64c","0x19f8a69c79364893687a73b381f73ffdc7ea63f571ffb7b4cfa72dd1999b6e8c","0x02908c8182d086d307ef894106f39df83cf7a7d74b03adeeca674369b53129a5"]} 17:15:33 [17:15:33.221] WARN: prover-node:epoch-proving-job Halted execution of epoch 1 prover job {"uuid":"53437a72-4738-42b1-9bc2-4aae75033c77","epochNumber":1,"details":"Halted execution due to state reorg"} 17:15:33 PASS src/job/epoch-proving-job.test.ts (10.329 s) 17:15:33 epoch-proving-job 17:15:33 ✓ works (1850 ms) 17:15:33 ✓ fails if fails to process txs for a block (1294 ms) 17:15:33 ✓ fails if does not process all txs for a block (1210 ms) 17:15:33 ✓ times out if deadline is hit (1197 ms) 17:15:33 ✓ halts if stopped externally (1288 ms) 17:15:33 ✓ halts if a new block for the epoch is found (1210 ms) 17:15:33 17:15:33 Test Suites: 1 passed, 1 total 17:15:33 Tests: 6 passed, 6 total 17:15:33 Snapshots: 0 total 17:15:33 Time: 10.447 s 17:15:33 Ran all test suites matching prover-node/src/job/epoch-proving-job.test.ts. 17:15:33 Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?