Parent Log: http://ci.aztec-labs.com/6accf1747685d10b Command: 0742425f32083b75 yarn-project/scripts/run_test.sh archiver/src/archiver/kv_archiver_store/kv_archiver_store.test.ts Commit: https://github.com/AztecProtocol/aztec-packages/commit/04abcb151443d51d2f1b002a2412a083a9384a51 Env: REF_NAME=gh-readonly-queue/next/pr-15077-263498dd65e8700218979d7a94dbfd25b2a8a8a8 CURRENT_VERSION=0.87.6 CI_FULL=1 Date: Tue Jun 17 20:50:33 UTC 2025 System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15077_amd64_x4-full Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s History: http://ci.aztec-labs.com/list/history_afeabe6543d27e83_next 20:50:36 [20:50:36.435] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:37 [20:50:37.869] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:39 [20:50:39.354] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:40 [20:50:40.318] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: InitialBlockNumberNotSequentialError: Cannot insert new block 2 given previous block number in store is undefined 20:50:40 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/block_store.ts:79:15 20:50:40 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:50:40 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:50:40 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) { 20:50:40 newBlockNumber: 2, 20:50:40 previousBlockNumber: undefined 20:50:40 } 20:50:40 [20:50:40.326] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:41 [20:50:41.522] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: BlockNumberNotSequentialError: Cannot insert new block 3 given previous block number in batch is 1 20:50:41 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/block_store.ts:86:17 20:50:41 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:50:41 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:50:41 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) 20:50:41 [20:50:41.525] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:42 [20:50:42.953] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:44 [20:50:44.246] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:45 [20:50:45.708] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: Error: Can only unwind blocks from the tip (requested 5 but current tip is 10) 20:50:45 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/block_store.ts:123:15 20:50:45 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:50:45 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:50:45 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) 20:50:45 [20:50:45.711] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:47 [20:50:47.332] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:49 [20:50:49.026] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:50 [20:50:50.981] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:52 [20:50:52.641] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:54 [20:50:54.335] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:56 [20:50:56.352] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:50:58 [20:50:58.577] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:00 [20:51:00.384] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:02 [20:51:02.465] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:05 [20:51:05.060] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:06 [20:51:06.412] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:08 [20:51:08.362] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:09 [20:51:09.697] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:11 [20:51:11.399] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:12 [20:51:12.750] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:14 [20:51:14.187] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:15 [20:51:15.558] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:17 [20:51:17.086] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:18 [20:51:18.650] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:21 [20:51:21.369] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:23 [20:51:23.943] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:26 [20:51:26.642] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:29 [20:51:29.347] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:32 [20:51:32.144] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:34 [20:51:34.878] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:37 [20:51:37.624] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:40 [20:51:40.153] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:43 [20:51:43.161] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:45 [20:51:45.985] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:48 [20:51:48.858] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:51 [20:51:51.682] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:55 [20:51:55.167] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:56 [20:51:56.543] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:58 [20:51:58.076] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:51:59 [20:51:59.806] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:01 [20:52:01.373] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:02 [20:52:02.922] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:04 [20:52:04.262] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:05 [20:52:05.848] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:07 [20:52:07.600] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:09 [20:52:09.626] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:11 [20:52:11.712] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:13 [20:52:13.400] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: MessageStoreError: Message 0x2f95b20e11768e12aadcb32806faef99ca84a0737ca4b4b270b8e9118f2f694d for L2 block 1 has wrong index 10 (expected 0) 20:52:13 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/message_store.ts:144:17 20:52:13 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:52:13 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:52:13 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) { 20:52:13 inboxMessage: { 20:52:13 index: 10n, 20:52:13 leaf: Fr<0x2f95b20e11768e12aadcb32806faef99ca84a0737ca4b4b270b8e9118f2f694d>, 20:52:13 l2BlockNumber: 1, 20:52:13 l1BlockNumber: 76n, 20:52:13 l1BlockHash: Buffer32<0x4d59f2c8725750bf8b6230f24a98cd5af1929d2b62d792a57f726b8c50822785>, 20:52:13 rollingHash: Buffer16<0xde2d3a3a58df66b52862e19f1b6e71ae> 20:52:13 } 20:52:13 } 20:52:13 [20:52:13.404] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:15 [20:52:15.199] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: MessageStoreError: Invalid index 224 for incoming L1 to L2 message 0x1a93aeb3658d125a71265c96be4c9397800769aff779e77b3f5ffc4c9180d4d9 at block 12 (expected value in range [176, 192)) 20:52:15 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/message_store.ts:119:17 20:52:15 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:52:15 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:52:15 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) { 20:52:15 inboxMessage: { 20:52:15 index: 224n, 20:52:15 leaf: Fr<0x1a93aeb3658d125a71265c96be4c9397800769aff779e77b3f5ffc4c9180d4d9>, 20:52:15 l2BlockNumber: 12, 20:52:15 l1BlockNumber: 81n, 20:52:15 l1BlockHash: Buffer32<0x34b7da33b30c374bf4ddf63208225f4c29fed6e26d694cc84405c11820c188b0>, 20:52:15 rollingHash: Buffer16<0x392d4faab7568468de121538cad1b61c> 20:52:15 } 20:52:15 } 20:52:15 [20:52:15.203] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:17 [20:52:17.036] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: MessageStoreError: Invalid rolling hash for incoming L1 to L2 message 0x19b8e07c97ecfa73abcdf461fd61f785f714218992fd82aabb951f973cf36153 with index 16 (expected 0x984a89f250adcc45fa0d4d81c4c1ccbb from previous hash 0xb994b8829bd83546c650bfe083ca2f82 but got 0xcd4e64f05c368b24d6370900edf047c0) 20:52:17 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/message_store.ts:108:17 20:52:17 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:52:17 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:52:17 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) { 20:52:17 inboxMessage: { 20:52:17 index: 16n, 20:52:17 leaf: Fr<0x19b8e07c97ecfa73abcdf461fd61f785f714218992fd82aabb951f973cf36153>, 20:52:17 l2BlockNumber: 2, 20:52:17 l1BlockNumber: 76n, 20:52:17 l1BlockHash: Buffer32<0x72885e02cba676b0d1e02ed680136e765b4ea16df8b17556823ecbdf64cb9d70>, 20:52:17 rollingHash: Buffer16<0xcd4e64f05c368b24d6370900edf047c0> 20:52:17 } 20:52:17 } 20:52:17 [20:52:17.038] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16 20:52:18 [20:52:18.703] ERROR: kv-store:lmdb-v2:archiver_test Failed to commit transaction: MessageStoreError: Invalid rolling hash for incoming L1 to L2 message 0x1d2922f07e96469214f1d2a8107196c740df306e5d5c8aa2b637cf701192c914 with index 32 (expected 0xfafed0d829fa14b4cdce9dfa95ef5add from previous hash 0x05b42177237a46c11b79505947efec6d but got 0x7bc1b8b598334997ba5a18ddef4d626a) 20:52:18 at /home/aztec-dev/aztec-packages/yarn-project/archiver/src/archiver/kv_archiver_store/message_store.ts:108:17 20:52:18 at /home/aztec-dev/aztec-packages/yarn-project/kv-store/dest/lmdb-v2/store.js:111:29 20:52:18 at /home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/serial_queue.js:58:33 20:52:18 at FifoMemoryQueue.process (/home/aztec-dev/aztec-packages/yarn-project/foundation/dest/queue/base_memory_queue.js:110:17) { 20:52:18 inboxMessage: { 20:52:18 index: 32n, 20:52:18 leaf: Fr<0x1d2922f07e96469214f1d2a8107196c740df306e5d5c8aa2b637cf701192c914>, 20:52:18 l2BlockNumber: 3, 20:52:18 l1BlockNumber: 35n, 20:52:18 l1BlockHash: Buffer32<0x2d397ba8c57fcfa6a9eedcda0a681b5eb2302156f295a268efda05aae86f8e99>, 20:52:18 rollingHash: Buffer16<0x7bc1b8b598334997ba5a18ddef4d626a> 20:52:18 } 20:52:18 } 20:52:18 [20:52:18.707] INFO: kv-store:lmdb-v2:archiver_test Starting data store with maxReaders 16