Parent Log: http://ci.aztec-labs.com/15637354560 Command: bootstrap_ec2 './bootstrap.sh ci-full' Date: Fri Jun 13 15:13:03 UTC 2025 Status: 0 Took: 1567s --- request build instance --- Requesting m6a.32xlarge spot instance (name: pr-15002_amd64_x2-full) (type: m6a.32xlarge) (ami: ami-069f88e34f8fb5021) (bid: 5.5424)... Waiting for instance id for spot request: sir-25kz4ytk... Timeout waiting for spot request. Requesting m6a.32xlarge on-demand instance (name: pr-15002_amd64_x2-full) (type: m6a.32xlarge) (ami: ami-069f88e34f8fb5021) (bid: 5.5424)... Waiting for SSH at 18.191.149.122... No tty, running in background... Loading CRS into tmpfs... Done in 17 seconds. Starting devbox... From https://github.com/aztecprotocol/aztec-packages * branch 267ac35e8ccb487bdc79bbe12d428f4c8b4d4d9f -> FETCH_HEAD HEAD is now at 267ac35 backport: use shared cache for pippenger_runtime_state (#14753) (#15002) Switched to a new branch 'gh-readonly-queue/master/pr-15002-1338cef09c870378254c56748f972ae827c4d482' + ci3/aws_handle_evict run + ci3/add_timestamps + DUP=1 + ci3/cache_log 'CI run' 1749826112918 CI run log id: http://ci.aztec-labs.com/1749826112918 14:48:33 env: REF_NAME=gh-readonly-queue/master/pr-15002-1338cef09c870378254c56748f972ae827c4d482 COMMIT_HASH=267ac35e8ccb487bdc79bbe12d428f4c8b4d4d9f CURRENT_VERSION=0.87.8 TARGET_BRANCH=master 14:48:33 + ./bootstrap.sh ci-full 14:48:40 --- pull submodules --- 14:48:40 Executing: git submodule update --init --recursive (http://ci.aztec-labs.com/6be3377eaf0e3d96) 14:48:48 0 . done (8s) 14:48:49 --- noir build --- 14:48:49 Executing: retry install_deps (http://ci.aztec-labs.com/29aab55b519f64fc) 14:48:55 0 . done (6s) 14:48:55 build_native Executing: build_native (http://ci.aztec-labs.com/a5b995084862dfee) 14:48:55 build_packages Executing: build_packages (http://ci.aztec-labs.com/e6872c39893c6ff0) 14:48:56 build_native 0 . done (1s) 14:49:03 build_packages 0 . done (8s) 14:49:04 --- bb cpp build --- 14:49:04 build_native Executing: build_native (http://ci.aztec-labs.com/812c2e2423a427a0) 14:49:04 build_wasm_threads Executing: build_wasm_threads (http://ci.aztec-labs.com/5504282ccb6a63ef) 14:49:04 build_nodejs_module Executing: build_nodejs_module (http://ci.aztec-labs.com/5dd75df61bc7f6e1) 14:49:04 build_wasm Executing: build_wasm (http://ci.aztec-labs.com/5821bb3fa44946bb) 14:49:04 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (http://ci.aztec-labs.com/216aa1f0b5f1ba1c) 14:49:04 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (http://ci.aztec-labs.com/4fbf4cd7d90711d9) 14:49:04 build_asan_fast Executing: build_asan_fast (http://ci.aztec-labs.com/6575590d493e7e73) 14:49:04 build_darwin Executing: build_darwin (http://ci.aztec-labs.com/1cdd63ddc0429cee) 14:49:04 build_fuzzing_syntax_check_only 0 . done (0s) 14:49:04 build_gcc_syntax_check_only 0 . done (0s) 14:49:04 build_wasm 0 . done (0s) 14:49:04 build_darwin 0 . done (0s) 14:49:04 build_wasm_threads 0 . done (0s) 14:49:05 build_nodejs_module 0 . done (1s) 14:49:06 build_native 0 . done (2s) 14:49:08 build_asan_fast 0 . done (4s) 14:49:12 --- bb.js build --- 14:49:13 Cache download and extraction of node-modules-aa659bb2a892ab4f.zst complete in 0s. 14:49:14 Cache download and extraction of bb.js-a4bf2509278deab9.tar.gz complete in 1s. 14:49:15 --- acir_tests build --- 14:49:15 Cache download and extraction of barretenberg-acir-tests-02f79a599fd31657.tar.gz complete in 0s. 14:49:16 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s. 14:49:16 Executing: cd browser-test-app && yarn build (http://ci.aztec-labs.com/d8451f85d06dc0e3) 14:49:21 0 . done (5s) 14:49:21 Executing: cd bbjs-test && yarn build (http://ci.aztec-labs.com/e659c3f86a2acc11) 14:49:24 0 . done (3s) 14:49:24 --- avm-transpiler build --- 14:49:24 Cache download and extraction of avm-transpiler-10ea3821ff0e6f60.tar.gz complete in 0s. 14:49:24 --- noir-projects build --- 14:49:24 Executing: prep (http://ci.aztec-labs.com/13e13dfc89ea1145) 14:49:27 0 . done (3s) 14:49:27 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/a7b19fbeaf82b1f6) 14:49:27 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/4901fbeabeb81eb5) 14:49:27 noir-contracts Executing: ./noir-contracts/bootstrap.sh (http://ci.aztec-labs.com/b9a2cf361fcb2202) 14:49:27 aztec-nr Executing: ./aztec-nr/bootstrap.sh (http://ci.aztec-labs.com/a98e5b9b56d1b9b4) 14:49:29 noir-contracts 0 ................................................................ 14:49:29 noir-contracts 64 ................................................................ 14:49:29 noir-contracts 128 ................................................................ 14:49:29 noir-contracts 192 ................................................................ 14:49:29 noir-contracts 256 ................................................................ 14:49:30 noir-contracts 320 ................................................................ 14:49:30 noir-contracts 384 ................................................................ 14:49:30 noir-contracts 448 ................................................................ 14:49:30 noir-contracts 512 ................................................................ 14:49:30 noir-contracts 576 ................................................................ 14:49:31 noir-contracts 640 ................................................................ 14:49:31 noir-contracts 704 ................................................................ 14:49:31 noir-contracts 768 ................................................................ 14:49:31 noir-contracts 832 ................................................................ 14:49:32 noir-contracts 896 ................................................................ 14:49:32 noir-contracts 960 ................................................................ 14:49:32 noir-contracts 1024 ................................................................ 14:49:32 noir-contracts 1088 ................................................................ 14:49:32 noir-contracts 1152 ................................................................ 14:49:32 noir-protocol-circuits 0 ................................................................ 14:49:32 noir-protocol-circuits 64 ................................................................ 14:49:33 noir-contracts 1216 ................................................................ 14:49:33 noir-contracts 1280 ................................................................ 14:49:33 noir-contracts 1344 ................................................................ 14:49:33 noir-contracts 1408 . done (6s) 14:49:33 noir-protocol-circuits 128 ................................................................ 14:49:34 aztec-nr 0 . done (7s) 14:49:34 noir-protocol-circuits 192 ................................................................ 14:49:34 mock-protocol-circuits 0 ................................................................ 14:49:35 noir-protocol-circuits 256 ................................................................ 14:49:36 noir-protocol-circuits 320 ................................................................ 14:49:37 mock-protocol-circuits 64 . done (10s) 14:49:41 noir-protocol-circuits 384 ................................................................ 14:49:41 noir-protocol-circuits 448 . done (14s) 14:49:41 --- l1-contracts build --- 14:49:41 Cache download and extraction of l1-contracts-b16c798236bc092e.tar.gz complete in 0s. 14:49:41 --- yarn-project build --- 14:49:41 Executing: ./bootstrap.sh clean-lite (http://ci.aztec-labs.com/5125a3cf02fc78be) 14:49:41 0 . done (0s) 14:49:47 Cache download and extraction of node-modules-45f5b7daac61c7b8.zst complete in 5s. 14:49:47 Executing: compile_all (http://ci.aztec-labs.com/18753d433459ec3c) 14:49:50 0 . done (3s) 14:49:51 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (http://ci.aztec-labs.com/ab0c5a81b47dcb26) 14:49:51 playground/bootstrap.sh Executing: playground/bootstrap.sh (http://ci.aztec-labs.com/1eb7a551e03b90db) 14:49:51 docs/bootstrap.sh Executing: docs/bootstrap.sh (http://ci.aztec-labs.com/263d35adee3920b6) 14:49:51 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (http://ci.aztec-labs.com/9e0673aa4f5a533b) 14:49:51 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (http://ci.aztec-labs.com/604ba9fa4a5e1ec4) 14:49:51 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (http://ci.aztec-labs.com/561b318a03271911) 14:49:52 aztec-up/bootstrap.sh 0 . done (1s) 14:49:54 spartan/bootstrap.sh 0 . done (3s) 14:49:55 boxes/bootstrap.sh 0 . done (4s) 14:49:56 playground/bootstrap.sh 0 . done (5s) 14:49:58 docs/bootstrap.sh 0 . done (7s) 14:50:29 release-image/bootstrap.sh 0 . done (38s) 14:50:29 --- test all --- 14:50:30 Waiting for TXE's to start... 14:50:33 Gathering tests to run... 14:51:42 Gathered 9399 tests. 14:51:42 Starting parallel run with max 64 jobs... 14:51:42 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (http://ci.aztec-labs.com/0037d898807e86c0) 14:52:14 0 ................................................................ 14:53:34 64 ................................................................ 14:54:37 128 ................................................................ 14:55:23 192 ................................................................ 14:55:39 256 ................................................................ 14:55:52 320 ................................................................ 14:56:12 384 ................................................................ 14:56:20 448 ................................................................ 14:56:27 512 ................................................................ 14:56:37 576 ................................................................ 14:56:47 640 ................................................................ 14:56:54 704 ................................................................ 14:57:02 768 ................................................................ 14:57:08 832 ................................................................ 14:57:14 896 ................................................................ 14:57:19 960 ................................................................ 14:57:25 1024 ................................................................ 14:57:30 1088 ................................................................ 14:57:32 1152 ................................................................ 14:57:36 1216 ................................................................ 14:57:39 1280 ................................................................ 14:57:42 1344 ................................................................ 14:57:47 1408 ................................................................ 14:58:07 1472 ................................................................ 14:58:21 1536 ................................................................ 14:58:29 1600 ................................................................ 14:58:34 1664 ................................................................ 14:58:36 1728 ................................................................ 14:58:39 1792 ................................................................ 14:58:41 1856 ................................................................ 14:58:43 1920 ................................................................ 14:58:45 1984 ................................................................ 14:58:47 2048 ................................................................ 14:58:49 2112 ................................................................ 14:58:51 2176 ................................................................ 14:58:52 2240 ................................................................ 14:58:54 2304 ................................................................ 14:58:55 2368 ................................................................ 14:59:03 2432 ................................................................ 14:59:04 2496 ................................................................ 14:59:06 2560 ................................................................ 14:59:07 2624 ................................................................ 14:59:09 2688 ................................................................ 14:59:17 2752 ................................................................ 14:59:27 2816 ................................................................ 14:59:32 2880 ................................................................ 14:59:34 2944 ................................................................ 14:59:36 3008 ................................................................ 14:59:38 3072 ................................................................ 14:59:40 3136 ................................................................ 14:59:42 3200 ................................................................ 14:59:45 3264 ................................................................ 14:59:47 3328 ................................................................ 14:59:50 3392 ................................................................ 14:59:51 3456 ................................................................ 14:59:53 3520 ................................................................ 14:59:54 3584 ................................................................ 14:59:55 3648 ................................................................ 14:59:57 3712 ................................................................ 14:59:58 3776 ................................................................ 15:00:00 3840 ................................................................ 15:00:03 3904 ................................................................ 15:00:06 3968 ................................................................ 15:00:10 4032 ................................................................ 15:00:13 4096 ................................................................ 15:00:15 4160 ................................................................ 15:00:17 4224 ................................................................ 15:00:19 4288 ................................................................ 15:00:22 4352 ................................................................ 15:00:24 4416 ................................................................ 15:00:26 4480 ................................................................ 15:00:27 4544 ................................................................ 15:00:29 4608 ................................................................ 15:00:31 4672 ................................................................ 15:00:33 4736 ................................................................ 15:00:34 4800 ................................................................ 15:00:35 4864 ................................................................ 15:00:37 4928 ................................................................ 15:00:38 4992 ................................................................ 15:00:40 5056 ................................................................ 15:00:44 5120 ................................................................ 15:00:47 5184 ................................................................ 15:00:49 5248 ................................................................ 15:00:52 5312 ................................................................ 15:00:55 5376 ................................................................ 15:00:57 5440 ................................................................ 15:01:00 5504 ................................................................ 15:01:02 5568 ................................................................ 15:01:04 5632 ................................................................ 15:01:06 5696 ................................................................ 15:01:09 5760 ................................................................ 15:01:12 5824 ................................................................ 15:01:15 5888 ................................................................ 15:01:17 5952 ................................................................ 15:01:20 6016 ................................................................ 15:01:22 6080 ................................................................ 15:01:25 6144 ................................................................ 15:01:27 6208 ................................................................ 15:01:30 6272 ................................................................ 15:01:32 6336 ................................................................ 15:01:34 6400 ................................................................ 15:01:36 6464 ................................................................ 15:01:38 6528 ................................................................ 15:01:40 6592 ................................................................ 15:01:43 6656 ................................................................ 15:01:44 6720 ................................................................ 15:01:45 6784 ................................................................ 15:01:47 6848 ................................................................ 15:01:48 6912 ................................................................ 15:01:50 6976 ................................................................ 15:01:51 7040 ................................................................ 15:01:53 7104 ................................................................ 15:01:55 7168 ................................................................ 15:01:58 7232 ................................................................ 15:02:00 7296 ................................................................ 15:02:01 7360 ................................................................ 15:02:01 7424 ................................................................ 15:02:02 7488 ................................................................ 15:02:04 7552 ................................................................ 15:02:05 7616 ................................................................ 15:02:06 7680 ................................................................ 15:02:07 7744 ................................................................ 15:02:08 7808 ................................................................ 15:02:09 7872 ................................................................ 15:02:11 7936 ................................................................ 15:02:13 8000 ................................................................ 15:02:14 8064 ................................................................ 15:02:15 8128 ................................................................ 15:02:16 8192 ................................................................ 15:02:17 8256 ................................................................ 15:02:18 8320 ................................................................ 15:02:19 8384 ................................................................ 15:02:21 8448 ................................................................ 15:02:22 8512 ................................................................ 15:02:23 8576 ................................................................ 15:02:24 8640 ................................................................ 15:02:25 8704 ................................................................ 15:02:26 8768 ................................................................ 15:02:27 8832 ................................................................ 15:02:28 8896 ................................................................ 15:02:30 8960 ................................................................ 15:02:31 9024 ................................................................ 15:02:32 9088 ................................................................ 15:02:33 9152 ................................................................ 15:02:34 9216 ................................................................ 15:02:35 9280 ................................................................ 15:02:36 9344 ................................................................ 15:02:37 9408 ................................................................ 15:02:38 9472 ................................................................ 15:02:39 9536 ................................................................ 15:04:04 9600 . done (741s) 15:04:04 WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them. 15:04:04 316.082: '025a2f2889357ef6:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes' 15:04:04 316.323: 'de30be46e225864a barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness' 15:04:04 321.401: '4bfd2d3585b9db66:ISOLATE=1:NAME=e2e_epochs/epochs_proof_fails yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_proof_fails.test.ts' 15:04:04 351.098: 'de30be46e225864a:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing' 15:04:04 354.815: 'de30be46e225864a:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing' 15:04:04 362.577: '4bfd2d3585b9db66:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building' 15:04:04 370.023: '97cfc341852a92fa aztec-up/scripts/run_test.sh amm_flow' 15:04:04 372.821: '4bfd2d3585b9db66:ISOLATE=1:NAME=e2e_epochs/epochs_l1_reorgs yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_l1_reorgs.test.ts' 15:04:04 394.150: '4bfd2d3585b9db66:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts' 15:04:04 435.141: '4bfd2d3585b9db66:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts' 15:04:04 454.620: '4bfd2d3585b9db66:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts' 15:04:04 561.819: '4bfd2d3585b9db66:ISOLATE=1:NAME=e2e_epochs/epochs_empty_blocks yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_empty_blocks.test.ts' 15:04:04 696.473: '4bfd2d3585b9db66:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full' 15:04:04 Completed run of 9399 tests in 742 seconds. 15:04:04 --- bench all --- 15:04:04 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (http://ci.aztec-labs.com/029889802dbc517e) 15:04:04 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (http://ci.aztec-labs.com/0f4d203e351e3e6f) 15:06:09 yarn-project/end-to-end 0 ................................................................ 15:06:38 barretenberg/cpp 0 . done (154s) 15:08:30 yarn-project/end-to-end 64 . done (266s) 15:08:31 Starting parallel run using max 64 cpus... 15:08:32 Test run log id: http://ci.aztec-labs.com/620e030b0bc2d3a0 15:12:54 Completed run of 90 tests in 263 seconds. 15:12:57 Completed 6.3 KiB/6.3 KiB (82.2 KiB/s) with 1 file(s) remaining upload: ../../../tmp/tmp.CkPQBMP4CR to s3://aztec-ci-artifacts/build-cache/bench-267ac35e8ccb487bdc79bbe12d428f4c8b4d4d9f.tar.gz 15:12:57 Cache upload of bench-267ac35e8ccb487bdc79bbe12d428f4c8b4d4d9f.tar.gz complete in 2s. 15:12:57 + local code=0 15:12:57 + set +x 15:12:57 dmesg log id: http://ci.aztec-labs.com/67c373ead083fe60 15:12:57 cpufile log id: http://ci.aztec-labs.com/68d6d53aee8484ee 15:12:57 memfile log id: http://ci.aztec-labs.com/0975f329aea6c238 CI run log id: http://ci.aztec-labs.com/1749826112918 + code=0 + case $code in + log_ci_run PASSED 1749826112918 + exit 0 SSH exited with code: 0 Terminating instance: i-0ac3bd66f52a3452f...