Parent Log: http://ci.aztec-labs.com/15686011643 Command: bootstrap_ec2 './bootstrap.sh ci-full' Date: Mon Jun 16 16:38:38 UTC 2025 Status: 0 Took: 1728s --- request build instance --- Requesting m6a.32xlarge spot instance (name: pr-15074_amd64_x2-full) (type: m6a.32xlarge) (ami: ami-069f88e34f8fb5021) (bid: 5.5424)... Waiting for instance id for spot request: sir-i36z74jh... Timeout waiting for spot request. Requesting m6a.32xlarge on-demand instance (name: pr-15074_amd64_x2-full) (type: m6a.32xlarge) (ami: ami-069f88e34f8fb5021) (bid: 5.5424)... Waiting for SSH at 18.191.48.108... No tty, running in background... Loading CRS into tmpfs... Done in 133 seconds. Starting devbox... From https://github.com/aztecprotocol/aztec-packages * branch c06f30929f18b85bcea93c0926a8d640a3819f00 -> FETCH_HEAD HEAD is now at c06f309 fix: reqresp tests (#15074) Switched to a new branch 'gh-readonly-queue/master/pr-15074-2707ec99de02004f3e30e7af0810e0d741ab9794' + ci3/aws_handle_evict run + ci3/add_timestamps + DUP=1 + ci3/cache_log 'CI run' 1750090435135 CI run log id: http://ci.aztec-labs.com/1750090435135 16:13:55 env: REF_NAME=gh-readonly-queue/master/pr-15074-2707ec99de02004f3e30e7af0810e0d741ab9794 COMMIT_HASH=c06f30929f18b85bcea93c0926a8d640a3819f00 CURRENT_VERSION=0.87.8 TARGET_BRANCH=master 16:13:55 + ./bootstrap.sh ci-full 16:14:03 --- pull submodules --- 16:14:03 Executing: git submodule update --init --recursive (http://ci.aztec-labs.com/426f27a391b95d99) 16:14:11 0 . done (8s) 16:14:31 --- noir build --- 16:14:31 Executing: retry install_deps (http://ci.aztec-labs.com/3f449bfd93719d34) 16:14:42 0 . done (11s) 16:14:42 build_native Executing: build_native (http://ci.aztec-labs.com/fb1c65f9f05d9956) 16:14:42 build_packages Executing: build_packages (http://ci.aztec-labs.com/e734c0412b9083e0) 16:14:43 build_native 0 . done (1s) 16:14:49 build_packages 0 . done (7s) 16:14:49 --- bb cpp build --- 16:14:50 build_native Executing: build_native (http://ci.aztec-labs.com/9533779eec226dfe) 16:14:50 build_nodejs_module Executing: build_nodejs_module (http://ci.aztec-labs.com/56f5574b786ce8a5) 16:14:50 build_wasm Executing: build_wasm (http://ci.aztec-labs.com/d9727f1f952a08b2) 16:14:50 build_wasm_threads Executing: build_wasm_threads (http://ci.aztec-labs.com/edaec905642d1b81) 16:14:50 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (http://ci.aztec-labs.com/0638a7b87bf2713e) 16:14:50 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (http://ci.aztec-labs.com/7db5da8e3352d1f8) 16:14:50 build_asan_fast Executing: build_asan_fast (http://ci.aztec-labs.com/4dcb9a2f164dac25) 16:14:50 build_darwin Executing: build_darwin (http://ci.aztec-labs.com/24af979ccf0bfcc1) 16:14:50 build_gcc_syntax_check_only 0 . done (0s) 16:14:50 build_fuzzing_syntax_check_only 0 . done (0s) 16:14:50 build_wasm 0 . done (0s) 16:14:50 build_darwin 0 . done (0s) 16:14:50 build_wasm_threads 0 . done (0s) 16:14:51 build_native 0 . done (1s) 16:14:51 build_nodejs_module 0 . done (1s) 16:14:51 build_asan_fast 0 . done (1s) 16:14:55 --- bb.js build --- 16:14:56 Cache download and extraction of node-modules-aa659bb2a892ab4f.zst complete in 1s. 16:14:56 Cache download and extraction of bb.js-5630c8ceca74baf6.tar.gz complete in 0s. 16:14:57 --- acir_tests build --- 16:14:57 Cache download and extraction of barretenberg-acir-tests-02f79a599fd31657.tar.gz complete in 0s. 16:14:58 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s. 16:14:58 Executing: cd browser-test-app && yarn build (http://ci.aztec-labs.com/c7f0bad85c178c46) 16:15:03 0 . done (5s) 16:15:03 Executing: cd bbjs-test && yarn build (http://ci.aztec-labs.com/4f0a067bbb2b0564) 16:15:06 0 . done (3s) 16:15:06 --- avm-transpiler build --- 16:15:06 Cache download and extraction of avm-transpiler-10ea3821ff0e6f60.tar.gz complete in 0s. 16:15:06 --- noir-projects build --- 16:15:06 Executing: prep (http://ci.aztec-labs.com/dc982876c55343a8) 16:15:09 0 . done (3s) 16:15:09 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/a4907f9dbf3a47b4) 16:15:09 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/6262ce6e49b50b46) 16:15:09 noir-contracts Executing: ./noir-contracts/bootstrap.sh (http://ci.aztec-labs.com/b2468e67a4d8272c) 16:15:09 aztec-nr Executing: ./aztec-nr/bootstrap.sh (http://ci.aztec-labs.com/767cf161c3b6c754) 16:15:10 noir-contracts 0 ................................................................ 16:15:10 noir-contracts 64 ................................................................ 16:15:10 noir-contracts 128 ................................................................ 16:15:10 noir-contracts 192 ................................................................ 16:15:11 noir-contracts 256 ................................................................ 16:15:11 noir-contracts 320 ................................................................ 16:15:11 noir-contracts 384 ................................................................ 16:15:11 noir-contracts 448 ................................................................ 16:15:11 noir-contracts 512 ................................................................ 16:15:11 noir-contracts 576 ................................................................ 16:15:12 noir-contracts 640 ................................................................ 16:15:12 noir-contracts 704 ................................................................ 16:15:12 noir-contracts 768 ................................................................ 16:15:12 noir-contracts 832 ................................................................ 16:15:12 noir-contracts 896 ................................................................ 16:15:12 noir-contracts 960 ................................................................ 16:15:13 noir-contracts 1024 ................................................................ 16:15:13 noir-contracts 1088 ................................................................ 16:15:13 noir-contracts 1152 ................................................................ 16:15:13 noir-contracts 1216 ................................................................ 16:15:13 noir-contracts 1280 ................................................................ 16:15:13 noir-protocol-circuits 0 ................................................................ 16:15:13 noir-protocol-circuits 64 ................................................................ 16:15:13 noir-contracts 1344 ................................................................ 16:15:13 noir-contracts 1408 . done (4s) 16:15:14 noir-protocol-circuits 128 ................................................................ 16:15:15 aztec-nr 0 . done (6s) 16:15:15 noir-protocol-circuits 192 ................................................................ 16:15:15 mock-protocol-circuits 0 ................................................................ 16:15:16 noir-protocol-circuits 256 ................................................................ 16:15:18 noir-protocol-circuits 320 ................................................................ 16:15:18 mock-protocol-circuits 64 . done (9s) 16:15:18 noir-protocol-circuits 384 ................................................................ 16:15:18 noir-protocol-circuits 448 . done (9s) 16:15:18 --- l1-contracts build --- 16:15:18 Cache download and extraction of l1-contracts-b16c798236bc092e.tar.gz complete in 0s. 16:15:18 --- yarn-project build --- 16:15:18 Executing: ./bootstrap.sh clean-lite (http://ci.aztec-labs.com/8164cc6a7cd4c8fa) 16:15:18 0 . done (0s) 16:15:23 Cache download and extraction of node-modules-45f5b7daac61c7b8.zst complete in 4s. 16:15:23 Executing: compile_all (http://ci.aztec-labs.com/ed3fc656dfe50de5) 16:15:27 0 . done (4s) 16:15:28 playground/bootstrap.sh Executing: playground/bootstrap.sh (http://ci.aztec-labs.com/309de0d49c4f5ce4) 16:15:28 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (http://ci.aztec-labs.com/5b4a042e98ce49a1) 16:15:28 docs/bootstrap.sh Executing: docs/bootstrap.sh (http://ci.aztec-labs.com/511762a8952e6e9f) 16:15:28 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (http://ci.aztec-labs.com/11540113dbf0681c) 16:15:28 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (http://ci.aztec-labs.com/d95b2e8882774406) 16:15:28 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (http://ci.aztec-labs.com/dd9209d7caa92c7d) 16:15:28 aztec-up/bootstrap.sh 0 . done (0s) 16:15:31 boxes/bootstrap.sh 0 . done (3s) 16:15:33 playground/bootstrap.sh 0 . done (5s) 16:15:35 docs/bootstrap.sh 0 . done (7s) 16:15:35 spartan/bootstrap.sh 0 . done (7s) 16:16:06 release-image/bootstrap.sh 0 . done (38s) 16:16:06 --- test all --- 16:16:06 Waiting for TXE's to start... 16:16:09 Gathering tests to run... 16:17:21 Gathered 9399 tests. 16:17:21 Starting parallel run with max 64 jobs... 16:17:21 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (http://ci.aztec-labs.com/ad41d2c01b79593b) 16:17:52 0 ................................................................ 16:19:12 64 ................................................................ 16:20:17 128 ................................................................ 16:21:03 192 ................................................................ 16:21:20 256 ................................................................ 16:21:32 320 ................................................................ 16:21:52 384 ................................................................ 16:22:00 448 ................................................................ 16:22:07 512 ................................................................ 16:22:17 576 ................................................................ 16:22:27 640 ................................................................ 16:22:34 704 ................................................................ 16:22:41 768 ................................................................ 16:22:47 832 ................................................................ 16:22:53 896 ................................................................ 16:22:58 960 ................................................................ 16:23:03 1024 ................................................................ 16:23:10 1088 ................................................................ 16:23:12 1152 ................................................................ 16:23:16 1216 ................................................................ 16:23:18 1280 ................................................................ 16:23:21 1344 ................................................................ 16:23:26 1408 ................................................................ 16:23:48 1472 ................................................................ 16:24:01 1536 ................................................................ 16:24:09 1600 ................................................................ 16:24:14 1664 ................................................................ 16:24:17 1728 ................................................................ 16:24:20 1792 ................................................................ 16:24:23 1856 ................................................................ 16:24:26 1920 ................................................................ 16:24:28 1984 ................................................................ 16:24:30 2048 ................................................................ 16:24:32 2112 ................................................................ 16:24:34 2176 ................................................................ 16:24:35 2240 ................................................................ 16:24:37 2304 ................................................................ 16:24:38 2368 ................................................................ 16:24:46 2432 ................................................................ 16:24:48 2496 ................................................................ 16:24:49 2560 ................................................................ 16:24:50 2624 ................................................................ 16:24:52 2688 ................................................................ 16:24:59 2752 ................................................................ 16:25:09 2816 ................................................................ 16:25:14 2880 ................................................................ 16:25:16 2944 ................................................................ 16:25:18 3008 ................................................................ 16:25:20 3072 ................................................................ 16:25:22 3136 ................................................................ 16:25:24 3200 ................................................................ 16:25:26 3264 ................................................................ 16:25:29 3328 ................................................................ 16:25:32 3392 ................................................................ 16:25:33 3456 ................................................................ 16:25:34 3520 ................................................................ 16:25:36 3584 ................................................................ 16:25:37 3648 ................................................................ 16:25:38 3712 ................................................................ 16:25:39 3776 ................................................................ 16:25:41 3840 ................................................................ 16:25:44 3904 ................................................................ 16:25:47 3968 ................................................................ 16:25:50 4032 ................................................................ 16:25:53 4096 ................................................................ 16:25:56 4160 ................................................................ 16:25:58 4224 ................................................................ 16:26:00 4288 ................................................................ 16:26:03 4352 ................................................................ 16:26:04 4416 ................................................................ 16:26:06 4480 ................................................................ 16:26:08 4544 ................................................................ 16:26:09 4608 ................................................................ 16:26:11 4672 ................................................................ 16:26:13 4736 ................................................................ 16:26:14 4800 ................................................................ 16:26:15 4864 ................................................................ 16:26:16 4928 ................................................................ 16:26:18 4992 ................................................................ 16:26:19 5056 ................................................................ 16:26:23 5120 ................................................................ 16:26:26 5184 ................................................................ 16:26:29 5248 ................................................................ 16:26:31 5312 ................................................................ 16:26:34 5376 ................................................................ 16:26:36 5440 ................................................................ 16:26:39 5504 ................................................................ 16:26:41 5568 ................................................................ 16:26:43 5632 ................................................................ 16:26:45 5696 ................................................................ 16:26:48 5760 ................................................................ 16:26:51 5824 ................................................................ 16:26:53 5888 ................................................................ 16:26:56 5952 ................................................................ 16:26:58 6016 ................................................................ 16:27:01 6080 ................................................................ 16:27:04 6144 ................................................................ 16:27:06 6208 ................................................................ 16:27:08 6272 ................................................................ 16:27:11 6336 ................................................................ 16:27:13 6400 ................................................................ 16:27:15 6464 ................................................................ 16:27:17 6528 ................................................................ 16:27:19 6592 ................................................................ 16:27:21 6656 ................................................................ 16:27:22 6720 ................................................................ 16:27:24 6784 ................................................................ 16:27:25 6848 ................................................................ 16:27:27 6912 ................................................................ 16:27:28 6976 ................................................................ 16:27:29 7040 ................................................................ 16:27:31 7104 ................................................................ 16:27:33 7168 ................................................................ 16:27:35 7232 ................................................................ 16:27:38 7296 ................................................................ 16:27:38 7360 ................................................................ 16:27:39 7424 ................................................................ 16:27:41 7488 ................................................................ 16:27:42 7552 ................................................................ 16:27:43 7616 ................................................................ 16:27:44 7680 ................................................................ 16:27:45 7744 ................................................................ 16:27:46 7808 ................................................................ 16:27:47 7872 ................................................................ 16:27:49 7936 ................................................................ 16:27:50 8000 ................................................................ 16:27:51 8064 ................................................................ 16:27:52 8128 ................................................................ 16:27:54 8192 ................................................................ 16:27:55 8256 ................................................................ 16:27:55 8320 ................................................................ 16:27:56 8384 ................................................................ 16:27:57 8448 ................................................................ 16:27:59 8512 ................................................................ 16:28:00 8576 ................................................................ 16:28:01 8640 ................................................................ 16:28:02 8704 ................................................................ 16:28:03 8768 ................................................................ 16:28:04 8832 ................................................................ 16:28:05 8896 ................................................................ 16:28:06 8960 ................................................................ 16:28:07 9024 ................................................................ 16:28:08 9088 ................................................................ 16:28:09 9152 ................................................................ 16:28:10 9216 ................................................................ 16:28:11 9280 ................................................................ 16:28:12 9344 ................................................................ 16:28:13 9408 ................................................................ 16:28:14 9472 ................................................................ 16:28:15 9536 ................................................................ 16:29:53 9600 . done (752s) 16:29:53 WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them. 16:29:53 314.769: '5eee81c0711c2870:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes' 16:29:53 322.852: 'de30be46e225864a barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness' 16:29:53 329.361: '21081b81c02e2d2a:ISOLATE=1:NAME=e2e_epochs/epochs_proof_fails yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_proof_fails.test.ts' 16:29:53 350.750: 'de30be46e225864a:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing' 16:29:53 352.296: 'de30be46e225864a:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing' 16:29:53 367.996: '0fc4f277b81a4cc2 aztec-up/scripts/run_test.sh amm_flow' 16:29:53 368.858: '21081b81c02e2d2a:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building' 16:29:53 376.220: '21081b81c02e2d2a:ISOLATE=1:NAME=e2e_epochs/epochs_l1_reorgs yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_l1_reorgs.test.ts' 16:29:53 396.857: '21081b81c02e2d2a:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts' 16:29:53 417.460: '21081b81c02e2d2a:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts' 16:29:53 447.436: '21081b81c02e2d2a:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts' 16:29:53 577.811: '21081b81c02e2d2a:ISOLATE=1:NAME=e2e_epochs/epochs_empty_blocks yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_empty_blocks.test.ts' 16:29:53 697.459: '21081b81c02e2d2a:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full' 16:29:53 Completed run of 9399 tests in 752 seconds. 16:29:53 --- bench all --- 16:29:53 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (http://ci.aztec-labs.com/295f705808af0bd4) 16:29:53 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (http://ci.aztec-labs.com/9b48637b4b99d985) 16:29:54 barretenberg/cpp 0 . done (1s) 16:31:37 yarn-project/end-to-end 0 ................................................................ 16:34:06 yarn-project/end-to-end 64 . done (253s) 16:34:07 Starting parallel run using max 64 cpus... 16:34:07 Test run log id: http://ci.aztec-labs.com/bcef9c9df9974d97 16:38:29 Completed run of 90 tests in 262 seconds. 16:38:32 Completed 6.3 KiB/6.3 KiB (113.5 KiB/s) with 1 file(s) remaining upload: ../../../tmp/tmp.3Btp4tO1DP to s3://aztec-ci-artifacts/build-cache/bench-c06f30929f18b85bcea93c0926a8d640a3819f00.tar.gz 16:38:32 Cache upload of bench-c06f30929f18b85bcea93c0926a8d640a3819f00.tar.gz complete in 1s. 16:38:32 + local code=0 16:38:32 + set +x 16:38:32 dmesg log id: http://ci.aztec-labs.com/e022844bcb345074 16:38:32 cpufile log id: http://ci.aztec-labs.com/672deba5fe793ade 16:38:32 memfile log id: http://ci.aztec-labs.com/0d95330cff15711b CI run log id: http://ci.aztec-labs.com/1750090435135 + code=0 + case $code in + log_ci_run PASSED 1750090435135 + exit 0 SSH exited with code: 0 Terminating instance: i-09f0e3606c66ce911...