Parent Log: http://ci.aztec-labs.com/15853803733 Command: bootstrap_ec2 './bootstrap.sh ci-full' Date: Tue Jun 24 15:19:49 UTC 2025 Status: 0 Took: 1814s --- request build instance --- Requesting m6a.32xlarge spot instance (name: pr-15247_amd64_x3-full) (type: m6a.32xlarge) (ami: ami-051d99fd9abf53087) (bid: 5.5424)... Waiting for instance id for spot request: sir-ipxq4jfk... Timeout waiting for spot request. Requesting m6a.32xlarge on-demand instance (name: pr-15247_amd64_x3-full) (type: m6a.32xlarge) (ami: ami-051d99fd9abf53087) (bid: 5.5424)... Waiting for SSH at 18.189.22.201... Stdout is not a tty, running in background... Loading CRS into tmpfs... Done in 9 seconds. Stdout is not a tty, starting build container in background... Starting devbox... From https://github.com/aztecprotocol/aztec-packages * branch 9ba478709ce5c48099f79f9531da8292811cb18a -> FETCH_HEAD HEAD is now at 9ba4787 chore(backport-to-staging): fix: ensure block gas limit for simulation is large enough (#15242) (#15247) Switched to a new branch 'gh-readonly-queue/staging/pr-15247-3863bc02d671057d0cffd4ced17ca317788ff190' + run + ci3/add_timestamps + DUP=1 + ci3/cache_log 'CI run' 1750776665092 CI run log id: http://ci.aztec-labs.com/1750776665092 14:51:05 + echo 'env: REF_NAME=gh-readonly-queue/staging/pr-15247-3863bc02d671057d0cffd4ced17ca317788ff190 COMMIT_HASH=9ba478709ce5c48099f79f9531da8292811cb18a CURRENT_VERSION=1.0.0 TARGET_BRANCH=staging' 14:51:05 env: REF_NAME=gh-readonly-queue/staging/pr-15247-3863bc02d671057d0cffd4ced17ca317788ff190 COMMIT_HASH=9ba478709ce5c48099f79f9531da8292811cb18a CURRENT_VERSION=1.0.0 TARGET_BRANCH=staging 14:51:05 + semver check gh-readonly-queue/staging/pr-15247-3863bc02d671057d0cffd4ced17ca317788ff190 14:51:05 + cpu_pid=417 14:51:05 + mpstat 2 14:51:05 + mem_pid=419 14:51:05 + set +e 14:51:05 + set -x 14:51:05 + ./bootstrap.sh ci-full 14:51:05 + vmstat -w -S M 2 14:51:05 ++ cache_log cpufile 14:51:05 ++ cache_log memfile 14:51:05 cpufile log id: http://ci.aztec-labs.com/d54df6ee9037aa5d 14:51:05 memfile log id: http://ci.aztec-labs.com/8485ac11e408d218 14:51:13 --- pull submodules --- 14:51:13 Executing: git submodule update --init --recursive (http://ci.aztec-labs.com/18ecf6fff4d7173b) 14:51:22 0 . done (9s) 14:51:23 --- noir build --- 14:51:23 Executing: retry install_deps (http://ci.aztec-labs.com/e90f56dca61e936f) 14:51:27 0 . done (4s) 14:51:28 build_native Executing: build_native (http://ci.aztec-labs.com/893a751459064844) 14:51:28 build_packages Executing: build_packages (http://ci.aztec-labs.com/8c10b0d2ef90cd5e) 14:51:29 build_native 0 . done (1s) 14:51:35 build_packages 0 . done (7s) 14:51:36 --- bb cpp build --- 14:51:37 build_wasm Executing: build_wasm (http://ci.aztec-labs.com/a44035c123ebc259) 14:51:37 build_nodejs_module Executing: build_nodejs_module (http://ci.aztec-labs.com/c174fd2381e21404) 14:51:37 build_wasm_threads Executing: build_wasm_threads (http://ci.aztec-labs.com/70cffb18a309a8c9) 14:51:37 build_native Executing: build_native (http://ci.aztec-labs.com/243b9b5c901f79f5) 14:51:37 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (http://ci.aztec-labs.com/d48d5489da15cff7) 14:51:37 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (http://ci.aztec-labs.com/7c76171a39157dcb) 14:51:37 build_asan_fast Executing: build_asan_fast (http://ci.aztec-labs.com/547baa3e14fa1a37) 14:51:37 build_darwin Executing: build_darwin (http://ci.aztec-labs.com/e699b67e3902ad29) 14:51:37 build_gcc_syntax_check_only 0 . done (0s) 14:51:37 build_fuzzing_syntax_check_only 0 . done (0s) 14:51:37 build_darwin 0 . done (0s) 14:51:37 build_wasm 0 . done (0s) 14:51:37 build_wasm_threads 0 . done (0s) 14:51:37 build_nodejs_module 0 . done (0s) 14:51:39 build_asan_fast 0 . done (2s) 14:51:39 build_native 0 . done (2s) 14:51:44 --- bb.js build --- 14:51:46 Cache download and extraction of node-modules-e85320bda7d59d54.zst complete in 1s. 14:51:46 Cache download and extraction of bb.js-93955768547ae31a.tar.gz complete in 0s. 14:51:47 --- acir_tests build --- 14:51:48 Cache download and extraction of barretenberg-acir-tests-000ad54a8a7b4100.tar.gz complete in 1s. 14:51:49 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s. 14:51:49 browser-test-app Executing: yarn build (http://ci.aztec-labs.com/e4fbe43ced05a5b3) 14:51:49 bbjs-test Executing: yarn build (http://ci.aztec-labs.com/a534ca029fbc1918) 14:51:52 bbjs-test 0 . done (3s) 14:51:54 browser-test-app 0 . done (5s) 14:51:55 --- avm-transpiler build --- 14:51:55 Cache download and extraction of avm-transpiler-6867cd9272ea7ebd.tar.gz complete in 0s. 14:51:55 --- noir-projects build --- 14:51:55 Executing: prep (http://ci.aztec-labs.com/33a371378417d3dd) 14:51:58 0 . done (3s) 14:51:58 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/0fb63369447c46aa) 14:51:58 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (http://ci.aztec-labs.com/738ecc22fcd7ede5) 14:51:58 noir-contracts Executing: ./noir-contracts/bootstrap.sh (http://ci.aztec-labs.com/d8b1bd00cf31ce3d) 14:51:58 aztec-nr Executing: ./aztec-nr/bootstrap.sh (http://ci.aztec-labs.com/bd3b46ab8f85baae) 14:52:00 noir-contracts 0 ................................................................ 14:52:00 noir-contracts 64 ................................................................ 14:52:00 noir-contracts 128 ................................................................ 14:52:00 noir-contracts 192 ................................................................ 14:52:01 noir-contracts 256 ................................................................ 14:52:01 noir-contracts 320 ................................................................ 14:52:02 noir-contracts 384 ................................................................ 14:52:02 noir-contracts 448 ................................................................ 14:52:02 noir-contracts 512 ................................................................ 14:52:02 noir-contracts 576 ................................................................ 14:52:02 noir-contracts 640 ................................................................ 14:52:03 noir-contracts 704 ................................................................ 14:52:03 noir-contracts 768 ................................................................ 14:52:03 noir-contracts 832 ................................................................ 14:52:03 noir-contracts 896 ................................................................ 14:52:03 noir-contracts 960 ................................................................ 14:52:03 noir-contracts 1024 ................................................................ 14:52:04 noir-protocol-circuits 0 ................................................................ 14:52:04 noir-protocol-circuits 64 ................................................................ 14:52:04 noir-contracts 1088 ................................................................ 14:52:04 noir-contracts 1152 ................................................................ 14:52:04 noir-contracts 1216 ................................................................ 14:52:04 noir-contracts 1280 ................................................................ 14:52:04 noir-contracts 1344 ................................................................ 14:52:05 noir-contracts 1408 ................................................................ 14:52:05 noir-protocol-circuits 128 ................................................................ 14:52:05 noir-contracts 1472 ................................................................ 14:52:05 noir-contracts 1536 ................................................................ 14:52:05 noir-contracts 1600 . done (7s) 14:52:06 noir-protocol-circuits 192 ................................................................ 14:52:06 aztec-nr 0 . done (8s) 14:52:06 mock-protocol-circuits 0 ................................................................ 14:52:06 noir-protocol-circuits 256 ................................................................ 14:52:08 noir-protocol-circuits 320 ................................................................ 14:52:08 mock-protocol-circuits 64 . done (10s) 14:52:08 noir-protocol-circuits 384 ................................................................ 14:52:08 noir-protocol-circuits 448 . done (10s) 14:52:09 --- l1-contracts build --- 14:52:09 Cache download and extraction of l1-contracts-0a920e2ab51d8ecb.tar.gz complete in 0s. 14:52:09 --- yarn-project build --- 14:52:09 Executing: ./bootstrap.sh clean-lite (http://ci.aztec-labs.com/dbc385a519f211a7) 14:52:09 0 . done (0s) 14:52:16 Cache download and extraction of node-modules-34bec2a54a394404.zst complete in 6s. 14:52:16 Executing: compile_all (http://ci.aztec-labs.com/dd1cd29d5800c43a) 14:52:20 0 . done (4s) 14:52:20 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (http://ci.aztec-labs.com/a1f32d697eb23d54) 14:52:20 playground/bootstrap.sh Executing: playground/bootstrap.sh (http://ci.aztec-labs.com/ddddd4d7dde5d932) 14:52:20 docs/bootstrap.sh Executing: docs/bootstrap.sh (http://ci.aztec-labs.com/c031628c90bab1ad) 14:52:20 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (http://ci.aztec-labs.com/cc2d2226309fae79) 14:52:20 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (http://ci.aztec-labs.com/9325d1c89aa2384e) 14:52:20 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (http://ci.aztec-labs.com/c1fb2dc69a53d14f) 14:52:21 aztec-up/bootstrap.sh 0 . done (1s) 14:52:25 boxes/bootstrap.sh 0 . done (5s) 14:52:26 playground/bootstrap.sh 0 . done (6s) 14:52:27 spartan/bootstrap.sh 0 . done (7s) 14:52:29 docs/bootstrap.sh 0 . done (9s) 14:52:58 release-image/bootstrap.sh 0 . done (38s) 14:52:58 --- test all --- 14:52:58 Waiting for TXE's to start... 14:53:01 Gathering tests to run... 14:54:43 Gathered 12158 tests. 14:54:43 Starting parallel run with max 64 jobs... 14:54:43 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (http://ci.aztec-labs.com/a83250cda7d74c19) 14:55:14 0 ................................................................ 14:56:17 64 ................................................................ 14:57:21 128 ................................................................ 14:58:25 192 ................................................................ 14:58:49 256 ................................................................ 14:59:01 320 ................................................................ 14:59:19 384 ................................................................ 14:59:35 448 ................................................................ 14:59:42 512 ................................................................ 14:59:48 576 ................................................................ 14:59:59 640 ................................................................ 15:00:08 704 ................................................................ 15:00:14 768 ................................................................ 15:00:20 832 ................................................................ 15:00:28 896 ................................................................ 15:00:33 960 ................................................................ 15:00:39 1024 ................................................................ 15:00:43 1088 ................................................................ 15:00:51 1152 ................................................................ 15:00:59 1216 ................................................................ 15:01:02 1280 ................................................................ 15:01:05 1344 ................................................................ 15:01:08 1408 ................................................................ 15:01:12 1472 ................................................................ 15:01:15 1536 ................................................................ 15:01:23 1600 ................................................................ 15:01:32 1664 ................................................................ 15:01:47 1728 ................................................................ 15:01:52 1792 ................................................................ 15:01:56 1856 ................................................................ 15:01:59 1920 ................................................................ 15:02:01 1984 ................................................................ 15:02:03 2048 ................................................................ 15:02:05 2112 ................................................................ 15:02:06 2176 ................................................................ 15:02:08 2240 ................................................................ 15:02:10 2304 ................................................................ 15:02:11 2368 ................................................................ 15:02:13 2432 ................................................................ 15:02:14 2496 ................................................................ 15:02:16 2560 ................................................................ 15:02:31 2624 ................................................................ 15:02:37 2688 ................................................................ 15:02:39 2752 ................................................................ 15:02:41 2816 ................................................................ 15:02:42 2880 ................................................................ 15:02:44 2944 ................................................................ 15:02:55 3008 ................................................................ 15:03:00 3072 ................................................................ 15:03:05 3136 ................................................................ 15:03:06 3200 ................................................................ 15:03:08 3264 ................................................................ 15:03:10 3328 ................................................................ 15:03:11 3392 ................................................................ 15:03:13 3456 ................................................................ 15:03:16 3520 ................................................................ 15:03:19 3584 ................................................................ 15:03:21 3648 ................................................................ 15:03:22 3712 ................................................................ 15:03:24 3776 ................................................................ 15:03:25 3840 ................................................................ 15:03:26 3904 ................................................................ 15:03:28 3968 ................................................................ 15:03:29 4032 ................................................................ 15:03:30 4096 ................................................................ 15:03:32 4160 ................................................................ 15:03:35 4224 ................................................................ 15:03:39 4288 ................................................................ 15:03:42 4352 ................................................................ 15:03:45 4416 ................................................................ 15:03:48 4480 ................................................................ 15:03:51 4544 ................................................................ 15:03:53 4608 ................................................................ 15:03:55 4672 ................................................................ 15:03:58 4736 ................................................................ 15:04:00 4800 ................................................................ 15:04:02 4864 ................................................................ 15:04:04 4928 ................................................................ 15:04:06 4992 ................................................................ 15:04:07 5056 ................................................................ 15:04:09 5120 ................................................................ 15:04:11 5184 ................................................................ 15:04:12 5248 ................................................................ 15:04:13 5312 ................................................................ 15:04:15 5376 ................................................................ 15:04:16 5440 ................................................................ 15:04:18 5504 ................................................................ 15:04:20 5568 ................................................................ 15:04:22 5632 ................................................................ 15:04:24 5696 ................................................................ 15:04:26 5760 ................................................................ 15:04:29 5824 ................................................................ 15:04:31 5888 ................................................................ 15:04:33 5952 ................................................................ 15:04:35 6016 ................................................................ 15:04:37 6080 ................................................................ 15:04:39 6144 ................................................................ 15:04:42 6208 ................................................................ 15:04:44 6272 ................................................................ 15:04:47 6336 ................................................................ 15:04:49 6400 ................................................................ 15:04:52 6464 ................................................................ 15:04:54 6528 ................................................................ 15:04:57 6592 ................................................................ 15:05:01 6656 ................................................................ 15:05:03 6720 ................................................................ 15:05:06 6784 ................................................................ 15:05:08 6848 ................................................................ 15:05:10 6912 ................................................................ 15:05:12 6976 ................................................................ 15:05:14 7040 ................................................................ 15:05:16 7104 ................................................................ 15:05:18 7168 ................................................................ 15:05:20 7232 ................................................................ 15:05:21 7296 ................................................................ 15:05:22 7360 ................................................................ 15:05:24 7424 ................................................................ 15:05:25 7488 ................................................................ 15:05:26 7552 ................................................................ 15:05:27 7616 ................................................................ 15:05:29 7680 ................................................................ 15:05:30 7744 ................................................................ 15:05:31 7808 ................................................................ 15:05:32 7872 ................................................................ 15:05:34 7936 ................................................................ 15:05:35 8000 ................................................................ 15:05:36 8064 ................................................................ 15:05:37 8128 ................................................................ 15:05:39 8192 ................................................................ 15:05:40 8256 ................................................................ 15:05:41 8320 ................................................................ 15:05:42 8384 ................................................................ 15:05:44 8448 ................................................................ 15:05:45 8512 ................................................................ 15:05:46 8576 ................................................................ 15:05:47 8640 ................................................................ 15:05:48 8704 ................................................................ 15:05:50 8768 ................................................................ 15:05:51 8832 ................................................................ 15:05:52 8896 ................................................................ 15:05:53 8960 ................................................................ 15:05:54 9024 ................................................................ 15:05:56 9088 ................................................................ 15:05:57 9152 ................................................................ 15:05:58 9216 ................................................................ 15:06:00 9280 ................................................................ 15:06:01 9344 ................................................................ 15:06:03 9408 ................................................................ 15:06:04 9472 ................................................................ 15:06:06 9536 ................................................................ 15:06:07 9600 ................................................................ 15:06:09 9664 ................................................................ 15:06:11 9728 ................................................................ 15:06:12 9792 ................................................................ 15:06:14 9856 ................................................................ 15:06:16 9920 ................................................................ 15:06:16 9984 ................................................................ 15:06:17 10048 ................................................................ 15:06:19 10112 ................................................................ 15:06:20 10176 ................................................................ 15:06:21 10240 ................................................................ 15:06:22 10304 ................................................................ 15:06:23 10368 ................................................................ 15:06:24 10432 ................................................................ 15:06:26 10496 ................................................................ 15:06:28 10560 ................................................................ 15:06:29 10624 ................................................................ 15:06:30 10688 ................................................................ 15:06:31 10752 ................................................................ 15:06:33 10816 ................................................................ 15:06:34 10880 ................................................................ 15:06:35 10944 ................................................................ 15:06:36 11008 ................................................................ 15:06:37 11072 ................................................................ 15:06:38 11136 ................................................................ 15:06:39 11200 ................................................................ 15:06:41 11264 ................................................................ 15:06:42 11328 ................................................................ 15:06:43 11392 ................................................................ 15:06:44 11456 ................................................................ 15:06:45 11520 ................................................................ 15:06:46 11584 ................................................................ 15:06:47 11648 ................................................................ 15:06:48 11712 ................................................................ 15:06:49 11776 ................................................................ 15:06:50 11840 ................................................................ 15:06:51 11904 ................................................................ 15:06:52 11968 ................................................................ 15:06:53 12032 ................................................................ 15:06:55 12096 ................................................................ 15:06:56 12160 ................................................................ 15:06:57 12224 ................................................................ 15:06:58 12288 ................................................................ 15:07:11 12352 ................................................................ 15:09:36 12416 . done (893s) 15:09:36 WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them. 15:09:36 304.422: 'ef85c2b8680d9e9c:ISOLATE=1:NAME=e2e_p2p/validators_sentinel yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/validators_sentinel.test.ts' 15:09:36 325.667: '6c32ccc844d598b0 barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness' 15:09:36 339.888: 'ef85c2b8680d9e9c:ISOLATE=1:NAME=e2e_p2p/data_withholding_slash yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/data_withholding_slash.test.ts' 15:09:36 346.482: 'beb8af968925c4a6:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes' 15:09:36 349.154: '6c32ccc844d598b0:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing' 15:09:36 357.619: '6c32ccc844d598b0:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing' 15:09:36 364.034: '8969ea11e31033a3 aztec-up/scripts/run_test.sh amm_flow' 15:09:36 404.547: 'ef85c2b8680d9e9c:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building' 15:09:36 424.037: 'ef85c2b8680d9e9c:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts' 15:09:36 429.671: 'ef85c2b8680d9e9c:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts' 15:09:36 443.523: 'ef85c2b8680d9e9c:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts' 15:09:36 731.694: 'ef85c2b8680d9e9c:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full' 15:09:36 Completed run of 12158 tests in 893 seconds. 15:09:36 --- bench all --- 15:09:36 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (http://ci.aztec-labs.com/1e38ff55baa55432) 15:09:36 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (http://ci.aztec-labs.com/6ba1ededc3e2ac21) 15:09:37 barretenberg/cpp 0 . done (1s) 15:11:39 yarn-project/end-to-end 0 ................................................................ 15:14:30 yarn-project/end-to-end 64 . done (294s) 15:14:32 Starting parallel run using max 64 cpus... 15:14:32 Test run log id: http://ci.aztec-labs.com/e2f78405cc25532c 15:19:37 Completed run of 95 tests in 305 seconds. 15:19:39 Completed 7.6 KiB/7.6 KiB (100.7 KiB/s) with 1 file(s) remaining upload: ../../../tmp/tmp.qfTzYjR1ik to s3://aztec-ci-artifacts/build-cache/bench-9ba478709ce5c48099f79f9531da8292811cb18a.tar.gz 15:19:39 Cache upload of bench-9ba478709ce5c48099f79f9531da8292811cb18a.tar.gz complete in 1s. 15:19:39 + local code=0 15:19:39 + set +x 15:19:39 dmesg log id: http://ci.aztec-labs.com/fe715740ffb6d7fe CI run log id: http://ci.aztec-labs.com/1750776665092 + code=0 + case $code in + log_ci_run PASSED 1750776665092 + exit 0 SSH exited with code: 0 Terminating instance: i-03e7806db39fe5952...