Parent Log:
http://ci.aztec-labs.com/15796018905
Command: bootstrap_ec2 './bootstrap.sh ci-full'
Date: Sat Jun 21 13:40:52 UTC 2025
Status: 0
Took: 1774s
--- request build instance ---
Requesting m6a.32xlarge spot instance (name: pr-15201_amd64_x4-full) (type: m6a.32xlarge) (ami: ami-051d99fd9abf53087) (bid: 5.5424)...
Waiting for instance id for spot request: sir-ifkf5g9g...
Waiting for SSH at 13.58.46.164...
Stdout is not a tty, running in background...
Loading CRS into tmpfs...
Done in 9 seconds.
Stdout is not a tty, starting build container in background...
Starting devbox...
From
https://github.com/aztecprotocol/aztec-packages
* branch f495236d35f9bb0d1086b7ec79256c2c2b7c98b8 -> FETCH_HEAD
HEAD is now at f495236 chore: merge next to staging (#15201)
Switched to a new branch 'gh-readonly-queue/staging/pr-15201-f0619dd82429a5973f3e1da8d7eb0877264908e3'
+ run
+ ci3/add_timestamps
+ DUP=1
+ ci3/cache_log 'CI run' 1750511533365
CI run log id:
http://ci.aztec-labs.com/1750511533365
13:12:13 + echo 'env: REF_NAME=gh-readonly-queue/staging/pr-15201-f0619dd82429a5973f3e1da8d7eb0877264908e3 COMMIT_HASH=f495236d35f9bb0d1086b7ec79256c2c2b7c98b8 CURRENT_VERSION=0.87.6 TARGET_BRANCH=staging'
13:12:13 env: REF_NAME=gh-readonly-queue/staging/pr-15201-f0619dd82429a5973f3e1da8d7eb0877264908e3 COMMIT_HASH=f495236d35f9bb0d1086b7ec79256c2c2b7c98b8 CURRENT_VERSION=0.87.6 TARGET_BRANCH=staging
13:12:13 + semver check gh-readonly-queue/staging/pr-15201-f0619dd82429a5973f3e1da8d7eb0877264908e3
13:12:13 + cpu_pid=413
13:12:13 + mem_pid=415
13:12:13 + set +e
13:12:13 + mpstat 2
13:12:13 + set -x
13:12:13 + ./bootstrap.sh ci-full
13:12:13 + vmstat -w -S M 2
13:12:13 ++ cache_log memfile
13:12:13 ++ cache_log cpufile
13:12:13 cpufile log id:
http://ci.aztec-labs.com/bdee6e046398ed16
13:12:13 memfile log id:
http://ci.aztec-labs.com/6d60083d0ce6befb
13:12:21
--- pull submodules ---
13:12:21 Executing: git submodule update --init --recursive (
http://ci.aztec-labs.com/e867dca2eaf9e5f6)
13:12:29 0 .
done (8s)
13:12:31
--- noir build ---
13:12:31 Executing: retry install_deps (
http://ci.aztec-labs.com/be28aea1bb1d2511)
13:12:35 0 .
done (4s)
13:12:36 build_native Executing: build_native (
http://ci.aztec-labs.com/cb623c2a4f173d3e)
13:12:36 build_packages Executing: build_packages (
http://ci.aztec-labs.com/f454cbfc454f9e7a)
13:12:37 build_native 0 .
done (1s)
13:12:42 build_packages 0 .
done (6s)
13:12:43
--- bb cpp build ---
13:12:43 build_wasm Executing: build_wasm (
http://ci.aztec-labs.com/b89d30361b505a66)
13:12:43 build_nodejs_module Executing: build_nodejs_module (
http://ci.aztec-labs.com/a6525e0e19ad07bd)
13:12:43 build_wasm_threads Executing: build_wasm_threads (
http://ci.aztec-labs.com/d308b53535e25690)
13:12:43 build_native Executing: build_native (
http://ci.aztec-labs.com/6f5c9aa02238b6ab)
13:12:43 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (
http://ci.aztec-labs.com/3fdb820756f4dd40)
13:12:43 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (
http://ci.aztec-labs.com/ef04ec8a2aa9764c)
13:12:43 build_asan_fast Executing: build_asan_fast (
http://ci.aztec-labs.com/ed3f87f8d2abf7b8)
13:12:43 build_darwin Executing: build_darwin (
http://ci.aztec-labs.com/fded73d63ca2ec88)
13:12:43 build_fuzzing_syntax_check_only 0 .
done (0s)
13:12:43 build_gcc_syntax_check_only 0 .
done (0s)
13:12:44 build_wasm_threads 0 .
done (1s)
13:12:44 build_darwin 0 .
done (1s)
13:12:44 build_wasm 0 .
done (1s)
13:12:44 build_nodejs_module 0 .
done (1s)
13:12:45 build_native 0 .
done (2s)
13:12:45 build_asan_fast 0 .
done (2s)
13:12:50
--- bb.js build ---
13:12:51 Cache download and extraction of node-modules-e85320bda7d59d54.zst complete in 1s.
13:12:51 Cache download and extraction of bb.js-93955768547ae31a.tar.gz complete in 0s.
13:12:52
--- acir_tests build ---
13:12:52 Cache download and extraction of barretenberg-acir-tests-000ad54a8a7b4100.tar.gz complete in 0s.
13:12:53 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s.
13:12:54 browser-test-app Executing: yarn build (
http://ci.aztec-labs.com/c67c5d63f15f90c0)
13:12:54 bbjs-test Executing: yarn build (
http://ci.aztec-labs.com/11459429fab46823)
13:12:57 bbjs-test 0 .
done (3s)
13:12:58 browser-test-app 0 .
done (4s)
13:12:58
--- avm-transpiler build ---
13:12:59 Cache download and extraction of avm-transpiler-6867cd9272ea7ebd.tar.gz complete in 1s.
13:12:59
--- noir-projects build ---
13:12:59 Executing: prep (
http://ci.aztec-labs.com/2b631bd8e414becf)
13:13:01 0 .
done (2s)
13:13:02 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/2788ee54e0e3afce)
13:13:02 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/a221f0948f65701a)
13:13:02 noir-contracts Executing: ./noir-contracts/bootstrap.sh (
http://ci.aztec-labs.com/dbd0ccea2e95b28c)
13:13:02 aztec-nr Executing: ./aztec-nr/bootstrap.sh (
http://ci.aztec-labs.com/7a22da912cd0e65a)
13:13:03 noir-contracts 0 ................................................................
13:13:03 noir-contracts 64 ................................................................
13:13:03 noir-contracts 128 ................................................................
13:13:03 noir-contracts 192 ................................................................
13:13:03 noir-contracts 256 ................................................................
13:13:04 noir-contracts 320 ................................................................
13:13:04 noir-contracts 384 ................................................................
13:13:04 noir-contracts 448 ................................................................
13:13:04 noir-contracts 512 ................................................................
13:13:04 noir-contracts 576 ................................................................
13:13:04 noir-contracts 640 ................................................................
13:13:05 noir-contracts 704 ................................................................
13:13:05 noir-contracts 768 ................................................................
13:13:05 noir-contracts 832 ................................................................
13:13:05 noir-contracts 896 ................................................................
13:13:05 noir-contracts 960 ................................................................
13:13:05 noir-contracts 1024 ................................................................
13:13:06 noir-contracts 1088 ................................................................
13:13:06 noir-contracts 1152 ................................................................
13:13:06 noir-contracts 1216 ................................................................
13:13:06 noir-contracts 1280 ................................................................
13:13:06 noir-contracts 1344 ................................................................
13:13:06 noir-contracts 1408 ................................................................
13:13:06 noir-contracts 1472 ................................................................
13:13:07 noir-contracts 1536 ................................................................
13:13:07 noir-contracts 1600 .
done (5s)
13:13:07 noir-protocol-circuits 0 ................................................................
13:13:07 noir-protocol-circuits 64 ................................................................
13:13:07 noir-protocol-circuits 128 ................................................................
13:13:08 noir-protocol-circuits 192 ................................................................
13:13:08 mock-protocol-circuits 0 ................................................................
13:13:09 noir-protocol-circuits 256 ................................................................
13:13:09 aztec-nr 0 .
done (7s)
13:13:10 noir-protocol-circuits 320 ................................................................
13:13:11 mock-protocol-circuits 64 .
done (9s)
13:13:11 noir-protocol-circuits 384 ................................................................
13:13:11 noir-protocol-circuits 448 .
done (9s)
13:13:11
--- l1-contracts build ---
13:13:11 Cache download and extraction of l1-contracts-0a920e2ab51d8ecb.tar.gz complete in 0s.
13:13:11
--- yarn-project build ---
13:13:11 Executing: ./bootstrap.sh clean-lite (
http://ci.aztec-labs.com/569253106b3e62e2)
13:13:12 0 .
done (1s)
13:13:17 Cache download and extraction of node-modules-34bec2a54a394404.zst complete in 5s.
13:13:17 Executing: compile_all (
http://ci.aztec-labs.com/6ca09fd9e6a15f31)
13:13:20 0 .
done (3s)
13:13:21 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (
http://ci.aztec-labs.com/4cf5b87c02b59c4c)
13:13:21 playground/bootstrap.sh Executing: playground/bootstrap.sh (
http://ci.aztec-labs.com/ce4d9a39781f9c66)
13:13:21 docs/bootstrap.sh Executing: docs/bootstrap.sh (
http://ci.aztec-labs.com/cf562eae2cb4bca8)
13:13:21 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (
http://ci.aztec-labs.com/3316b4f391e30105)
13:13:21 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (
http://ci.aztec-labs.com/f224a23eda7d0b3f)
13:13:21 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (
http://ci.aztec-labs.com/dfbfa93df446f465)
13:13:21 aztec-up/bootstrap.sh 0 .
done (0s)
13:13:24 boxes/bootstrap.sh 0 .
done (3s)
13:13:25 playground/bootstrap.sh 0 .
done (4s)
13:13:27 spartan/bootstrap.sh 0 .
done (6s)
13:13:28 docs/bootstrap.sh 0 .
done (7s)
13:13:58 release-image/bootstrap.sh 0 .
done (37s)
13:13:58
--- test all ---
13:13:58 Waiting for TXE's to start...
13:14:01 Gathering tests to run...
13:15:42 Gathered 12158 tests.
13:15:42 Starting parallel run with max 64 jobs...
13:15:42 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (
http://ci.aztec-labs.com/f58a71454a9abef7)
13:16:13 0 ................................................................
13:17:22 64 ................................................................
13:18:24 128 ................................................................
13:19:26 192 ................................................................
13:19:51 256 ................................................................
13:20:01 320 ................................................................
13:20:20 384 ................................................................
13:20:36 448 ................................................................
13:20:44 512 ................................................................
13:20:50 576 ................................................................
13:21:02 640 ................................................................
13:21:10 704 ................................................................
13:21:17 768 ................................................................
13:21:23 832 ................................................................
13:21:31 896 ................................................................
13:21:37 960 ................................................................
13:21:42 1024 ................................................................
13:21:45 1088 ................................................................
13:21:55 1152 ................................................................
13:22:02 1216 ................................................................
13:22:05 1280 ................................................................
13:22:09 1344 ................................................................
13:22:12 1408 ................................................................
13:22:15 1472 ................................................................
13:22:19 1536 ................................................................
13:22:26 1600 ................................................................
13:22:36 1664 ................................................................
13:22:51 1728 ................................................................
13:22:55 1792 ................................................................
13:23:00 1856 ................................................................
13:23:03 1920 ................................................................
13:23:04 1984 ................................................................
13:23:06 2048 ................................................................
13:23:09 2112 ................................................................
13:23:10 2176 ................................................................
13:23:12 2240 ................................................................
13:23:14 2304 ................................................................
13:23:16 2368 ................................................................
13:23:17 2432 ................................................................
13:23:18 2496 ................................................................
13:23:19 2560 ................................................................
13:23:35 2624 ................................................................
13:23:42 2688 ................................................................
13:23:43 2752 ................................................................
13:23:45 2816 ................................................................
13:23:46 2880 ................................................................
13:23:49 2944 ................................................................
13:24:00 3008 ................................................................
13:24:07 3072 ................................................................
13:24:11 3136 ................................................................
13:24:13 3200 ................................................................
13:24:14 3264 ................................................................
13:24:16 3328 ................................................................
13:24:17 3392 ................................................................
13:24:19 3456 ................................................................
13:24:22 3520 ................................................................
13:24:25 3584 ................................................................
13:24:27 3648 ................................................................
13:24:29 3712 ................................................................
13:24:30 3776 ................................................................
13:24:31 3840 ................................................................
13:24:32 3904 ................................................................
13:24:34 3968 ................................................................
13:24:35 4032 ................................................................
13:24:36 4096 ................................................................
13:24:38 4160 ................................................................
13:24:41 4224 ................................................................
13:24:45 4288 ................................................................
13:24:50 4352 ................................................................
13:24:53 4416 ................................................................
13:24:56 4480 ................................................................
13:24:58 4544 ................................................................
13:25:00 4608 ................................................................
13:25:02 4672 ................................................................
13:25:05 4736 ................................................................
13:25:07 4800 ................................................................
13:25:09 4864 ................................................................
13:25:11 4928 ................................................................
13:25:13 4992 ................................................................
13:25:15 5056 ................................................................
13:25:17 5120 ................................................................
13:25:19 5184 ................................................................
13:25:20 5248 ................................................................
13:25:21 5312 ................................................................
13:25:23 5376 ................................................................
13:25:24 5440 ................................................................
13:25:27 5504 ................................................................
13:25:29 5568 ................................................................
13:25:31 5632 ................................................................
13:25:33 5696 ................................................................
13:25:35 5760 ................................................................
13:25:37 5824 ................................................................
13:25:40 5888 ................................................................
13:25:42 5952 ................................................................
13:25:44 6016 ................................................................
13:25:46 6080 ................................................................
13:25:48 6144 ................................................................
13:25:51 6208 ................................................................
13:25:54 6272 ................................................................
13:25:56 6336 ................................................................
13:25:58 6400 ................................................................
13:26:01 6464 ................................................................
13:26:04 6528 ................................................................
13:26:07 6592 ................................................................
13:26:10 6656 ................................................................
13:26:13 6720 ................................................................
13:26:15 6784 ................................................................
13:26:17 6848 ................................................................
13:26:20 6912 ................................................................
13:26:21 6976 ................................................................
13:26:24 7040 ................................................................
13:26:26 7104 ................................................................
13:26:28 7168 ................................................................
13:26:30 7232 ................................................................
13:26:31 7296 ................................................................
13:26:32 7360 ................................................................
13:26:33 7424 ................................................................
13:26:35 7488 ................................................................
13:26:36 7552 ................................................................
13:26:37 7616 ................................................................
13:26:38 7680 ................................................................
13:26:39 7744 ................................................................
13:26:41 7808 ................................................................
13:26:42 7872 ................................................................
13:26:43 7936 ................................................................
13:26:45 8000 ................................................................
13:26:46 8064 ................................................................
13:26:47 8128 ................................................................
13:26:48 8192 ................................................................
13:26:49 8256 ................................................................
13:26:50 8320 ................................................................
13:26:52 8384 ................................................................
13:26:53 8448 ................................................................
13:26:54 8512 ................................................................
13:26:55 8576 ................................................................
13:26:56 8640 ................................................................
13:26:58 8704 ................................................................
13:26:59 8768 ................................................................
13:27:00 8832 ................................................................
13:27:01 8896 ................................................................
13:27:02 8960 ................................................................
13:27:04 9024 ................................................................
13:27:05 9088 ................................................................
13:27:07 9152 ................................................................
13:27:08 9216 ................................................................
13:27:09 9280 ................................................................
13:27:11 9344 ................................................................
13:27:12 9408 ................................................................
13:27:14 9472 ................................................................
13:27:15 9536 ................................................................
13:27:16 9600 ................................................................
13:27:18 9664 ................................................................
13:27:20 9728 ................................................................
13:27:21 9792 ................................................................
13:27:23 9856 ................................................................
13:27:24 9920 ................................................................
13:27:25 9984 ................................................................
13:27:26 10048 ................................................................
13:27:27 10112 ................................................................
13:27:28 10176 ................................................................
13:27:29 10240 ................................................................
13:27:30 10304 ................................................................
13:27:32 10368 ................................................................
13:27:33 10432 ................................................................
13:27:34 10496 ................................................................
13:27:35 10560 ................................................................
13:27:37 10624 ................................................................
13:27:38 10688 ................................................................
13:27:39 10752 ................................................................
13:27:40 10816 ................................................................
13:27:41 10880 ................................................................
13:27:42 10944 ................................................................
13:27:43 11008 ................................................................
13:27:44 11072 ................................................................
13:27:45 11136 ................................................................
13:27:46 11200 ................................................................
13:27:47 11264 ................................................................
13:27:48 11328 ................................................................
13:27:49 11392 ................................................................
13:27:50 11456 ................................................................
13:27:51 11520 ................................................................
13:27:52 11584 ................................................................
13:27:53 11648 ................................................................
13:27:54 11712 ................................................................
13:27:55 11776 ................................................................
13:27:56 11840 ................................................................
13:27:57 11904 ................................................................
13:27:58 11968 ................................................................
13:27:59 12032 ................................................................
13:28:00 12096 ................................................................
13:28:01 12160 ................................................................
13:28:02 12224 ................................................................
13:28:03 12288 ................................................................
13:28:04 12352 ................................................................
13:30:37 12416 .
done (895s)
13:30:37
WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them.
13:30:37 301.435: '68e9475192d7aeac:ISOLATE=1:NAME=e2e_p2p/validators_sentinel yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/validators_sentinel.test.ts'
13:30:37 310.379: '6c8b1f2818608a3b noir-projects/scripts/run_test.sh noir-protocol-circuits rollup_lib block_root::block_root_rollup_inputs::tests::correct_blobs_with_non_empty_fields'
13:30:37 326.131: '6c32ccc844d598b0 barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness'
13:30:37 344.891: '68e9475192d7aeac:ISOLATE=1:NAME=e2e_p2p/data_withholding_slash yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/data_withholding_slash.test.ts'
13:30:37 349.072: '540e879918a5d1f3:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes'
13:30:37 360.782: '6c32ccc844d598b0:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing'
13:30:37 362.470: '6c32ccc844d598b0:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing'
13:30:37 370.138: 'a7ed594c3203ee82 aztec-up/scripts/run_test.sh amm_flow'
13:30:37 384.940: '68e9475192d7aeac:ISOLATE=1:NAME=p2p/src/client/p2p_client.integration.test.ts LOG_LEVEL=debug yarn-project/scripts/run_test.sh p2p/src/client/p2p_client.integration.test.ts'
13:30:37 395.648: '68e9475192d7aeac:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building'
13:30:37 424.215: '68e9475192d7aeac:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts'
13:30:37 430.294: '68e9475192d7aeac:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts'
13:30:37 443.654: '68e9475192d7aeac:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts'
13:30:37 737.119: '68e9475192d7aeac:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full'
13:30:37 Completed run of 12158 tests in 895 seconds.
13:30:37
--- bench all ---
13:30:37 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (
http://ci.aztec-labs.com/beabda9592ea2249)
13:30:37 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (
http://ci.aztec-labs.com/249ffc1c37ab6081)
13:30:38 barretenberg/cpp 0 .
done (1s)
13:32:38 yarn-project/end-to-end 0 ................................................................
13:35:26 yarn-project/end-to-end 64 ................................................................
13:35:26 yarn-project/end-to-end 128 ................................................................
13:35:27 yarn-project/end-to-end 192 ................................................................
13:35:27 yarn-project/end-to-end 256 ................................................................
13:35:27 yarn-project/end-to-end 320 ................................................................
13:35:27 yarn-project/end-to-end 384 ................................................................
13:35:27 yarn-project/end-to-end 448 ................................................................
13:35:27 yarn-project/end-to-end 512 ................................................................
13:35:27 yarn-project/end-to-end 576 ................................................................
13:35:27 yarn-project/end-to-end 640 ................................................................
13:35:27 yarn-project/end-to-end 704 ................................................................
13:35:27 yarn-project/end-to-end 768 ................................................................
13:35:31 yarn-project/end-to-end 832 .
done (294s)
13:35:32 Starting parallel run using max 64 cpus...
13:35:32 Test run log id:
http://ci.aztec-labs.com/ef23eebb5a3ffed6
13:40:39 Completed run of 95 tests in 307 seconds.
13:40:42 Completed 7.6 KiB/7.6 KiB (82.8 KiB/s) with 1 file(s) remaining
upload: ../../../tmp/tmp.DNrx4CxYqx to s3://aztec-ci-artifacts/build-cache/bench-f495236d35f9bb0d1086b7ec79256c2c2b7c98b8.tar.gz
13:40:42 Cache upload of bench-f495236d35f9bb0d1086b7ec79256c2c2b7c98b8.tar.gz complete in 1s.
13:40:42 + local code=0
13:40:42 + set +x
13:40:42 dmesg log id:
http://ci.aztec-labs.com/19af6558e64a5f18
CI run log id:
http://ci.aztec-labs.com/1750511533365
+ code=0
+ case $code in
+ log_ci_run PASSED 1750511533365
+ exit 0
SSH exited with code: 0
Terminating instance: i-0a40dd50180026cc2...