Parent Log:
http://ci.aztec-labs.com/15782013234
Command: bootstrap_ec2 './bootstrap.sh ci-full'
Date: Fri Jun 20 15:46:28 UTC 2025
Status: 0
Took: 2134s
--- request build instance ---
Requesting m6a.32xlarge spot instance (name: pr-15154_amd64_x3-full) (type: m6a.32xlarge) (ami: ami-051d99fd9abf53087) (bid: 5.5424)...
Waiting for instance id for spot request: sir-614753ek...
Waiting for SSH at 18.118.37.69...
Stdout is not a tty, running in background...
Loading CRS into tmpfs...
Done in 13 seconds.
Stdout is not a tty, starting build container in background...
Starting devbox...
From
https://github.com/aztecprotocol/aztec-packages
* branch 28bf32bfc2566751e8baa0a9125a9db20d474bee -> FETCH_HEAD
HEAD is now at 28bf32b feat: auto-update exported metrics (#15154)
Switched to a new branch 'gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4'
+ run
+ ci3/add_timestamps
+ DUP=1
+ ci3/cache_log 'CI run' 1750432308681
CI run log id:
http://ci.aztec-labs.com/1750432308681
15:11:48 + echo 'env: REF_NAME=gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4 COMMIT_HASH=28bf32bfc2566751e8baa0a9125a9db20d474bee CURRENT_VERSION=0.87.6 TARGET_BRANCH=next'
15:11:48 env: REF_NAME=gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4 COMMIT_HASH=28bf32bfc2566751e8baa0a9125a9db20d474bee CURRENT_VERSION=0.87.6 TARGET_BRANCH=next
15:11:48 + semver check gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4
15:11:48 + cpu_pid=415
15:11:48 + mpstat 2
15:11:48 + mem_pid=416
15:11:48 + set +e
15:11:48 + set -x
15:11:48 + ./bootstrap.sh ci-full
15:11:48 + vmstat -w -S M 2
15:11:48 ++ cache_log cpufile
15:11:48 ++ cache_log memfile
15:11:48 memfile log id:
http://ci.aztec-labs.com/82cfd117f775d9be
15:11:48 cpufile log id:
http://ci.aztec-labs.com/b93be033d2eed252
15:11:56
--- pull submodules ---
15:11:56 Executing: git submodule update --init --recursive (
http://ci.aztec-labs.com/c4e9282088caffec)
15:12:04 0 .
done (8s)
15:12:06
--- noir build ---
15:12:06 Executing: retry install_deps (
http://ci.aztec-labs.com/3cfdb315a9691594)
15:12:10 0 .
done (4s)
15:12:11 build_native Executing: build_native (
http://ci.aztec-labs.com/c84f3dc23b0813e5)
15:12:11 build_packages Executing: build_packages (
http://ci.aztec-labs.com/6b25e193e832ac7e)
15:12:11 build_native 0 .
done (0s)
15:12:17 build_packages 0 .
done (6s)
15:12:18
--- bb cpp build ---
15:12:19 build_native Executing: build_native (
http://ci.aztec-labs.com/bc517493a8a32801)
15:12:19 build_wasm Executing: build_wasm (
http://ci.aztec-labs.com/e429e3162c5e60bf)
15:12:19 build_wasm_threads Executing: build_wasm_threads (
http://ci.aztec-labs.com/aab0c1110babf65e)
15:12:19 build_nodejs_module Executing: build_nodejs_module (
http://ci.aztec-labs.com/bbc167dca6f28b08)
15:12:19 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (
http://ci.aztec-labs.com/7759ba61321f5aaf)
15:12:19 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (
http://ci.aztec-labs.com/9212b223781555e0)
15:12:19 build_asan_fast Executing: build_asan_fast (
http://ci.aztec-labs.com/173890b41c909d4e)
15:12:19 build_darwin Executing: build_darwin (
http://ci.aztec-labs.com/dc2348546337d1a5)
15:12:19 build_gcc_syntax_check_only 0 .
done (0s)
15:12:19 build_fuzzing_syntax_check_only 0 .
done (0s)
15:12:19 build_darwin 0 .
done (0s)
15:12:19 build_wasm 0 .
done (0s)
15:12:19 build_wasm_threads 0 .
done (0s)
15:12:19 build_nodejs_module 0 .
done (0s)
15:12:21 build_native 0 .
done (2s)
15:12:22 build_asan_fast 0 .
done (3s)
15:12:27
--- bb.js build ---
15:12:28 Cache download and extraction of node-modules-e85320bda7d59d54.zst complete in 1s.
15:12:29 Cache download and extraction of bb.js-ee60cf044fbe25d5.tar.gz complete in 1s.
15:12:30
--- acir_tests build ---
15:12:30 Cache download and extraction of barretenberg-acir-tests-95b8ad4482daac2e.tar.gz complete in 0s.
15:12:31 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s.
15:12:31 browser-test-app Executing: yarn build (
http://ci.aztec-labs.com/6552a180079f3f61)
15:12:31 bbjs-test Executing: yarn build (
http://ci.aztec-labs.com/e5a1c4a54e16a60f)
15:12:35 bbjs-test 0 .
done (4s)
15:12:36 browser-test-app 0 .
done (5s)
15:12:37
--- avm-transpiler build ---
15:12:37 Cache download and extraction of avm-transpiler-6867cd9272ea7ebd.tar.gz complete in 0s.
15:12:37
--- noir-projects build ---
15:12:37 Executing: prep (
http://ci.aztec-labs.com/a9b46015f04db167)
15:12:40 0 .
done (3s)
15:12:40 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/4adaef74f0db1170)
15:12:40 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/c293a2118bddd6f3)
15:12:40 noir-contracts Executing: ./noir-contracts/bootstrap.sh (
http://ci.aztec-labs.com/20ca978b0d8e845e)
15:12:41 aztec-nr Executing: ./aztec-nr/bootstrap.sh (
http://ci.aztec-labs.com/cf856bc62baacfe0)
15:12:42 noir-contracts 0 ................................................................
15:12:42 noir-contracts 64 ................................................................
15:12:42 noir-contracts 128 ................................................................
15:12:42 noir-contracts 192 ................................................................
15:12:43 noir-contracts 256 ................................................................
15:12:43 noir-contracts 320 ................................................................
15:12:43 noir-contracts 384 ................................................................
15:12:43 noir-contracts 448 ................................................................
15:12:43 noir-contracts 512 ................................................................
15:12:44 noir-contracts 576 ................................................................
15:12:44 noir-contracts 640 ................................................................
15:12:44 noir-contracts 704 ................................................................
15:12:44 noir-contracts 768 ................................................................
15:12:45 noir-contracts 832 ................................................................
15:12:45 noir-contracts 896 ................................................................
15:12:45 noir-contracts 960 ................................................................
15:12:45 noir-contracts 1024 ................................................................
15:12:45 noir-contracts 1088 ................................................................
15:12:46 noir-contracts 1152 ................................................................
15:12:46 noir-contracts 1216 ................................................................
15:12:46 noir-contracts 1280 ................................................................
15:12:46 noir-contracts 1344 ................................................................
15:12:46 noir-protocol-circuits 0 ................................................................
15:12:46 noir-protocol-circuits 64 ................................................................
15:12:46 noir-contracts 1408 ................................................................
15:12:46 noir-contracts 1472 ................................................................
15:12:46 noir-contracts 1536 ................................................................
15:12:46 noir-contracts 1600 .
done (6s)
15:12:47 noir-protocol-circuits 128 ................................................................
15:12:47 noir-protocol-circuits 192 ................................................................
15:12:48 noir-protocol-circuits 256 ................................................................
15:12:48 aztec-nr 0 .
done (8s)
15:12:49 mock-protocol-circuits 0 ................................................................
15:12:49 noir-protocol-circuits 320 ................................................................
15:12:50 noir-protocol-circuits 384 ................................................................
15:12:52 noir-protocol-circuits 448 ................................................................
15:12:54 mock-protocol-circuits 64 ................................................................
15:15:25 noir-protocol-circuits 512 ................................................................
15:15:25 noir-protocol-circuits 576 ................................................................
15:15:25 noir-protocol-circuits 640 .
done (165s)
15:15:31 mock-protocol-circuits 128 .
done (171s)
15:15:32
--- l1-contracts build ---
15:15:32 Cache download and extraction of l1-contracts-52d0554817b52bf7.tar.gz complete in 0s.
15:15:32
--- yarn-project build ---
15:15:32 Executing: ./bootstrap.sh clean-lite (
http://ci.aztec-labs.com/75929ee19c5f7475)
15:15:32 0 .
done (0s)
15:15:37 Cache download and extraction of node-modules-34bec2a54a394404.zst complete in 4s.
15:15:37 Executing: compile_all (
http://ci.aztec-labs.com/0ec0a580a3ebc4a1)
15:15:47 0 ................................................................
15:18:09 64 .
done (152s)
15:18:09 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (
http://ci.aztec-labs.com/ee9187bb9a954c2a)
15:18:09 playground/bootstrap.sh Executing: playground/bootstrap.sh (
http://ci.aztec-labs.com/8a4d3768adb6aa96)
15:18:09 docs/bootstrap.sh Executing: docs/bootstrap.sh (
http://ci.aztec-labs.com/0b5d2e255d09b96e)
15:18:09 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (
http://ci.aztec-labs.com/be09a607969894c2)
15:18:09 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (
http://ci.aztec-labs.com/42627ec49a0f06a2)
15:18:09 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (
http://ci.aztec-labs.com/76e39aa82db65b9f)
15:18:10 aztec-up/bootstrap.sh 0 .
done (1s)
15:18:13 spartan/bootstrap.sh 0 .
done (4s)
15:18:17 docs/bootstrap.sh 0 .
done (8s)
15:18:42 release-image/bootstrap.sh 0 .
done (33s)
15:18:51 boxes/bootstrap.sh 0 .
done (42s)
15:19:00 playground/bootstrap.sh 0 .
done (51s)
15:19:00
--- test all ---
15:19:00 Waiting for TXE's to start...
15:19:03 Gathering tests to run...
15:20:45 Gathered 12132 tests.
15:20:45 Starting parallel run with max 64 jobs...
15:20:45 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (
http://ci.aztec-labs.com/e0d14622d89aaee4)
15:21:16 0 ................................................................
15:22:26 64 ................................................................
15:23:30 128 ................................................................
15:24:31 192 ................................................................
15:24:56 256 ................................................................
15:25:08 320 ................................................................
15:25:26 384 ................................................................
15:25:42 448 ................................................................
15:25:51 512 ................................................................
15:25:57 576 ................................................................
15:26:10 640 ................................................................
15:26:18 704 ................................................................
15:26:26 768 ................................................................
15:26:31 832 ................................................................
15:26:39 896 ................................................................
15:26:45 960 ................................................................
15:26:50 1024 ................................................................
15:26:53 1088 ................................................................
15:27:03 1152 ................................................................
15:27:12 1216 ................................................................
15:27:15 1280 ................................................................
15:27:19 1344 ................................................................
15:27:22 1408 ................................................................
15:27:25 1472 ................................................................
15:27:29 1536 ................................................................
15:27:37 1600 ................................................................
15:27:47 1664 ................................................................
15:28:03 1728 ................................................................
15:28:08 1792 ................................................................
15:28:13 1856 ................................................................
15:28:16 1920 ................................................................
15:28:18 1984 ................................................................
15:28:20 2048 ................................................................
15:28:23 2112 ................................................................
15:28:25 2176 ................................................................
15:28:27 2240 ................................................................
15:28:29 2304 ................................................................
15:28:31 2368 ................................................................
15:28:33 2432 ................................................................
15:28:35 2496 ................................................................
15:28:37 2560 ................................................................
15:28:48 2624 ................................................................
15:28:50 2688 ................................................................
15:28:52 2752 ................................................................
15:28:53 2816 ................................................................
15:28:55 2880 ................................................................
15:28:57 2944 ................................................................
15:29:14 3008 ................................................................
15:29:20 3072 ................................................................
15:29:24 3136 ................................................................
15:29:26 3200 ................................................................
15:29:28 3264 ................................................................
15:29:29 3328 ................................................................
15:29:31 3392 ................................................................
15:29:33 3456 ................................................................
15:29:37 3520 ................................................................
15:29:39 3584 ................................................................
15:29:41 3648 ................................................................
15:29:43 3712 ................................................................
15:29:44 3776 ................................................................
15:29:46 3840 ................................................................
15:29:47 3904 ................................................................
15:29:49 3968 ................................................................
15:29:50 4032 ................................................................
15:29:52 4096 ................................................................
15:29:54 4160 ................................................................
15:29:58 4224 ................................................................
15:30:03 4288 ................................................................
15:30:07 4352 ................................................................
15:30:10 4416 ................................................................
15:30:13 4480 ................................................................
15:30:15 4544 ................................................................
15:30:17 4608 ................................................................
15:30:20 4672 ................................................................
15:30:22 4736 ................................................................
15:30:24 4800 ................................................................
15:30:26 4864 ................................................................
15:30:28 4928 ................................................................
15:30:30 4992 ................................................................
15:30:32 5056 ................................................................
15:30:34 5120 ................................................................
15:30:36 5184 ................................................................
15:30:38 5248 ................................................................
15:30:40 5312 ................................................................
15:30:42 5376 ................................................................
15:30:44 5440 ................................................................
15:30:47 5504 ................................................................
15:30:50 5568 ................................................................
15:30:53 5632 ................................................................
15:30:55 5696 ................................................................
15:30:58 5760 ................................................................
15:31:00 5824 ................................................................
15:31:02 5888 ................................................................
15:31:04 5952 ................................................................
15:31:06 6016 ................................................................
15:31:08 6080 ................................................................
15:31:11 6144 ................................................................
15:31:14 6208 ................................................................
15:31:16 6272 ................................................................
15:31:18 6336 ................................................................
15:31:21 6400 ................................................................
15:31:24 6464 ................................................................
15:31:27 6528 ................................................................
15:31:29 6592 ................................................................
15:31:32 6656 ................................................................
15:31:35 6720 ................................................................
15:31:37 6784 ................................................................
15:31:39 6848 ................................................................
15:31:41 6912 ................................................................
15:31:43 6976 ................................................................
15:31:45 7040 ................................................................
15:31:47 7104 ................................................................
15:31:49 7168 ................................................................
15:31:51 7232 ................................................................
15:31:52 7296 ................................................................
15:31:53 7360 ................................................................
15:31:54 7424 ................................................................
15:31:56 7488 ................................................................
15:31:57 7552 ................................................................
15:31:59 7616 ................................................................
15:32:00 7680 ................................................................
15:32:01 7744 ................................................................
15:32:02 7808 ................................................................
15:32:04 7872 ................................................................
15:32:05 7936 ................................................................
15:32:06 8000 ................................................................
15:32:08 8064 ................................................................
15:32:09 8128 ................................................................
15:32:10 8192 ................................................................
15:32:12 8256 ................................................................
15:32:13 8320 ................................................................
15:32:14 8384 ................................................................
15:32:16 8448 ................................................................
15:32:17 8512 ................................................................
15:32:18 8576 ................................................................
15:32:19 8640 ................................................................
15:32:21 8704 ................................................................
15:32:22 8768 ................................................................
15:32:23 8832 ................................................................
15:32:24 8896 ................................................................
15:32:25 8960 ................................................................
15:32:27 9024 ................................................................
15:32:28 9088 ................................................................
15:32:30 9152 ................................................................
15:32:31 9216 ................................................................
15:32:33 9280 ................................................................
15:32:35 9344 ................................................................
15:32:36 9408 ................................................................
15:32:38 9472 ................................................................
15:32:39 9536 ................................................................
15:32:41 9600 ................................................................
15:32:43 9664 ................................................................
15:32:44 9728 ................................................................
15:32:46 9792 ................................................................
15:32:48 9856 ................................................................
15:32:49 9920 ................................................................
15:32:50 9984 ................................................................
15:32:52 10048 ................................................................
15:32:53 10112 ................................................................
15:32:54 10176 ................................................................
15:32:55 10240 ................................................................
15:32:56 10304 ................................................................
15:32:57 10368 ................................................................
15:32:58 10432 ................................................................
15:33:00 10496 ................................................................
15:33:02 10560 ................................................................
15:33:03 10624 ................................................................
15:33:04 10688 ................................................................
15:33:06 10752 ................................................................
15:33:07 10816 ................................................................
15:33:08 10880 ................................................................
15:33:09 10944 ................................................................
15:33:10 11008 ................................................................
15:33:11 11072 ................................................................
15:33:12 11136 ................................................................
15:33:14 11200 ................................................................
15:33:15 11264 ................................................................
15:33:16 11328 ................................................................
15:33:17 11392 ................................................................
15:33:18 11456 ................................................................
15:33:19 11520 ................................................................
15:33:20 11584 ................................................................
15:33:22 11648 ................................................................
15:33:23 11712 ................................................................
15:33:24 11776 ................................................................
15:33:25 11840 ................................................................
15:33:26 11904 ................................................................
15:33:27 11968 ................................................................
15:33:28 12032 ................................................................
15:33:29 12096 ................................................................
15:33:30 12160 ................................................................
15:33:31 12224 ................................................................
15:33:33 12288 ................................................................
15:36:00 12352 .
done (915s)
15:36:00
WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them.
15:36:00 306.469: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/validators_sentinel yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/validators_sentinel.test.ts'
15:36:00 328.838: 'f6d0f30efb64f09d noir-projects/scripts/run_test.sh noir-protocol-circuits rollup_lib block_root::block_root_rollup_inputs::tests::correct_blobs_with_non_empty_fields'
15:36:00 346.832: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/data_withholding_slash yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/data_withholding_slash.test.ts'
15:36:00 351.793: '24d076b411d925ca:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes'
15:36:00 353.532: 'ebb2d5bf21766e6e barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness'
15:36:00 378.690: 'ebb2d5bf21766e6e:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing'
15:36:00 384.782: 'ebb2d5bf21766e6e:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing'
15:36:00 401.816: '9945abf2c88d4de8 aztec-up/scripts/run_test.sh amm_flow'
15:36:00 402.822: '55a5c1e5cd3f3ca6:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building'
15:36:00 411.724: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=p2p/src/client/p2p_client.integration.test.ts LOG_LEVEL=debug yarn-project/scripts/run_test.sh p2p/src/client/p2p_client.integration.test.ts'
15:36:00 429.992: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts'
15:36:00 444.477: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts'
15:36:00 484.690: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts'
15:36:00 773.403: '55a5c1e5cd3f3ca6:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full'
15:36:00 Completed run of 12132 tests in 915 seconds.
15:36:00
--- bench all ---
15:36:00 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (
http://ci.aztec-labs.com/0823b1a1021a812f)
15:36:00 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (
http://ci.aztec-labs.com/1605dbe184613332)
15:36:01 barretenberg/cpp 0 .
done (1s)
15:38:08 yarn-project/end-to-end 0 ................................................................
15:40:59 yarn-project/end-to-end 64 ................................................................
15:40:59 yarn-project/end-to-end 128 ................................................................
15:40:59 yarn-project/end-to-end 192 ................................................................
15:40:59 yarn-project/end-to-end 256 ................................................................
15:40:59 yarn-project/end-to-end 320 ................................................................
15:40:59 yarn-project/end-to-end 384 ................................................................
15:40:59 yarn-project/end-to-end 448 ................................................................
15:40:59 yarn-project/end-to-end 512 ................................................................
15:41:01 yarn-project/end-to-end 576 .
done (301s)
15:41:02 Starting parallel run using max 64 cpus...
15:41:02 Test run log id:
http://ci.aztec-labs.com/fd0e3e816980fe12
15:46:15 Completed run of 95 tests in 313 seconds.
15:46:18 Completed 7.6 KiB/7.6 KiB (156.6 KiB/s) with 1 file(s) remaining
upload: ../../../tmp/tmp.e7qfafmKRa to s3://aztec-ci-artifacts/build-cache/bench-28bf32bfc2566751e8baa0a9125a9db20d474bee.tar.gz
15:46:18 Cache upload of bench-28bf32bfc2566751e8baa0a9125a9db20d474bee.tar.gz complete in 1s.
15:46:18 + local code=0
15:46:18 + set +x
15:46:18 dmesg log id:
http://ci.aztec-labs.com/78546e08880a7978
CI run log id:
http://ci.aztec-labs.com/1750432308681
+ code=0
+ case $code in
+ log_ci_run PASSED 1750432308681
+ exit 0
SSH exited with code: 0
Terminating instance: i-03964918eb532cb7d...