Parent Log:
http://ci.aztec-labs.com/15782013234
Command: bootstrap_ec2 './bootstrap.sh ci-full'
Date: Fri Jun 20 15:46:00 UTC 2025
Status: 0
Took: 2106s
--- request build instance ---
Requesting m6a.32xlarge spot instance (name: pr-15154_amd64_x1-full) (type: m6a.32xlarge) (ami: ami-051d99fd9abf53087) (bid: 5.5424)...
Waiting for instance id for spot request: sir-6jyq71pj...
Waiting for SSH at 52.15.88.219...
Stdout is not a tty, running in background...
Loading CRS into tmpfs...
Done in 14 seconds.
Stdout is not a tty, starting build container in background...
Starting devbox...
From
https://github.com/aztecprotocol/aztec-packages
* branch 28bf32bfc2566751e8baa0a9125a9db20d474bee -> FETCH_HEAD
HEAD is now at 28bf32b feat: auto-update exported metrics (#15154)
Switched to a new branch 'gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4'
+ run
+ ci3/add_timestamps
+ DUP=1
+ ci3/cache_log 'CI run' 1750432311024
CI run log id:
http://ci.aztec-labs.com/1750432311024
15:11:51 + echo 'env: REF_NAME=gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4 COMMIT_HASH=28bf32bfc2566751e8baa0a9125a9db20d474bee CURRENT_VERSION=0.87.6 TARGET_BRANCH=next'
15:11:51 env: REF_NAME=gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4 COMMIT_HASH=28bf32bfc2566751e8baa0a9125a9db20d474bee CURRENT_VERSION=0.87.6 TARGET_BRANCH=next
15:11:51 + semver check gh-readonly-queue/next/pr-15154-9071986bfe3af58c70d2c80c10f523e22bfe4cb4
15:11:51 + cpu_pid=415
15:11:51 + mpstat 2
15:11:51 + mem_pid=417
15:11:51 + set +e
15:11:51 + set -x
15:11:51 + ./bootstrap.sh ci-full
15:11:51 + vmstat -w -S M 2
15:11:51 ++ cache_log cpufile
15:11:51 ++ cache_log memfile
15:11:51 cpufile log id:
http://ci.aztec-labs.com/4fd8d6b5d3823506
15:11:51 memfile log id:
http://ci.aztec-labs.com/21576685dac0f71f
15:11:59
--- pull submodules ---
15:11:59 Executing: git submodule update --init --recursive (
http://ci.aztec-labs.com/1d2550e8e1612ee9)
15:12:08 0 .
done (9s)
15:12:10
--- noir build ---
15:12:10 Executing: retry install_deps (
http://ci.aztec-labs.com/5444def7a2e4a73c)
15:12:14 0 .
done (4s)
15:12:14 build_native Executing: build_native (
http://ci.aztec-labs.com/7bc8c4b8d6545a3f)
15:12:14 build_packages Executing: build_packages (
http://ci.aztec-labs.com/1a018fe0d4cc0af4)
15:12:15 build_native 0 .
done (1s)
15:12:20 build_packages 0 .
done (6s)
15:12:22
--- bb cpp build ---
15:12:22 build_wasm_threads Executing: build_wasm_threads (
http://ci.aztec-labs.com/e5b0fa8490f2889d)
15:12:22 build_nodejs_module Executing: build_nodejs_module (
http://ci.aztec-labs.com/683b75f6a57137b3)
15:12:22 build_native Executing: build_native (
http://ci.aztec-labs.com/b7122480380983a6)
15:12:22 build_wasm Executing: build_wasm (
http://ci.aztec-labs.com/a094276b6c7d1ffa)
15:12:22 build_gcc_syntax_check_only Executing: build_gcc_syntax_check_only (
http://ci.aztec-labs.com/59edea920f65615e)
15:12:22 build_fuzzing_syntax_check_only Executing: build_fuzzing_syntax_check_only (
http://ci.aztec-labs.com/3f66a3b9be700861)
15:12:22 build_asan_fast Executing: build_asan_fast (
http://ci.aztec-labs.com/cc6a5e58e6c7ba1f)
15:12:22 build_darwin Executing: build_darwin (
http://ci.aztec-labs.com/d1bcb29012e2907c)
15:12:22 build_fuzzing_syntax_check_only 0 .
done (0s)
15:12:22 build_gcc_syntax_check_only 0 .
done (0s)
15:12:22 build_wasm 0 .
done (0s)
15:12:22 build_darwin 0 .
done (0s)
15:12:22 build_wasm_threads 0 .
done (0s)
15:12:23 build_nodejs_module 0 .
done (1s)
15:12:23 build_native 0 .
done (1s)
15:12:24 build_asan_fast 0 .
done (2s)
15:12:28
--- bb.js build ---
15:12:30 Cache download and extraction of node-modules-e85320bda7d59d54.zst complete in 1s.
15:12:30 Cache download and extraction of bb.js-ee60cf044fbe25d5.tar.gz complete in 0s.
15:12:31
--- acir_tests build ---
15:12:32 Cache download and extraction of barretenberg-acir-tests-95b8ad4482daac2e.tar.gz complete in 1s.
15:12:33 Cache download and extraction of node-modules-fec19dc5fe5a76d5.zst complete in 1s.
15:12:33 browser-test-app Executing: yarn build (
http://ci.aztec-labs.com/2dc94cb83576259c)
15:12:33 bbjs-test Executing: yarn build (
http://ci.aztec-labs.com/9957652af98f7962)
15:12:36 bbjs-test 0 .
done (3s)
15:12:38 browser-test-app 0 .
done (5s)
15:12:38
--- avm-transpiler build ---
15:12:38 Cache download and extraction of avm-transpiler-6867cd9272ea7ebd.tar.gz complete in 0s.
15:12:38
--- noir-projects build ---
15:12:38 Executing: prep (
http://ci.aztec-labs.com/b6a2075496f54f99)
15:12:41 0 .
done (3s)
15:12:42 mock-protocol-circuits Executing: ./mock-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/270d014f53dcf410)
15:12:42 noir-protocol-circuits Executing: ./noir-protocol-circuits/bootstrap.sh (
http://ci.aztec-labs.com/770c1e0c048ff84e)
15:12:42 noir-contracts Executing: ./noir-contracts/bootstrap.sh (
http://ci.aztec-labs.com/79018e496c61cf8c)
15:12:42 aztec-nr Executing: ./aztec-nr/bootstrap.sh (
http://ci.aztec-labs.com/7f588d6633efc7a2)
15:12:44 noir-contracts 0 ................................................................
15:12:44 noir-contracts 64 ................................................................
15:12:44 noir-contracts 128 ................................................................
15:12:44 noir-contracts 192 ................................................................
15:12:44 noir-contracts 256 ................................................................
15:12:45 noir-contracts 320 ................................................................
15:12:45 noir-contracts 384 ................................................................
15:12:45 noir-contracts 448 ................................................................
15:12:45 noir-contracts 512 ................................................................
15:12:45 noir-contracts 576 ................................................................
15:12:45 noir-contracts 640 ................................................................
15:12:46 noir-contracts 704 ................................................................
15:12:46 noir-contracts 768 ................................................................
15:12:46 noir-contracts 832 ................................................................
15:12:46 noir-contracts 896 ................................................................
15:12:46 noir-contracts 960 ................................................................
15:12:47 noir-contracts 1024 ................................................................
15:12:47 noir-contracts 1088 ................................................................
15:12:47 noir-protocol-circuits 0 ................................................................
15:12:47 noir-protocol-circuits 64 ................................................................
15:12:47 noir-contracts 1152 ................................................................
15:12:47 noir-contracts 1216 ................................................................
15:12:48 noir-contracts 1280 ................................................................
15:12:48 noir-contracts 1344 ................................................................
15:12:48 noir-contracts 1408 ................................................................
15:12:48 noir-protocol-circuits 128 ................................................................
15:12:48 noir-contracts 1472 ................................................................
15:12:48 noir-contracts 1536 ................................................................
15:12:48 noir-contracts 1600 .
done (6s)
15:12:49 noir-protocol-circuits 192 ................................................................
15:12:49 noir-protocol-circuits 256 ................................................................
15:12:50 aztec-nr 0 .
done (8s)
15:12:50 noir-protocol-circuits 320 ................................................................
15:12:51 mock-protocol-circuits 0 ................................................................
15:12:52 noir-protocol-circuits 384 ................................................................
15:12:53 noir-protocol-circuits 448 ................................................................
15:12:55 mock-protocol-circuits 64 ................................................................
15:15:21 mock-protocol-circuits 128 .
done (159s)
15:15:26 noir-protocol-circuits 512 ................................................................
15:15:26 noir-protocol-circuits 576 ................................................................
15:15:26 noir-protocol-circuits 640 .
done (164s)
15:15:27
--- l1-contracts build ---
15:15:27 Cache download and extraction of l1-contracts-52d0554817b52bf7.tar.gz complete in 0s.
15:15:27
--- yarn-project build ---
15:15:27 Executing: ./bootstrap.sh clean-lite (
http://ci.aztec-labs.com/1b7599128ad75161)
15:15:27 0 .
done (0s)
15:15:32 Cache download and extraction of node-modules-34bec2a54a394404.zst complete in 4s.
15:15:32 Executing: compile_all (
http://ci.aztec-labs.com/699bbe6e9e2d76ca)
15:15:43 0 ................................................................
15:17:56 64 .
done (144s)
15:17:56 playground/bootstrap.sh Executing: playground/bootstrap.sh (
http://ci.aztec-labs.com/5f5b186ab97677c8)
15:17:56 boxes/bootstrap.sh Executing: boxes/bootstrap.sh (
http://ci.aztec-labs.com/3ace687289dd7a3b)
15:17:56 docs/bootstrap.sh Executing: docs/bootstrap.sh (
http://ci.aztec-labs.com/2513a8b28d10e795)
15:17:56 release-image/bootstrap.sh Executing: release-image/bootstrap.sh (
http://ci.aztec-labs.com/f475eb9f35101cec)
15:17:56 spartan/bootstrap.sh Executing: spartan/bootstrap.sh (
http://ci.aztec-labs.com/94db2e3e430b5e7d)
15:17:56 aztec-up/bootstrap.sh Executing: aztec-up/bootstrap.sh (
http://ci.aztec-labs.com/c23e7aaa9f169b06)
15:17:58 aztec-up/bootstrap.sh 0 .
done (2s)
15:18:00 spartan/bootstrap.sh 0 .
done (4s)
15:18:05 docs/bootstrap.sh 0 .
done (9s)
15:18:28 release-image/bootstrap.sh 0 .
done (32s)
15:18:39 boxes/bootstrap.sh 0 .
done (43s)
15:18:52 playground/bootstrap.sh 0 .
done (56s)
15:18:52
--- test all ---
15:18:52 Waiting for TXE's to start...
15:18:55 Gathering tests to run...
15:20:34 Gathered 12132 tests.
15:20:34 Starting parallel run with max 64 jobs...
15:20:34 Executing: parallel -j64 --memsuspend 64G --line-buffer --joblog joblog.txt --halt now,fail=1 'run_test_cmd {}' (
http://ci.aztec-labs.com/1f75c1257e3b33bd)
15:21:05 0 ................................................................
15:22:14 64 ................................................................
15:23:18 128 ................................................................
15:24:19 192 ................................................................
15:24:43 256 ................................................................
15:24:54 320 ................................................................
15:25:13 384 ................................................................
15:25:30 448 ................................................................
15:25:37 512 ................................................................
15:25:44 576 ................................................................
15:25:56 640 ................................................................
15:26:04 704 ................................................................
15:26:11 768 ................................................................
15:26:16 832 ................................................................
15:26:25 896 ................................................................
15:26:32 960 ................................................................
15:26:37 1024 ................................................................
15:26:40 1088 ................................................................
15:26:50 1152 ................................................................
15:26:58 1216 ................................................................
15:27:00 1280 ................................................................
15:27:04 1344 ................................................................
15:27:08 1408 ................................................................
15:27:11 1472 ................................................................
15:27:15 1536 ................................................................
15:27:22 1600 ................................................................
15:27:32 1664 ................................................................
15:27:47 1728 ................................................................
15:27:52 1792 ................................................................
15:27:57 1856 ................................................................
15:28:00 1920 ................................................................
15:28:02 1984 ................................................................
15:28:04 2048 ................................................................
15:28:06 2112 ................................................................
15:28:07 2176 ................................................................
15:28:09 2240 ................................................................
15:28:11 2304 ................................................................
15:28:13 2368 ................................................................
15:28:14 2432 ................................................................
15:28:16 2496 ................................................................
15:28:17 2560 ................................................................
15:28:27 2624 ................................................................
15:28:29 2688 ................................................................
15:28:31 2752 ................................................................
15:28:32 2816 ................................................................
15:28:34 2880 ................................................................
15:28:37 2944 ................................................................
15:28:59 3008 ................................................................
15:29:02 3072 ................................................................
15:29:05 3136 ................................................................
15:29:06 3200 ................................................................
15:29:08 3264 ................................................................
15:29:10 3328 ................................................................
15:29:11 3392 ................................................................
15:29:13 3456 ................................................................
15:29:17 3520 ................................................................
15:29:19 3584 ................................................................
15:29:21 3648 ................................................................
15:29:23 3712 ................................................................
15:29:24 3776 ................................................................
15:29:25 3840 ................................................................
15:29:27 3904 ................................................................
15:29:28 3968 ................................................................
15:29:29 4032 ................................................................
15:29:31 4096 ................................................................
15:29:33 4160 ................................................................
15:29:38 4224 ................................................................
15:29:42 4288 ................................................................
15:29:45 4352 ................................................................
15:29:48 4416 ................................................................
15:29:51 4480 ................................................................
15:29:53 4544 ................................................................
15:29:55 4608 ................................................................
15:29:58 4672 ................................................................
15:30:01 4736 ................................................................
15:30:03 4800 ................................................................
15:30:05 4864 ................................................................
15:30:07 4928 ................................................................
15:30:09 4992 ................................................................
15:30:11 5056 ................................................................
15:30:13 5120 ................................................................
15:30:15 5184 ................................................................
15:30:17 5248 ................................................................
15:30:18 5312 ................................................................
15:30:19 5376 ................................................................
15:30:21 5440 ................................................................
15:30:24 5504 ................................................................
15:30:27 5568 ................................................................
15:30:29 5632 ................................................................
15:30:31 5696 ................................................................
15:30:34 5760 ................................................................
15:30:36 5824 ................................................................
15:30:38 5888 ................................................................
15:30:40 5952 ................................................................
15:30:42 6016 ................................................................
15:30:44 6080 ................................................................
15:30:47 6144 ................................................................
15:30:50 6208 ................................................................
15:30:52 6272 ................................................................
15:30:54 6336 ................................................................
15:30:57 6400 ................................................................
15:31:00 6464 ................................................................
15:31:03 6528 ................................................................
15:31:05 6592 ................................................................
15:31:08 6656 ................................................................
15:31:11 6720 ................................................................
15:31:13 6784 ................................................................
15:31:15 6848 ................................................................
15:31:17 6912 ................................................................
15:31:19 6976 ................................................................
15:31:21 7040 ................................................................
15:31:23 7104 ................................................................
15:31:25 7168 ................................................................
15:31:27 7232 ................................................................
15:31:28 7296 ................................................................
15:31:29 7360 ................................................................
15:31:30 7424 ................................................................
15:31:32 7488 ................................................................
15:31:33 7552 ................................................................
15:31:34 7616 ................................................................
15:31:36 7680 ................................................................
15:31:37 7744 ................................................................
15:31:38 7808 ................................................................
15:31:39 7872 ................................................................
15:31:41 7936 ................................................................
15:31:42 8000 ................................................................
15:31:43 8064 ................................................................
15:31:44 8128 ................................................................
15:31:45 8192 ................................................................
15:31:47 8256 ................................................................
15:31:48 8320 ................................................................
15:31:49 8384 ................................................................
15:31:51 8448 ................................................................
15:31:52 8512 ................................................................
15:31:53 8576 ................................................................
15:31:54 8640 ................................................................
15:31:55 8704 ................................................................
15:31:56 8768 ................................................................
15:31:58 8832 ................................................................
15:31:59 8896 ................................................................
15:32:00 8960 ................................................................
15:32:02 9024 ................................................................
15:32:03 9088 ................................................................
15:32:04 9152 ................................................................
15:32:06 9216 ................................................................
15:32:07 9280 ................................................................
15:32:09 9344 ................................................................
15:32:10 9408 ................................................................
15:32:11 9472 ................................................................
15:32:13 9536 ................................................................
15:32:14 9600 ................................................................
15:32:16 9664 ................................................................
15:32:18 9728 ................................................................
15:32:20 9792 ................................................................
15:32:21 9856 ................................................................
15:32:22 9920 ................................................................
15:32:23 9984 ................................................................
15:32:24 10048 ................................................................
15:32:26 10112 ................................................................
15:32:27 10176 ................................................................
15:32:28 10240 ................................................................
15:32:29 10304 ................................................................
15:32:30 10368 ................................................................
15:32:31 10432 ................................................................
15:32:33 10496 ................................................................
15:32:34 10560 ................................................................
15:32:36 10624 ................................................................
15:32:37 10688 ................................................................
15:32:38 10752 ................................................................
15:32:40 10816 ................................................................
15:32:40 10880 ................................................................
15:32:42 10944 ................................................................
15:32:43 11008 ................................................................
15:32:44 11072 ................................................................
15:32:45 11136 ................................................................
15:32:46 11200 ................................................................
15:32:47 11264 ................................................................
15:32:48 11328 ................................................................
15:32:49 11392 ................................................................
15:32:51 11456 ................................................................
15:32:51 11520 ................................................................
15:32:52 11584 ................................................................
15:32:54 11648 ................................................................
15:32:55 11712 ................................................................
15:32:56 11776 ................................................................
15:32:57 11840 ................................................................
15:32:58 11904 ................................................................
15:32:59 11968 ................................................................
15:33:00 12032 ................................................................
15:33:01 12096 ................................................................
15:33:02 12160 ................................................................
15:33:03 12224 ................................................................
15:33:05 12288 ................................................................
15:35:37 12352 .
done (903s)
15:35:37
WARNING: The following tests exceed 5 minute runtimes. Break them up or rethink them.
15:35:37 306.967: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/validators_sentinel yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/validators_sentinel.test.ts'
15:35:37 315.934: 'f6d0f30efb64f09d noir-projects/scripts/run_test.sh noir-protocol-circuits rollup_lib block_root::block_root_rollup_inputs::tests::correct_blobs_with_non_empty_fields'
15:35:37 341.040: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/data_withholding_slash yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/data_withholding_slash.test.ts'
15:35:37 342.815: 'ebb2d5bf21766e6e barretenberg/cpp/scripts/run_test.sh dsl_tests AcirAvm2RecursionConstraint.TestGenerateVKFromConstraintsWithoutWitness'
15:35:37 346.251: '24d076b411d925ca:ONLY_TERM_PARENT=1 BOX=vanilla BROWSER=* run_compose_test vanilla-all-browsers box boxes'
15:35:37 365.519: 'ebb2d5bf21766e6e:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityFailing'
15:35:37 372.287: '9945abf2c88d4de8 aztec-up/scripts/run_test.sh amm_flow'
15:35:37 383.528: 'ebb2d5bf21766e6e:CPUS=4:MEM=8g barretenberg/cpp/scripts/run_test.sh client_ivc_tests ClientIVCKernelCapacity.MaxCapacityPassing'
15:35:37 390.838: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=p2p/src/client/p2p_client.integration.test.ts LOG_LEVEL=debug yarn-project/scripts/run_test.sh p2p/src/client/p2p_client.integration.test.ts'
15:35:37 418.226: '55a5c1e5cd3f3ca6:ISOLATE=1:TIMEOUT=15m:NAME=e2e_block_building yarn-project/end-to-end/scripts/run_test.sh simple e2e_block_building'
15:35:37 430.270: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_p2p/reex yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/reex.test.ts'
15:35:37 443.653: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=e2e_l1_with_wall_time yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_l1_with_wall_time.test.ts'
15:35:37 450.782: '55a5c1e5cd3f3ca6:ISOLATE=1:NAME=prover-client/src/test/bb_prover_full_rollup.test.ts:CPUS=16:MEM=96g LOG_LEVEL=verbose yarn-project/scripts/run_test.sh prover-client/src/test/bb_prover_full_rollup.test.ts'
15:35:37 775.078: '55a5c1e5cd3f3ca6:ISOLATE=1:TIMEOUT=15m:CPUS=16:MEM=96g:NAME=e2e_prover_full_real yarn-project/end-to-end/scripts/run_test.sh simple e2e_prover/full'
15:35:37 Completed run of 12132 tests in 903 seconds.
15:35:37
--- bench all ---
15:35:37 barretenberg/cpp Executing: barretenberg/cpp/bootstrap.sh build_bench (
http://ci.aztec-labs.com/110032e0c158f206)
15:35:37 yarn-project/end-to-end Executing: yarn-project/end-to-end/bootstrap.sh build_bench (
http://ci.aztec-labs.com/a25da108ee576ecc)
15:35:38 barretenberg/cpp 0 .
done (1s)
15:37:43 yarn-project/end-to-end 0 ................................................................
15:40:30 yarn-project/end-to-end 64 ................................................................
15:40:30 yarn-project/end-to-end 128 ................................................................
15:40:30 yarn-project/end-to-end 192 ................................................................
15:40:30 yarn-project/end-to-end 256 ................................................................
15:40:30 yarn-project/end-to-end 320 ................................................................
15:40:30 yarn-project/end-to-end 384 ................................................................
15:40:30 yarn-project/end-to-end 448 ................................................................
15:40:30 yarn-project/end-to-end 512 ................................................................
15:40:30 yarn-project/end-to-end 576 ................................................................
15:40:33 yarn-project/end-to-end 640 .
done (296s)
15:40:34 Starting parallel run using max 64 cpus...
15:40:35 Test run log id:
http://ci.aztec-labs.com/545d04cf8e37e531
15:45:47 Completed run of 95 tests in 313 seconds.
15:45:50 Completed 7.6 KiB/7.6 KiB (161.7 KiB/s) with 1 file(s) remaining
upload: ../../../tmp/tmp.nn1ygRaYkr to s3://aztec-ci-artifacts/build-cache/bench-28bf32bfc2566751e8baa0a9125a9db20d474bee.tar.gz
15:45:50 Cache upload of bench-28bf32bfc2566751e8baa0a9125a9db20d474bee.tar.gz complete in 1s.
15:45:50 + local code=0
15:45:50 + set +x
15:45:50 dmesg log id:
http://ci.aztec-labs.com/70ed9931a551d3bd
CI run log id:
http://ci.aztec-labs.com/1750432311024
+ code=0
+ case $code in
+ log_ci_run PASSED 1750432311024
+ exit 0
SSH exited with code: 0
Terminating instance: i-0ca5594233b6c7e44...