Parent Log:
http://ci.aztec-labs.com/f92f0e17b07b44aa
Command: 4cadf2b318302dc1 yarn-project/scripts/run_test.sh prover-client/src/proving_broker/broker_prover_facade.test.ts
Commit:
https://github.com/AztecProtocol/aztec-packages/commit/4bc54c2acb532388aae32aea2765a66709edfdc9
Env: REF_NAME=gh-readonly-queue/next/pr-15249-4873eb2f8189ef2d21a8b7fcfbfebe4dbc0f4145 CURRENT_VERSION=0.87.6 CI_FULL=1
Date: Wed Jun 25 20:09:23 UTC 2025
System: ARCH=amd64 CPUS=128 MEM=493Gi HOSTNAME=pr-15249_amd64_x2-full
Resources: CPU_LIST=0-127 CPUS=2 MEM=8g TIMEOUT=600s
History:
http://ci.aztec-labs.com/list/history_b85794959204d424_next
20:09:29 [20:09:29.052]
INFO:
prover-client:proving-broker Proving Broker started
20:09:29 [20:09:29.064]
INFO:
prover-client:proving-broker New proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:29 [20:09:29.164]
INFO:
prover-client:proving-agent Starting job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY inputsUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22input...
20:09:29 [20:09:29.169]
INFO:
prover-client:proving-agent:job-controller-dc5053f9 Job controller started jobId=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"jobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:29 [20:09:29.172]
INFO:
prover-client:proving-agent Job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY completed outputUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22resul...
20:09:29 [20:09:29.173]
INFO:
prover-client:proving-broker Proving job complete id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY totalAttempts=1
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:30 [20:09:30.069]
INFO:
prover-client:proving-broker Proving Broker started
20:09:30 [20:09:30.078]
INFO:
prover-client:proving-broker New proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:30 [20:09:30.172]
INFO:
prover-client:proving-agent Starting job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY inputsUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22input...
20:09:30 [20:09:30.174]
INFO:
prover-client:proving-agent:job-controller-3b4a25b9 Job controller started jobId=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"jobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:30 [20:09:30.177]
INFO:
prover-client:proving-agent Job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY completed outputUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22resul...
20:09:30 [20:09:30.178]
INFO:
prover-client:proving-broker Proving job complete id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY totalAttempts=1
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:31 [20:09:31.075]
WARN:
prover-client:proving-broker Cached proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42. Not enqueuing again
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:32 [20:09:32.350]
INFO:
prover-client:proving-broker Proving Broker started
20:09:32 [20:09:32.354]
INFO:
prover-client:proving-broker New proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:32 [20:09:32.452]
INFO:
prover-client:proving-agent Starting job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY inputsUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22input...
20:09:32 [20:09:32.453]
INFO:
prover-client:proving-agent:job-controller-36d23dbc Job controller started jobId=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"jobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:32 [20:09:32.467]
ERROR:
prover-client:proving-agent Job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY failed err=TEST ERROR retry=false: Error: TEST ERROR
20:09:32 at Object.<anonymous> (/home/aztec-dev/aztec-packages/yarn-project/prover-client/src/proving_broker/broker_prover_facade.test.ts:124:26)
20:09:32 at runNextTicks (node:internal/process/task_queues:65:5)
20:09:32 at listOnTimeout (node:internal/timers:549:9)
20:09:32 at processTimers (node:internal/timers:523:7)
20:09:32 [20:09:32.468]
INFO:
prover-client:proving-broker Marking proving job as failed id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY totalAttempts=1 err=TEST ERROR
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:33 [20:09:33.351]
ERROR:
prover-client:broker-circuit-prover-facade Resolving proving job with error id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: 'TEST ERROR'
20:09:33 [20:09:33.352]
ERROR:
prover-client:broker-circuit-prover-facade Job errored with 'TEST ERROR' id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: {
20:09:33 provingJobId: '42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb',
20:09:33 provingJobType: 'BASE_PARITY'
20:09:33 }
20:09:33 [20:09:33.352]
INFO:
prover-client:broker-circuit-prover-facade Stored proof inputs for failed job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY at data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22inputs%22%3A%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAA%3D%3D%22%7D
{"id":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb","type":9,"uri":"data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22inputs%22%3A%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAA%3D%3D%22%7D"}
20:09:33 [20:09:33.355]
WARN:
prover-client:proving-broker Cached proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42. Not enqueuing again
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.355]
ERROR:
prover-client:broker-circuit-prover-facade Resolving proving job with error id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: 'TEST ERROR'
20:09:34 [20:09:34.356]
ERROR:
prover-client:broker-circuit-prover-facade Job errored with 'TEST ERROR' id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: {
20:09:34 provingJobId: '42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb',
20:09:34 provingJobType: 'BASE_PARITY'
20:09:34 }
20:09:34 [20:09:34.357]
INFO:
prover-client:broker-circuit-prover-facade Stored proof inputs for failed job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY at data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22inputs%22%3A%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAA%3D%3D%22%7D
{"id":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb","type":9,"uri":"data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22inputs%22%3A%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAA%3D%3D%22%7D"}
20:09:34 [20:09:34.362]
INFO:
prover-client:proving-broker Proving Broker started
20:09:34 [20:09:34.364]
INFO:
prover-client:proving-broker New proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.463]
INFO:
prover-client:proving-agent Starting job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY inputsUri=data:application/json;charset=utf-8,%7B%22type%22%3A9%2C%22input...
20:09:34 [20:09:34.464]
INFO:
prover-client:proving-agent:job-controller-dd383804 Job controller started jobId=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"jobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.466]
INFO:
prover-client:proving-broker Cancelling job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.466]
INFO:
prover-client:proving-broker Marking proving job as failed id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY totalAttempts=1 err=Aborted
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.565]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.666]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.767]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.869]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:34 [20:09:34.970]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.071]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.172]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.273]
WARN:
prover-client:proving-broker Proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb has already been completed
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.363]
ERROR:
prover-client:broker-circuit-prover-facade Resolving proving job with error id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: 'Aborted'
20:09:35 [20:09:35.364]
ERROR:
prover-client:broker-circuit-prover-facade Job errored with 'Aborted' id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: {
20:09:35 provingJobId: '42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb',
20:09:35 provingJobType: 'BASE_PARITY'
20:09:35 }
20:09:35 [20:09:35.365]
WARN:
prover-client:proving-agent:job-controller-dd383804 Aborted job controller for jobId=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb
{"jobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.366]
INFO:
prover-client:proving-broker Proving Broker started
20:09:35 [20:09:35.367]
INFO:
prover-client:proving-broker New proving job id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb epochNumber=42
{"provingJobId":"42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb"}
20:09:35 [20:09:35.368]
ERROR:
prover-client:broker-circuit-prover-facade Job errored with 'Broker facade stopped' id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: {
20:09:35 provingJobId: '42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb',
20:09:35 provingJobType: 'BASE_PARITY'
20:09:35 }
20:09:35 [20:09:35.368]
WARN:
prover-client:broker-circuit-prover-facade Running promise was not started
20:09:35 [20:09:35.369]
INFO:
prover-client:proving-broker Proving Broker started
20:09:35 [20:09:35.471]
ERROR:
prover-client:broker-circuit-prover-facade Job errored with 'Broker facade stopped' id=42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb type=BASE_PARITY: {
20:09:35 provingJobId: '42:BASE_PARITY:6fc11753565e9d635f3028b7106c009c3a16b7b239b7eaffb427ed8326c1c9fb',
20:09:35 provingJobType: 'BASE_PARITY'
20:09:35 }
20:09:35 [20:09:35.482]
WARN:
prover-client:broker-circuit-prover-facade Running promise was not started
20:09:35
PASS src/proving_broker/broker_prover_facade.test.ts (
10.586 s)
20:09:35 BrokerCircuitProverFacade
20:09:35
✓ sends jobs to the broker (1019 ms)
20:09:35
✓ handles multiple calls for the same job (2281 ms)
20:09:35
✓ handles proof errors (2012 ms)
20:09:35
✓ handles aborts (1003 ms)
20:09:35
✓ rejects jobs when the facade is stopped (4 ms)
20:09:35
✓ handles stopping while sending a proof to the broker (114 ms)
20:09:35
20:09:35
Test Suites: 1 passed, 1 total
20:09:35
Tests: 6 passed, 6 total
20:09:35
Snapshots: 0 total
20:09:35
Time: 10.676 s
20:09:35
Ran all test suites matching prover-client/src/proving_broker/broker_prover_facade.test.ts
.
20:09:35
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?