Skip to content

refactor: unify contract compilation pipeline via bb aztec_process#22590

Merged
benesjan merged 1 commit intomerge-train/fairiesfrom
be/unify-contract-compilation
Apr 17, 2026
Merged

refactor: unify contract compilation pipeline via bb aztec_process#22590
benesjan merged 1 commit intomerge-train/fairiesfrom
be/unify-contract-compilation

Conversation

@benesjan
Copy link
Copy Markdown
Contributor

@benesjan benesjan commented Apr 16, 2026

When working on another PR of mine I realized that we are having a problem and that is that we have 2 paths by which contracts get compiled:

  1. one is via aztec-packages bootstrap,
  2. another is via aztec compile

This started to become problematic as our contract compilation pipeline was getting more complicated:

  1. nargo compile,
  2. transpile,
  3. strip prefixes,
  4. and with that PR also inject aztec version

This meant that the duplication started to have real cost. In this PR I clean this up a bit by moving the prefix stripping into the bb aztec_process command. This results that the compilation pipeline is simply:

  1. nargo compile,
  2. bb aztec_process

Given that it's just 2 commands it is fine to have it be duplicated between bootstrap.sh and aztec compile. Once this is merged I will try to refactor #22550 such that the version is also injected only in bb aztec_process.

AI Summary

  • Moved __aztec_nr_internals__ prefix stripping into bb aztec_process (C++), which already handles transpilation and VK generation
  • Simplified the bootstrap compile function to just call nargo compile + bb aztec_process, removing ~50 lines of manual VK generation shell code
  • Removed the separate stripInternalPrefixes() TypeScript function from the CLI path
  • Deleted the now-unused strip_aztec_nr_prefix.sh script
  • Updated protocol-fuzzer scripts and docs to reflect the unified pipeline

Both compilation paths (bootstrap and aztec compile) now go through bb aztec_process for all post-nargo processing, eliminating the duplication.

Both the bootstrap scripts and the `aztec compile` CLI independently
called the transpiler and stripped `__aztec_nr_internals__` prefixes
from contract artifacts. This duplication was brittle and a source of
tech debt.

Move prefix stripping into `bb aztec_process` (which already handles
transpilation and VK generation), then simplify both callers to use
that single command. This eliminates the manual VK generation shell
pipeline (~50 lines of bash), the separate strip script, and the
TypeScript stripInternalPrefixes function.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@benesjan benesjan added ci-full Run all master checks. ci-draft Run CI on draft PRs. labels Apr 16, 2026
@benesjan benesjan marked this pull request as ready for review April 16, 2026 05:17
@benesjan benesjan requested a review from a team as a code owner April 16, 2026 05:17
@benesjan benesjan requested review from Thunkar and mverzilli April 16, 2026 05:22
@benesjan benesjan requested review from Thunkar and nchamo and removed request for Thunkar April 16, 2026 13:31
# Set common flags for parallel.
export PARALLEL_FLAGS="-j${PARALLELISM:-16} --halt now,fail=1 --memsuspend $(memsuspend_limit)"

# This computes a vk and adds it to the input function json if it's private, else returns same input.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a lot of bootstrap code removed. Have we lost per-vk CI caches? will this impact bootstrap performance?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No meaningful perf loss — likely a slight improvement.

The old per-VK cache was actually redundant with the outer contract cache:

Old flow — two caches, both keyed by $contract_hash:

  • Outer contract-$contract_hash.tar.gz: compiled+transpiled JSON without VKs
  • Inner vk-$contract_hash-$funchash.tar.gz: one per private function

The per-VK key included $contract_hash as a namespace, so it could only hit when $contract_hash matched a prior run — i.e., exactly the same scenarios where the outer cache also hit. Cross-contract_hash VK reuse was never possible. The per-VK cache was only necessary because the outer cache didn't contain VKs.

New flow — one cache, same key:

  • contract-$contract_hash.tar.gz now contains the fully processed JSON including VKs (produced in one shot by bb aztec_process).

Perf comparison:

  • Cold build (contract_hash changed): equivalent work in both. New path uses C++ threads with nested parallelism in generate_vks_for_functions instead of GNU parallel shelling out per function — likely a bit faster with less overhead.
  • Warm build: strictly faster — one tarball download instead of 1 + N.
  • Bonus: there's a new local VK cache at ~/.bb/$BB_VERSION/vk_cache/<bytecode_hash>.vk keyed on pure bytecode hash (no contract_hash namespace), giving dev-machine reuse across contracts that the old flow didn't have.

The only theoretical regression would be if the old per-VK cache were keyed on just $funchash (enabling cross-contract_hash reuse) — but it wasn't, so nothing's lost.

Copy link
Copy Markdown
Contributor

@mverzilli mverzilli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though please note I currently have very little context on this code

@benesjan benesjan merged commit 9c8bb7b into merge-train/fairies Apr 17, 2026
59 checks passed
@benesjan benesjan deleted the be/unify-contract-compilation branch April 17, 2026 10:15
@AztecBot
Copy link
Copy Markdown
Collaborator

❌ Failed to cherry-pick to v4-next due to conflicts. (🤖) View backport run.

AztecBot pushed a commit that referenced this pull request Apr 17, 2026
This commit records the raw output of git cherry-pick 9c8bb7b for PR #22590
(refactor: unify contract compilation pipeline via bb aztec_process).

Conflicts left intact in this commit:
- noir-projects/noir-contracts/bootstrap.sh — content conflict
- noir-projects/protocol-fuzzer/SANDBOX_INSTRUCTIONS.md — modify/delete (deleted in HEAD)
- noir-projects/protocol-fuzzer/setup-nightly-sandbox.sh — modify/delete (deleted in HEAD)

Subsequent commits resolve the conflicts and fix the build.
AztecBot added a commit that referenced this pull request Apr 17, 2026
- noir-contracts/bootstrap.sh: accept incoming removal of the legacy
  process_function VK-generation block. 'bb aztec_process' now handles
  VK generation directly, so the tmp_dir / jq / parallel plumbing that
  was still present on v4-next is no longer needed.
- protocol-fuzzer/SANDBOX_INSTRUCTIONS.md and
  protocol-fuzzer/setup-nightly-sandbox.sh: drop. These files were
  introduced in a commit that landed on next after v4-next diverged,
  so they never existed on v4-next. PR #22590 only touched them to
  stay consistent with that later work; since v4-next doesn't have
  them, keep them absent.
AztecBot added a commit that referenced this pull request Apr 17, 2026
PR #22590 removed 'mkdir -p $tmp_dir' from bootstrap.sh's build function.
On next that's fine because next's bootstrap.sh has no pinned-contracts
logic. v4-next however still extracts pinned-protocol-contracts.tar.gz
right after 'rm -rf target', so without the implicit target/ creation
the tar step fails with:

  tar: target: Cannot open: No such file or directory

Add an explicit 'mkdir -p target' after the rm -rf to restore the
v4-next build.
benesjan added a commit that referenced this pull request Apr 17, 2026
…ackport #22590) (#22634)

Backport of #22590
to v4-next.

## Original PR summary

Moves `__aztec_nr_internals__` prefix stripping into `bb aztec_process`
(C++), so contract compilation collapses from four steps (`nargo
compile` → `transpile` → `strip prefixes` → `write_vk`) to two (`nargo
compile` → `bb aztec_process`). This removes the duplicated pipeline
between `noir-projects/noir-contracts/bootstrap.sh` and `aztec compile`
(CLI), deletes `strip_aztec_nr_prefix.sh`, and removes the TypeScript
`stripInternalPrefixes()` helper.

## Backport structure (3 commits)

1. **`chore: cherry-pick PR #22590 (conflicts unresolved)`** — raw `git
cherry-pick` output with conflict markers, preserved for review.
2. **`fix: resolve cherry-pick conflicts for PR #22590`** — resolves
only the conflict markers.
3. No build-fix commit was needed — bash syntax, grep for dangling
references, and the already-present prefix-stripping logic in
`aztec_process.cpp` all check out.

## Conflicts resolved

- **`noir-projects/noir-contracts/bootstrap.sh`** — content conflict.
v4-next still had the legacy `process_function` block plus `tmp_dir` /
`jq` / `parallel` plumbing for VK generation. Accepted the incoming PR's
removal since `bb aztec_process` now handles VK generation directly.
- **`noir-projects/protocol-fuzzer/SANDBOX_INSTRUCTIONS.md`** and
**`noir-projects/protocol-fuzzer/setup-nightly-sandbox.sh`** —
modify/delete. These files were introduced on `next` after v4-next
diverged (commit `5b0c0e166e`, "feat(fuzz): protocol fuzzer with bridge
server and parallel batching") and never existed on v4-next. PR #22590
only touched them to stay consistent with that later work, so they're
dropped here.

## Diff vs. original PR

| | Original PR | This backport |
|---|---|---|
| Files | 10 | 8 |
| Insertions | 42 | 38 |
| Deletions | 160 | 142 |

The size delta is entirely from the two protocol-fuzzer files that don't
exist on v4-next.

## Verification

- `bash -n` on all modified shell scripts: OK
- No dangling references to `strip_aztec_nr_prefix`,
`STRIP_AZTEC_NR_PREFIX`, `stripInternalPrefixes`, or the removed
protocol-fuzzer files anywhere in the tree
- `__aztec_nr_internals__` prefix-stripping logic is present in
`barretenberg/cpp/src/barretenberg/api/aztec_process.cpp` (applied
cleanly from the PR)

ClaudeBox log: https://claudebox.work/s/0347afb057f4542d?run=1
github-merge-queue Bot pushed a commit that referenced this pull request Apr 17, 2026
BEGIN_COMMIT_OVERRIDE
fix(pxe): stop block synchronizer on PXE shutdown (#22604)
fix(aztec): respect TEST_ACCOUNTS env var in local network mode (#22600)
fix: check all aztec-nr dependency tags, not just aztec (#22483)
fix: reuse anchor block in kernel oracle (#22631)
refactor: unify contract compilation pipeline via bb aztec_process
(#22590)
fix(pxe): queue registerSender wipe to avoid racing with in-flight jobs
(#22623)
fix(pxe): bounds-check PropertySelector in pick_notes (#22614)
fix(pxe): guard private event store rollback against in-flight jobs
(#22615)
refactor(pxe): rename sideEffectCounter to initialSideEffectCounter
(#22599)
fix(pxe): correct stale authwitness comment and inverted tagging error
(#22537)
fix(aztec-nr): range-check auth witness fields before byte cast (#22624)
fix(pxe): serialize block stream event handling to prevent race
conditions (#22635)
fix(pxe): throw error on origin/contract address mismatch in simulation
(#22637)
END_COMMIT_OVERRIDE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v4-next ci-draft Run CI on draft PRs. ci-full Run all master checks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants