Skip to content

sea: generate a compile cache for the VFS modules - #66768

Open
mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:sea-vfs-code-cache
Open

mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:sea-vfs-code-cache

Conversation

@mcollina

@mcollina mcollina commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

"useCodeCache": true is currently rejected together with "useVfs": true. The reason is a plumbing gap rather than a V8 limitation: the SEA code cache is only consumed by the embedder compile paths (compileFunctionForCJSLoader(..., is_sea_main=true) and the kEmbedder branch in module_wrap.cc), while a main script in the VFS runs through the regular CJS/ESM loaders. A main-only cache would also be of little use for a VFS bundle, where the dependency graph is what costs at startup.

This PR implements the build-time half of the fix:

  • When both flags are set, --build-sea / --experimental-sea-config spins up a Node.js instance (CommonEnvironmentSetup + LoadEnvironment), mounts the assets (or the vfsArchive) exactly like the runtime does, and compiles every .js/.cjs/.mjs (and .ts/.cts/.mts with --strip-types) module in the mount without executing it, via wrapSafe / compileSourceTextModule, with a portable CompileCacheHandler whose keys are relative to the mount point.
  • The resulting entries are serialized in the compile cache file layout and embedded as reserved node:sea:compile-cache/<key> assets. The SEA provider hides node:sea: keys so they don't show up as files in the mount. Modules that fail to compile are skipped.
  • CompileCacheHandler gains a key_root for portable caches and SerializeEntries(), which shares its serialization with Persist(). The internal enableCompileCache binding takes the key root as a 4th argument.
  • The build instance runs with a cloned environment where NODE_COMPILE_CACHE* is removed, so the user's compile cache settings cannot hijack the build.

Reading the embedded entries at startup (a read-only, blob-backed CompileCacheHandler enabled from initSeaVfs, including in workers under vfsArchive) is left to a follow-up PR, which will also decide whether the main-only code cache blob still needs to be generated under VFS.

Tests: test/sea/test-single-executable-application-vfs-code-cache.js checks the number of embedded entries for the plain, ESM and ZIP fixtures (5/4/3), that NODE_COMPILE_CACHE / NODE_DISABLE_COMPILE_CACHE in the build environment do not affect the result, and that the built executable still runs. The rejection case was removed from test-build-sea-vfs-incompatible-options.js. The existing VFS SEA tests and test/parallel/test-compile-cache* pass.

—-

AI generated, reviewed by me

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/single-executable

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. single-executable Issues and PRs related to single-executable applications. vfs Issues and PRs related to the virtual filesystem subsystem. labels Oct 15, 2026
@mcollina
mcollina force-pushed the sea-vfs-code-cache branch 2 times, most recently from a35d240 to 5ee0908 Compare October 15, 2026 15:58
"useCodeCache" was rejected together with "useVfs" because the SEA code
cache is only consumed by the embedder compile paths, while a main
script in the VFS runs through the regular loaders and a main-only cache
would not cover the modules it depends on anyway.

Compile every JavaScript (and, with --strip-types, TypeScript) module in
the VFS when the preparation blob is generated, without executing them,
through a portable CompileCacheHandler keyed by the path relative to the
mount point. The entries are serialized in the cache file layout and
embedded as reserved node:sea:compile-cache/<key> assets, which the SEA
provider hides from the mount. Modules that fail to compile are skipped.

CompileCacheHandler gains a key root for portable caches and in-memory
serialization of its entries, shared with Persist(). The build runs in a
Node.js instance whose environment ignores NODE_COMPILE_CACHE.

Reading the embedded entries at startup is left to a follow-up.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.02902% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.64%. Comparing base (98dbf60) to head (5d06ff8).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
src/node_sea.cc 81.52% 8 Missing and 9 partials ⚠️
lib/internal/vfs/sea_compile_cache.js 93.58% 14 Missing and 1 partial ⚠️
src/compile_cache.cc 94.44% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66768      +/-   ##
==========================================
- Coverage   90.65%   90.64%   -0.02%     
==========================================
  Files         797      798       +1     
  Lines      278268   278618     +350     
  Branches    53517    53587      +70     
==========================================
+ Hits       252261   252550     +289     
- Misses      16386    16438      +52     
- Partials     9621     9630       +9     
Files with missing lines Coverage Δ
lib/internal/vfs/providers/sea.js 100.00% <100.00%> (ø)
lib/internal/vfs/sea.js 100.00% <100.00%> (ø)
src/compile_cache.h 100.00% <ø> (ø)
src/env.cc 84.23% <100.00%> (-0.07%) ⬇️
src/env.h 97.40% <ø> (ø)
src/node_modules.cc 79.67% <100.00%> (+<0.01%) ⬆️
src/compile_cache.cc 81.18% <94.44%> (+1.12%) ⬆️
lib/internal/vfs/sea_compile_cache.js 93.58% <93.58%> (ø)
src/node_sea.cc 87.81% <81.52%> (+0.08%) ⬆️

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 28, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 28, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. single-executable Issues and PRs related to single-executable applications. vfs Issues and PRs related to the virtual filesystem subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants