Skip to content

fix(chokidar, dictionaries-entry): fix memory leak in intlayer watch#400

Merged
aymericzip merged 1 commit intoaymericzip:mainfrom
modanub:fix/watch-memory-leak
Mar 21, 2026
Merged

fix(chokidar, dictionaries-entry): fix memory leak in intlayer watch#400
aymericzip merged 1 commit intoaymericzip:mainfrom
modanub:fix/watch-memory-leak

Conversation

@modanub
Copy link
Copy Markdown
Contributor

@modanub modanub commented Mar 21, 2026

Summary

intlayer watch leaks ~1MB per file change event, eventually hitting OOM.

Root cause: getUnmergedDictionaries() and getDictionaries() used clearModuleCache() + require() to reload dictionary JSON on every change. V8 retains internal module bookkeeping even after require.cache eviction, so each clear+reload cycle leaked the full dictionary payload (~850KB for 23 JSON files).

Fix:

  • Generated CJS entry points now use JSON.parse(fs.readFileSync(...)) instead of require() for JSON imports
  • dictionaries-entry and unmerged-dictionaries-entry execute the entry point via readFileSync + vm.runInThisContext instead of clearModuleCache + require()

This bypasses require.cache entirely.

Improvement from ~972KB/call to ~1KB/call. Memory stays stable at ~200-300MB during extended watch sessions.

Test plan

  • intlayer build generates dictionaries correctly
  • intlayer watch processes file changes normally
  • bun run dev works end-to-end
  • Isolated benchmark: getUnmergedDictionaries() — ~1KB/call (was ~972KB)
  • Realistic 20-change session: memory stable at 215-335MB

getUnmergedDictionaries() and getDictionaries() used clearModuleCache()
+ require() to reload dictionary JSON on every file change. V8 retains
internal module bookkeeping after cache eviction, leaking ~1MB/call.

Replace require() with readFileSync + JSON.parse for dictionary JSON,
and vm.runInThisContext for CJS entry point execution.

Improvement from ~972KB/call to ~1KB/call
Copy link
Copy Markdown
Owner

@aymericzip aymericzip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merging to test

Thanks @modanub, good catch

@aymericzip aymericzip merged commit 2302b90 into aymericzip:main Mar 21, 2026
1 check passed
@aymericzip
Copy link
Copy Markdown
Owner

aymericzip commented Mar 22, 2026

restoring dictionary-entry file for now because of build optimization error on vite apps

bun run build
$ vite build
Failed to transform with Babel plugin: Error: Calling `require` for "../dictionary/app.json" in an environment that doesn't expose the `require` function. See https://rolldown.rs/in-depth/bundling-cjs#require-external-modules for more details.
    at file:///Users/aymericpineau/Documents/intlayer_/packages/@intlayer/dictionaries-entry/dist/esm/_virtual/_rolldown/runtime.mjs:4:8
    at /Users/aymericpineau/Documents/intlayer_/examples/vite-preact-app/.intlayer/main/dictionaries.cjs:2:31
    at getDictionaries (file:///Users/aymericpineau/Documents/intlayer_/packages/@intlayer/dictionaries-entry/dist/esm/index.mjs:20:128)
    at intlayerOptimize (file:///Users/aymericpineau/Documents/intlayer_/packages/vite-intlayer/dist/esm/intlayerOptimizePlugin.mjs:27:24)
    at intlayerPlugin (file:///Users/aymericpineau/Documents/intlayer_/packages/vite-intlayer/dist/esm/intlayerPlugin.mjs:65:15)
    at file:///Users/aymericpineau/Documents/intlayer_/examples/vite-preact-app/node_modules/.vite-temp/vite.config.ts.timestamp-1774155494704-17e03ab0b6724.mjs:10:2
    at ModuleJob.run (node:internal/modules/esm/module_job:263:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:540:24)
    at async loadConfigFromBundledFile (file:///Users/aymericpineau/Documents/intlayer_/node_modules/.bun/vite@8.0.1+5044978246f43f13/node_modules/vite/dist/node/chunks/node.js:34544:12)
    at async bundleAndLoadConfigFile (file:///Users/aymericpineau/Documents/intlayer_/node_modules/.bun/vite@8.0.1+5044978246f43f13/node_modules/vite/dist/node/chunks/node.js:34405:17)

@modanub
Copy link
Copy Markdown
Contributor Author

modanub commented Mar 22, 2026

Hey, sorry about the Vite build breakage — the vm.runInThisContext approach passes require which doesn't exist in ESM. Opened #401 as a follow-up that reads JSON files directly via readdirSync + readFileSync instead. Tested against vite-react, vite-preact, vite-vue, vite-svelte, vite-solid, and nextjs-15 examples — all build fine.

@aymericzip
Copy link
Copy Markdown
Owner

nice, merged

FYI, the cache clean logic was implemented for intlayer-editor & live sync feature, once a change was made on the json, we had to restart the editor to get the updated jsons, cleaning cache was fixing that

On vite / next / expo, that entry file is aliased for web compatibility

I will test it tomorrow

@aymericzip
Copy link
Copy Markdown
Owner

hey @modanub

Works well, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants