Skip to content

fix(ruvector-cli): bundle ONNX runtime assets in dist/ + extend verify-dist gate#405

Open
ronmikailov wants to merge 1 commit intoruvnet:mainfrom
ronmikailov:fix/ruvector-onnx-asset-bundling
Open

fix(ruvector-cli): bundle ONNX runtime assets in dist/ + extend verify-dist gate#405
ronmikailov wants to merge 1 commit intoruvnet:mainfrom
ronmikailov:fix/ruvector-onnx-asset-bundling

Conversation

@ronmikailov
Copy link
Copy Markdown

Summary

0.2.230.2.25 ship dist/core/onnx/pkg/ containing only package.json. No loader.js, no wasm-bindgen JS bridge, no .wasm binary. The embedder reads those at startup via path.join(__dirname, 'onnx', ...), so on a clean install of any of those versions:

$ npx ruvector embed text "hello"
Embedding failed: Failed to initialize ONNX embedder:
  ONNX WASM files not bundled. The onnx/ directory is missing.

This PR makes the build/publish actually ship what it claims to ship.

Root cause

Old build script:

"build": "tsc && cp src/core/onnx/pkg/package.json dist/core/onnx/pkg/"

tsc only emits .js for .ts inputs. The JS loader, the wasm-bindgen bridge, and the .wasm binary itself are all non-TS, so all three were silently skipped. Only the single-file cp ran.

The verify-dist gate added in #403 caught the original dist/ regression but didn't catch this one because it only scans bin/cli.js for require('../dist/...js'). These assets are read via path.join + fs.readFileSync + dynamicImport, not require(), so the original gate is structurally blind to them.

Changes

1. scripts/copy-onnx-assets.js (new)

Explicit, cross-platform Node script that copies the 8 runtime assets the embedder needs. Listed explicitly (not a recursive cp) so adding a new asset is a deliberate choice, not an accident. Fails the build with a non-zero exit if any source asset is missing.

2. package.json

- "build": "tsc && cp src/core/onnx/pkg/package.json dist/core/onnx/pkg/"
+ "build": "tsc && node scripts/copy-onnx-assets.js"

3. scripts/verify-dist.js

Adds a second pass that asserts the four runtime asset paths read by dist/core/onnx-embedder.js actually exist. Same publish-gate semantics as the existing bin/cli.js scan, just covering the path-join'd asset reads as well.

Verification

$ rm -rf dist && npm run build
copy-onnx-assets: 8 asset(s) copied to dist/core/onnx/.

$ npm run verify-dist
verify-dist: 13 dist require path(s) + 4 runtime asset(s) present.

$ ls dist/core/onnx/pkg/
LICENSE
package.json
ruvector_onnx_embeddings_wasm.d.ts
ruvector_onnx_embeddings_wasm.js
ruvector_onnx_embeddings_wasm_bg.js
ruvector_onnx_embeddings_wasm_bg.wasm
ruvector_onnx_embeddings_wasm_bg.wasm.d.ts

The "ONNX WASM files not bundled" error no longer fires — the existsSync check at dist/core/onnx-embedder.js:184 passes because the files exist.

Out of scope (followup needed)

With all assets present, Node still fails the next step:

Failed to initialize ONNX embedder: Unknown file extension ".wasm"
for .../dist/core/onnx/pkg/ruvector_onnx_embeddings_wasm_bg.wasm

The shipped ruvector_onnx_embeddings_wasm.js is the wasm-bindgen bundler target (import * as wasm from "./xxx.wasm"). Node ESM rejects bare .wasm imports without --experimental-wasm-modules. The embedder code expects the web/no-modules target (with wasmModule.default(wasmBytes) for deferred init), but it never gets that branch because the bundler-target import fails before the embedder's branch logic runs.

The clean fix is to rebuild ruvector-onnx-embeddings-wasm with --target nodejs (or web with deferred init) so the entry exposes a default(bytes) initializer and doesn't try to import the .wasm directly. That's a Rust/wasm-bindgen build-config change, not a JS one — out of scope for this PR.

This PR is the necessary precondition for that work: the assets actually need to ship before any wasm-target fix can possibly run end-to-end.

Related

🤖 Generated with claude-flow

…-dist gate

Issue: 0.2.23 → 0.2.25 ship `dist/core/onnx/pkg/` containing only
`package.json`, with no `loader.js`, no wasm-bindgen JS bridge, and no
`.wasm` binary. The embedder reads those at startup via
`path.join(__dirname, 'onnx', ...)`, so on a clean install:

```
$ npx ruvector embed text "hello"
Embedding failed: Failed to initialize ONNX embedder:
  ONNX WASM files not bundled. The onnx/ directory is missing.
```

The previous build script copied exactly one file:

```
"build": "tsc && cp src/core/onnx/pkg/package.json dist/core/onnx/pkg/"
```

Root cause: tsc only emits .js for .ts inputs, so the JS loader, the
wasm-bindgen bridge, and the .wasm itself were silently skipped. The
verify-dist gate added in ruvnet#403 didn't catch this because it only scanned
bin/cli.js for `require('../dist/...js')` calls — these assets are read
via path.join + fs.readFileSync + dynamicImport, not require().

Changes:

1. `scripts/copy-onnx-assets.js` (new) — explicit, cross-platform Node
   script that copies all 8 runtime assets from src/core/onnx/ to
   dist/core/onnx/. Listed explicitly (not a recursive cp) so adding a
   new asset is a deliberate choice, not an accident. Fails the build
   if any source asset is missing.

2. `package.json` — `build` now runs `node scripts/copy-onnx-assets.js`
   instead of the single-file `cp`.

3. `scripts/verify-dist.js` — adds a second pass that asserts the four
   runtime asset paths read by `dist/core/onnx-embedder.js` actually
   exist. This catches the same class of regression (build copies the
   wrong files) at publish time, not at user-install time.

Verification:

```
$ rm -rf dist && npm run build
copy-onnx-assets: 8 asset(s) copied to dist/core/onnx/.

$ npm run verify-dist
verify-dist: 13 dist require path(s) + 4 runtime asset(s) present.

$ ls dist/core/onnx/pkg/
LICENSE
package.json
ruvector_onnx_embeddings_wasm.d.ts
ruvector_onnx_embeddings_wasm.js
ruvector_onnx_embeddings_wasm_bg.js
ruvector_onnx_embeddings_wasm_bg.wasm
ruvector_onnx_embeddings_wasm_bg.wasm.d.ts
```

Out of scope: with all assets present, Node still fails to dynamic-import
the wasm-bindgen bundler-target entry (`import * as wasm from "./xxx.wasm"`
isn't supported by Node ESM without `--experimental-wasm-modules`). That
needs a `--target nodejs` (or `web` with deferred init) rebuild of
ruvector-onnx-embeddings-wasm. Tracking separately.

This PR makes the build/publish actually ship what it claims to ship —
necessary precondition for any of the followup wasm-target work.

Co-Authored-By: claude-flow <ruv@ruv.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant