A numeric error is thrown by pipeline during text-generation model loading

[DawitJung]

System Info

1. Apple M2 Pro
2. mac OS Sequoia v15.0.1
3. Google Chrome v130.0.6723.116
4. @huggingface/transformers v3.0.2

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I'm encountering an issue while trying to load a text-generation model using the pipeline function. My code searches for models on Hugging Face by keyword, filters for "text-generation", "transformers.js", and "onnx" tags, and attempts to load the resulting models. When I attempt to load certain models, an error with a numeric code (e.g., 3330359752) is thrown. The numeric code changes each time, suggesting it may not hold specific meaning.

Notably, it seems that smaller models load successfully, while larger models may consistently throw this error, though I’m not entirely certain.

Models that loaded successfully:

• onnx-community/Llama-3.2-1B-Instruct-q4f16
• onnx-community/AMD-OLMo-1B-SFT-DPO

Models that failed to load:

• onnx-community/Llama-3.2-1B-Instruct
• onnx-community/Llama-3.2-3B-Instruct
• Xenova/Phi-3-mini-4k-instruct

Reproduction

generator = await pipeline(
    "text-generation",
    "onnx-community/Llama-3.2-3B-Instruct",
    { device: "webgpu" }
  );

[xenova]

While this might be an out-of-memory issues, the models have been tested and work in Node.js, so maybe a runtime error occurring for WebGPU. cc @guschmue

[dreamingtulpa]

Experiencing the same issue.

this.model = await AutoModel.from_pretrained('briaai/RMBG-2.0');
this.processor = await AutoProcessor.from_pretrained('briaai/RMBG-2.0');

Results in Model loading failed: 763518952.

You think this is an OOM issue as well?

[arneyjfs]

Facing the same issue. I'm admittedly out my depth in terms of being able to figure out the issue however I've noticed the dtypes being important
e.g.

• as per OPs message Llama-3.2-1B-Instruct will not load but the q4f16 version will.
• onnx-community/AMD-OLMo-1B-SFT-DPO will not automatically load, but being explicit with {device: "webgpu", dtype: "q4" } it will load

Additional Context:
Also on an Apple-Silicone M1 chip.
I have been able to load even the Llama-3.1-8B-Instruct-q4f32_1-MLC model using WebLLM which makes me think memory is maybe not the issue

[mattppal]

Experiencing the same issue.

this.model = await AutoModel.from_pretrained('briaai/RMBG-2.0');
this.processor = await AutoProcessor.from_pretrained('briaai/RMBG-2.0');

Results in Model loading failed: 763518952.

You think this is an OOM issue as well?

Same issue with RMBG-2.0 on an M3 Max...

[arneyjfs]

Just updated to the latest version and I still fail to load almost all models. The models that do load I would guess are not making use of WebGPU as they are running far slower than the same models on WebLLM. I am very keen to use transformer.js though as we need to implement models other than just LLMs.

Is there any more details I can provide to help reproduce this?
I am currently using the latest version of chrome (Version 132.0.6834.160 (Official Build) (arm64)) on MacOS 15.3 (M1 Pro)

[xenova]

Closing as stale (#1382 provides better runtime support, and we now also support more optimized exports).