Eval bug: OOM Error for Vulkan Backend on models that don't strictly fit in VRAM

[jhemmond]

Name and Version

b7502

Operating systems

Windows

GGML backends

Vulkan

Hardware

Ryzen AI 370HX with 64GB LPDDR5X, and 890m iGPU

Models

Command: ${llamasvr} -m ${mpath}\Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf --jinja --no-mmap --ctx-size 20000 --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.00

Issue also happens with GLM-4.5-Air, and LLama3.3-70B.

Problem description & steps to reproduce

After version b7502, and all the way to current b7642 today, any models loaded into memory will throw:

.........srv  log_server_r: request: GET /health 127.0.0.1 503

................srv  log_server_r: request: GET /health 127.0.0.1 503

..............srv  log_server_r: request: GET /health 127.0.0.1 503

..........llama_model_load: error loading model: vk::Queue::submit: ErrorOutOfDeviceMemory

llama_model_load_from_file_impl: failed to load model

srv  log_server_r: request: GET /health 127.0.0.1 503

common_init_from_params: failed to load model 'C:\Users\PC\llama-cpp\models\\Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf'

srv    load_model: failed to load model, 'C:\Users\PC\llama-cpp\models\\Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf'

srv    operator(): operator(): cleaning up before exit...

main: exiting due to model loading error

I have 32GB of my system RAM allocated in BIOS to the 890m iGPU. Models fitting inside the 32GB, run fine. Models larger than that cause issues. If I remove mmap, the model loads, but it goes into disk for whatever reason, which takes a really long time. Prior versions do not have this issue.

First Bad Commit

b7502

Relevant log output

Logs

[jeffbolznv]

The OOM error on vkQueueSubmit is weird and has come up a couple times before (e.g. #17336). It sounds like enabling the nommap/directio path is somehow contributing to this error. It might be worth trying #18166 with directio disabled. But from past evidence this kind of failure seems like a driver bug or system limitation.

[0cc4m]

If you're not running the latest driver, try updating it.

[jhemmond]

For AMD, I'm on the latest 25.12.1

[jhemmond]

Tried with —no-direct-io and the model doesn’t load due to the same OOM error.

I’m not sure what else to try at this point. All the large functioning models are rendered unusable with this limitation.

[jhemmond]

I tested the latest build b7719 this morning, and it’s loading large models like GLM-4.5 AIR, Llama3.3-70B, but still can’t load Qwen3-Next at Q4_K_XL despite having enough memory.

[0cc4m]

Please upload the output of vulkaninfo.

[jhemmond]

vulkaninfo.txt

Additional testing done today:

• Tried the Q4_0 version of the model, and that didn't load
• Tried the Q3_K_XL version (about 10GB less than Q4_K_XL), and that loaded fine

For reference the GLM-Air loads fine despite being ~1GB larger in size that Q4_K_XL. There's about 7-9GB left in RAM free for the Qwen Next, but it still complains about OOM.

[0cc4m]

The AMD proprietary driver gives you access to all memory you assign in BIOS as VRAM (of course) and half of the RAM. In your case, that is 48GB. Qwen3-Next Q4_K_XL is 46GB on its own, you have no space left for KV cache here. Depending on what other models you tried, the KV cache size is different, so that may cause one to work and another to fail.

The only way I know to give your GPU access to close to all of your unified memory would be to use Linux.