Eval bug: Infinite //////... generation with llama-cli

[Elefant-Freeciv]

Name and Version

version: 8870 (82209ef)
built with GNU 11.4.0 for Linux x86_64

Operating systems

Linux

GGML backends

HIP

Hardware

i9 13900k+2x7900xtx, 128gb ddr4

Models

qwen3.5-397b ud iq2 m

https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF?show_file_info=UD-IQ2_M%2FQwen3.5-397B-A17B-UD-IQ2_M-00001-of-00004.gguf

Problem description & steps to reproduce

Using release b8870 llama-cli ROCM on linux backend with unsloth's qwen3.5-397b ud iq2 m model, if my prompt has more than one line or if I give it a follow up prompt, it generates nothing but ///////... forever. This would seem to be caused by how it handles carriage returns in the prompt. Is this an issue with llama.cpp or with the model?

cmd:LLAMA_CACHE="unsloth/Qwen3.5-397B-A17B-GGUF" ./llama-cli -hf unsloth/Qwen3.5-397B-A17B-GGUF:UD-IQ2_M --ctx-size 65536 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.00 --chat-template-kwargs '{"enable_thinking":true}' -ngl 99 --n-cpu-moe 50

First Bad Commit

Unknown

Relevant log output

Logs

...and smell the spice upon the breeze,
Know Captain Link is drawing near,
With fork and knife and pirate cheer.
And when he dies on ocean foam,
Don't bring him gold to be his home,
Just lay a link upon his chest,
And let the greedy pirate rest.

[ Prompt: 13.1 t/s | Generation: 7.6 t/s ]

> Add a pet parrot.

[Start thinking]
////////////////////////////

[chejh-amd]

A few things you could try to narrow it down with that stack (397B MoE, IQ2_M, HIP, dual 7900, thinking + CPU MoE — lots of moving parts):

• Run once with thinking off — if the // loop goes away, it's probably the thinking path interacting with newlines, not generic gen.
• Same text but forced single-line, to see if newlines are really the trigger.
• All MoE on GPU (no CPU MoE) for one run, just to rule out the offload path.
• A heavier quant if feasible — if slashes stop, IQ2 is the likely culprit.

[Elefant-Freeciv]

Putting the prompt on a single line fixes the issue. The problem persists when a followup is asked and i dont know how the fix that. Is there an option to always have the context a single line? I don't have enough vram for a heavier quantity or completely gpu inference.

[Elefant-Freeciv]

Llama-server works fine, so it has something to do with llama-cli, not the quant. Probably.