[BUG] Local Thinking model (Qwen 3.5) return empty content when reasoning consumes all tokens

[hththt]

Bug Description

When using Qwen 3.5 (or other thinking/reasoning models) via openai_compat provider, the LLM occasionally returns empty content with all tokens consumed by reasoning_content. This causes the agent to reply with the default error message:

"I've completed processing but have no response to give. Increase max_tool_iterations in config.json."

The message is misleading — the issue is not max_tool_iterations (default 20, only 3 used), but the LLM returning empty content.

Environment

• Model: Qwen3.5-35B-A3B-IQ4_XS.gguf (thinking model)
• Backend: llama.cpp (build b8176) via OpenAI-compatible API
• Config: max_tokens: 132768, temperature: 0.7

Steps to Reproduce

1. Configure a thinking model (Qwen 3.5, DeepSeek R1, etc.) in model_list
2. Send a message that triggers tool calls (e.g., "How to install third-party skills?")
3. The model calls tools (e.g., find_skills) across 2 iterations
4. On iteration 3, the model enters thinking mode, spends all tokens on reasoning_content, and returns empty content

Log Output

[INFO] agent: Tool call: find_skills({"limit":10,"query":"github integration"})
[INFO] tool: Tool execution completed {tool=find_skills, duration_ms=989, result_length=2351}
[INFO] agent: Tool call: find_skills({"limit":5,"query":"weather"})
[INFO] tool: Tool execution completed {tool=find_skills, duration_ms=665, result_length=1067}
[INFO] agent: LLM response without tool calls (direct answer) {iteration=3, content_chars=0}
[INFO] agent: Response: I've completed processing but have no response to give.

Root Cause Analysis

1. Empty content not handled in agent loop

In pkg/agent/loop.go:782-790, when the LLM returns no tool calls, only response.Content is used:

if len(response.ToolCalls) == 0 {
    finalContent = response.Content  // empty string when reasoning consumed all tokens
    break
}

The response.ReasoningContent (which may contain useful information) is ignored as a fallback. Then at line 530:

if finalContent == "" {
    finalContent = opts.DefaultResponse  // misleading error message
}

2. API Response from thinking model

Thinking models return:

{
  "content": "",
  "reasoning_content": "好的，用户想知道如何安装技能...(long reasoning)...",
  "finish_reason": "length"
}

The finish_reason: "length" confirms reasoning consumed the entire token budget, leaving nothing for actual content.

3. No mechanism to pass model-specific parameters

Thinking models (Qwen 3.5, DeepSeek R1, etc.) support disabling thinking mode via API request body parameters (e.g., chat_template_kwargs: {"enable_thinking": false} for llama.cpp). However, ModelConfig has no way to pass arbitrary parameters to the API request body.

Server-side flags (--chat-template-kwargs) do not work reliably (llama.cpp#13160). Only passing chat_template_kwargs in the request body works — which requires picoclaw support.

4. Important: disabling thinking is NOT a complete fix

Testing shows that disabling thinking mode causes Qwen 3.5 to lose its ability to decide when to stop calling tools — it loops indefinitely making tool calls instead of providing a direct answer. Thinking mode is essential for proper tool-use reasoning with these models.

Proposed Fix

Fix 1: Fallback to ReasoningContent when Content is empty

In pkg/agent/loop.go, when Content is empty but ReasoningContent is available, use it as fallback:

if len(response.ToolCalls) == 0 {
    finalContent = response.Content
    if finalContent == "" && response.ReasoningContent != "" {
        finalContent = response.ReasoningContent
    }
    break
}

Fix 2: Log finish_reason for better debugging

Currently finish_reason is not logged at INFO level, making it impossible to distinguish "model chose to stop" vs "max tokens hit":

logger.InfoCF("agent", "LLM response without tool calls (direct answer)",
    map[string]any{
        "content_chars":     len(finalContent),
        "finish_reason":     response.FinishReason,
        "reasoning_fallback": finalContent != response.Content,
    })

Fix 3: Support extra_body in ModelConfig

Add an extra_body field to ModelConfig to allow passing arbitrary parameters to the API request body:

{
  "model_name": "qwen-3.5",
  "model": "Qwen3.5-35B-A3B-IQ4_XS.gguf",
  "api_base": "http://localhost:9015/v1",
  "extra_body": {
    "chat_template_kwargs": { "enable_thinking": false }
  }
}

Files to modify:

• pkg/config/config.go — Add ExtraBody map[string]any to ModelConfig
• pkg/providers/openai_compat/provider.go — Store and merge extraBody into request body
• pkg/providers/http_provider.go — Thread ExtraBody through to provider
• pkg/providers/factory_provider.go — Pass cfg.ExtraBody when creating providers

[huaaudio]

Thanks for the issue. This is a very good catch. I've created #992 for this.

[yinwm]

Review Summary

Issue Confirmed ✅ Great analysis!

The current error message Increase max_tool_iterations is misleading and needs to be fixed.

---

Proposed Fix

Single line change: pkg/agent/loop.go:58

// Current
const defaultResponse = "I've completed processing but have no response to give. Increase `max_tool_iterations` in config.json."

// Change to
const defaultResponse = "The model returned an empty response. This may indicate insufficient max_tokens or a model configuration issue."

---

Notes on ReasoningContent Fallback

Not recommended as default behavior:

• ReasoningContent is the model's internal thinking process, format may not be user-friendly
• Silently falling back would mask the underlying issue
• Could be added as an optional feature if needed

---

Notes on Fix 3 (extra_body)

Suggest splitting into a separate Issue/PR to not block this fix.

---

Thanks for the high-quality bug report! Feel free to submit a PR.

[nikolasdehor]

The issue reporter analysis is accurate. Here is additional context from reading the code.

Empty content handling path:

In pkg/agent/loop.go, the path is:

1. Line 724-733: When len(response.ToolCalls) == 0, finalContent = response.Content (empty string)
2. Line 622-624: If finalContent is empty, falls back to opts.DefaultResponse which is the misleading max_tool_iterations message (line 60)

The response.ReasoningContent and response.Reasoning fields are available but never consulted as fallbacks.

Regarding the three proposed fixes:

Fix 1 (ReasoningContent fallback) - I would add a nuance. Dumping raw reasoning content to the user is not ideal since it may contain internal chain-of-thought that is incoherent as a final answer. A better approach would be to check response.FinishReason == "length" and provide a targeted error message like "The model consumed its entire token budget on reasoning without producing a final answer. Consider increasing max_tokens or using a model with a larger context window." This only triggers when finish_reason is "length" (not when the model intentionally returns empty content).

Fix 2 (Log finish_reason) - Straightforward and valuable. The current debug log at line 714-722 includes response.Reasoning but not FinishReason. Adding it to both the debug and info logs would help diagnose this class of issues.

Fix 3 (extra_body in ModelConfig) - This is the right long-term approach for thinking model control. The extra_body fields would need to be merged into the request body in pkg/providers/openai_compat/provider.go at line 117-118, after building requestBody. This needs to be threaded through: ModelConfig.ExtraBody -> factory_provider.go -> HTTPProvider -> openai_compat.Provider via a new WithExtraBody() option. One risk: some OpenAI-compat endpoints reject unknown fields, so the merge should happen after all standard fields are set so extra_body can intentionally override them (e.g., overriding temperature per-model).