Skip to content

Empty LLM output is converted to a user-visible "Error: Run completed..." message #76

@siglimumuni

Description

@siglimumuni

Empty LLM output is converted to a user-visible "Error: Run completed without any response text..." message in the connector channel

Summary

When an agent's LLM run completes without producing visible text, chat-turn-finalization.ts:300-338 infers an error and posts Error: Run completed without any response text, tool calls, or explicit error details. Check the provider configuration and try again. to the user-visible connector channel (Slack, etc.).

This is wrong in three common cases:

  1. Soul-driven intentional silence — the agent's system prompt instructs it to produce no output ("end turn silently if no work to do" patterns). The agent obeys, the runtime treats compliance as failure.
  2. Multi-agent shared channels — when N agents all dispatch turns in response to the same human prompt, only one or two may have substantive output; the others correctly stay quiet but each posts an "Error..." message, polluting the channel.
  3. Honest empty turns — sometimes the right response is silence. The runtime forces verbosity.

The error message also amplifies multi-agent loop dynamics: each empty-output run produces a posted message → that message arrives at peer agents' connectors → they see a "new bot message" and may dispatch follow-ups.

Repro (from production, 2026-05-07)

Setup: two SwarmClaw agents in a Slack channel with groupPolicy: 'open', both with Soul rules that include "if no new information to add, end the turn silently — produce no output."

  1. Human posts a question in #strategy
  2. Both agents dispatch turns
  3. Agent A produces a real reply; Agent B's LLM correctly produces empty output (per Soul rule)
  4. Agent B's run reaches chat-turn-finalization.ts:338:
    let finalText = (fullResponse || '').trim() || (!internal && errorMessage ? `Error: ${errorMessage}` : '')
  5. Since fullResponse is empty and errorMessage was inferred by deriveTerminalRunError, finalText becomes Error: Run completed without any response text...
  6. Slack receives the error post in the user-visible channel

Result for the human: instead of seeing one substantive reply (Agent A) and silence from Agent B, they see Agent A's reply plus a confusing system-error-looking message from Agent B.

Diagnosis

Two functions are interacting:

src/lib/server/chat-execution/chat-execution-tool-events.ts:104-116:

function deriveTerminalRunError(params: {
  errorMessage: string | undefined
  fullResponse: string
  streamErrors: string[]
  toolEvents: MessageToolEvent[]
  internal: boolean
}): string | undefined {
  if (params.errorMessage) return params.errorMessage
  if (params.streamErrors.length > 0 && !params.fullResponse.trim()) {
    return params.streamErrors[params.streamErrors.length - 1]
  }
  if (!params.internal && !params.fullResponse.trim() && params.toolEvents.length === 0) {
    return 'Run completed without any response text, tool calls, or explicit error details. Check the provider configuration and try again.'
  }
  return undefined
}

src/lib/server/chat-execution/chat-turn-finalization.ts:300-338:

const terminalError = deriveTerminalRunError({...})
if (terminalError && terminalError !== errorMessage) {
  // ...log it...
  errorMessage = terminalError    // ← line 334
}
// ...
let finalText = (fullResponse || '').trim() || (!internal && errorMessage ? `Error: ${errorMessage}` : '')   // ← line 338

The problem: deriveTerminalRunError infers a synthetic error message when output is empty. That inference is a heuristic — empty output isn't actually an error in many legitimate cases. But line 338 promotes it to user-visible Error: ... text.

The "honest" inputs to errorMessage (an actual errorMessage parameter passed in, or a real streamErrors entry) are real errors and reasonable to surface. The third branch — pure inference from empty output + no tool events — is overreaching.

Suggested fix

Two tiers, either works:

Minimal: in deriveTerminalRunError, drop the third (heuristic) branch entirely. Real errors (errorMessage or streamErrors) still get surfaced; empty-output-as-inferred-error becomes silent (no Slack post, just internal log).

   if (params.errorMessage) return params.errorMessage
   if (params.streamErrors.length > 0 && !params.fullResponse.trim()) {
     return params.streamErrors[params.streamErrors.length - 1]
   }
-  if (!params.internal && !params.fullResponse.trim() && params.toolEvents.length === 0) {
-    return 'Run completed without any response text, tool calls, or explicit error details. Check the provider configuration and try again.'
-  }
   return undefined

The internal log at chat-turn-finalization.ts:309 (the Run ended without a visible response warning) still fires, so operators can find empty runs in app.log. Just don't post the message to the user.

Better: keep the inference but don't promote it to user-visible text. Let it remain in logs only.

// chat-turn-finalization.ts:338
- let finalText = (fullResponse || '').trim() || (!internal && errorMessage ? `Error: ${errorMessage}` : '')
+ // Only surface an Error: message when there's a real error (caller passed errorMessage,
+ // or streamErrors fired). Don't surface inferred-empty-output as user-visible text.
+ const hasRealError = !!(params.errorMessage || streamErrors.length > 0)
+ let finalText = (fullResponse || '').trim() || (!internal && hasRealError ? `Error: ${errorMessage}` : '')

(Variable plumbing simplified for clarity; actual change would need to track whether errorMessage came from input vs. inference.)

Best: per-agent or per-connector config flag. agent.suppressEmptyOutputError: true (default true) lets agents that legitimately produce silent turns opt into clean behavior; agents that should always produce output keep the diagnostic message.

Why this matters

  • Multi-agent workspaces: in any channel with multiple SwarmClaw agents and groupPolicy: 'open', every human prompt triggers N dispatches, of which usually only 1-2 produce substantive output. The rest produce Error: ... posts. With 3 agents, that's 1 useful response and 2 error-noise posts per prompt. Channels become unusable.
  • Soul-driven silence patterns: many compass-ops Souls have rules like "end turn silently if no work to do" or "if no new information, produce no output." These rules are explicit, deliberate, and the right behavior — but the runtime undermines them by posting an error.
  • Loop amplification: in shared channels with anti-loop gates (in our case, Patch 6 — a custom human-in-the-loop counter), each error post counts as a bot message and contributes to the gate's budget, accelerating the block. Silent runs would not.

Environment

  • swarmclaw 1.9.25
  • Node 22 LTS, macOS arm64
  • Provider: DeepSeek V4 Pro / V4 Flash (bug is in connector / chat-execution code, not provider-specific)
  • Connector: Slack with groupPolicy: 'open' and multiple agents per channel
  • Multi-agent setup: 8 agents, several sharing channels

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions