Skip to content

fix(providers): robust CLI tool call extraction and mixed response handling#1813

Closed
securityguy wants to merge 19 commits intosipeed:mainfrom
securityguy:fix/cli-tool-call-extraction
Closed

fix(providers): robust CLI tool call extraction and mixed response handling#1813
securityguy wants to merge 19 commits intosipeed:mainfrom
securityguy:fix/cli-tool-call-extraction

Conversation

@securityguy
Copy link
Copy Markdown
Contributor

Problem

CLI providers (claude-cli, gemini-cli, codex-cli) instruct the LLM to return tool calls as a JSON block. The extraction code used a literal strings.Index search for {"tool_calls" which failed in several real-world cases:

  1. Pretty-printed JSON — LLMs often return {\n "tool_calls" with whitespace after {, which the literal search missed entirely, causing the raw JSON to leak to the user as plain text.
  2. Markdown code fences — some models wrap their JSON response in ```json ``` blocks, which the parser did not strip.
  3. Arguments as a JSON object — the spec allows arguments to be either a JSON-encoded string or a plain object; only the string form was handled, causing a silent parse failure.
  4. Mixed responses — when a response contained both text and a tool call, the text was saved to session history but never published to the user.

Fix

  • Strip markdown code fences before parsing
  • Find the JSON candidate using the first { and last } positions rather than a literal string match
  • Unmarshal directly and check for a top-level "tool_calls" key
  • Accept arguments as either a JSON-encoded string or a plain JSON object
  • Publish response.Content to the bus immediately when a response contains both text and tool calls
  • Remove the now-unused findMatchingBrace function

29 tests covering all cases are included in tool_call_extract_test.go.

securityguy and others added 19 commits March 15, 2026 20:53
Some providers (via OpenRouter) reject assistant messages with
"content": "" alongside tool_calls. The OpenAI spec permits content to
be absent when tool_calls is set. Switch openaiMessage.Content from
string to *string with omitempty and introduce msgContent() to return
nil when content is empty and tool calls are present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ields

Some OpenAI-compatible providers (e.g. OpenRouter routing to strict
backends) reject non-standard fields in the request body such as
reasoning_content in messages and extra_content / thought_signature
in tool calls. Add a per-model strict_compat: true config option that
strips these fields before serialization.

Implementation:
- Add StrictCompat bool to config.ModelConfig
- Add WithStrictCompat option to openai_compat.Provider
- Refactor HTTPProvider constructors into a single NewHTTPProviderWithOptions
  using variadic openai_compat.Option, eliminating the growing list of
  named constructors
- Thread StrictCompat through CreateProviderFromConfig via composed options

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the claude CLI exits with a non-zero status, the previous error
handler only checked stderr. However, the CLI writes its output
(including error details) to stdout, especially when invoked with
--output-format json. This left the caller with only "exit status 1"
and no actionable information.

Now includes both stderr and stdout in the error message so the actual
failure reason is visible in logs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add claude-cli and codex-cli to the supported vendors table and
include vendor-specific configuration examples explaining:
- No API key is required (uses existing CLI subscription)
- The claude-code sentinel model ID skips --model flag so the CLI
  uses its own configured default model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add channels.telegram_bots config allowing multiple Telegram bot tokens
to be configured, each mapped to a separate channel (e.g. telegram-amber,
telegram-karen). Each channel can be independently bound to an agent via
the bindings config, enabling distinct AI personas behind separate bots.

Backward compatibility is preserved: the existing channels.telegram
single-entry config continues to work unchanged. On load it is normalized
into telegram_bots as an entry with id "default", which produces the
channel name "telegram" so all existing bindings remain valid.

Key changes:
- config: add TelegramBotConfig struct with ChannelName/AsTelegramConfig
  helpers; add TelegramBots field to ChannelsConfig; normalize legacy
  single entry into list on load
- telegram: add NewTelegramChannelFromConfig constructor accepting
  TelegramConfig + explicit channel name (avoids import cycle)
- channels: add TelegramBotFactory registry; add injectChannelDependencies
  helper to eliminate injection code duplication; add duplicate channel
  name guard in initTelegramBot; update initChannels to iterate over
  TelegramBots; add prefix-based rate limit fallback for named bots

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…le.json

Add two disabled example bots (alice, bob) under channels.telegram_bots
and corresponding top-level bindings to illustrate how multiple Telegram
bots map to separate named channels and agents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds GeminiCliProvider that wraps the Gemini CLI as a subprocess,
following the same pattern as the existing claude-cli and codex-cli
providers.

The provider invokes:
  gemini --yolo --output-format json --prompt ""

with the prompt sent via stdin. The --prompt "" flag enables
non-interactive (headless) mode, reading the full prompt from stdin.

Key details:
- Model sentinel: "gemini-cli" skips --model flag (uses CLI default)
- Explicit model: "gemini-cli/gemini-2.5-pro" passes --model gemini-2.5-pro
- System messages prepended to stdin (no --system-prompt flag in gemini)
- Parses JSON response format: {"response": "...", "stats": {"models": {...}}}
- Token usage summed across all models in stats.models (gemini uses
  multiple internal models per request)
- Tool calls extracted from response text using shared extractToolCallsFromText
- New protocol: "gemini-cli" / alias "geminicli"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add PR sipeed#1633 (gemini-cli provider) to contributions table
- Add configuration guide section covering:
  - claude-cli, codex-cli, and gemini-cli providers with model_list examples
  - Multiple Telegram bots with bindings and per-agent config
  - Agent workspace and personality file notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds GeminiCliProvider (PR sipeed#1633 against sipeed/picoclaw).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace Amber/Karen with Alice/Bob in all README examples for consistency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously all agents shared a single LLMProvider instance created from
agents.defaults.model_name. Per-agent model config (agents.list[].model)
only changed the model string passed to Chat() — it never changed which
provider binary was invoked. This caused cross-provider fallback chains
(e.g. gemini-cli falling back to claude-cli) to fail, and made it
impossible to assign different CLI providers to different agents.

Introduces ProviderDispatcher which lazily creates and caches provider
instances keyed by "protocol/modelID". The fallback chain's run closure
now resolves the correct provider via the dispatcher before falling back
to agent.Provider for backward compatibility.

References sipeed#1634

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings in ProviderDispatcher fix (PR sipeed#1637 against sipeed/picoclaw).
References sipeed#1634.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ndling

Tool call detection previously relied on a literal strings.Index for
'{"tool_calls"' which failed whenever the LLM returned pretty-printed
JSON (newline after '{') or wrapped the output in markdown code fences.
Arguments typed as a JSON object instead of an encoded string also
caused a silent parse failure and leaked the raw JSON block to the user.

Changes:
- Strip markdown code fences (` + "```json" + ` / ` + "```" + `) before parsing
- Locate JSON candidate via first '{' / last '}' instead of literal match
- Unmarshal directly and check for top-level "tool_calls" key
- Accept arguments as either a JSON-encoded string or a plain JSON object
- Remove dead findMatchingBrace function and its tests
- Publish response.Content to the user immediately when a response
  contains both text and tool calls (previously the text was silently
  discarded into session history)
- Fix pre-existing test bug: TestCreateProvider_GeminiCliDefaultWorkspace
  now clears Agents.Defaults.Workspace before testing the '.' fallback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
securityguy added a commit to securityguy/picoclaw that referenced this pull request Mar 19, 2026
@sipeed-bot sipeed-bot Bot added type: bug Something isn't working domain: provider go Pull requests that update go code labels Mar 19, 2026
@sipeed-bot
Copy link
Copy Markdown

sipeed-bot Bot commented Apr 3, 2026

@securityguy Hi! The CLI tool call extraction fix has been inactive for a while. Closing it to keep things organized. If it's still relevant, just reopen and we can continue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: provider go Pull requests that update go code type: bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant