fix: emit input_json_delta for tool args bundled in first streaming chunk#25533
Conversation
…hunk
Some providers (xAI, Gemini) include tool_call function arguments in the
same streaming chunk as the function name/id. The AnthropicStreamWrapper
was discarding the trigger chunk entirely when starting a new content
block, which silently dropped the input_json_delta carrying tool
arguments. This caused tool_use blocks to arrive with empty input {}.
Now queue the processed_chunk after content_block_start when it carries
non-empty input_json_delta data. Backward compatible: providers that send
empty arguments in the first chunk (OpenAI-style) are unaffected since
the condition checks for truthy partial_json.
Covers the fix for providers (xAI, Gemini) that bundle tool_call arguments in the same streaming chunk as the function name/id. Verifies the AnthropicStreamWrapper emits input_json_delta after content_block_start, and that empty-arg chunks (OpenAI-style) are unaffected.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes a silent data-loss bug in Confidence Score: 5/5Safe to merge — the fix is minimal, correctly scoped, and all four test scenarios pass. No P0 or P1 findings. The condition chain properly handles all edge cases: empty-string args ("") is falsy, None args, and non-empty args. Symmetric application in both sync and async paths is correct. Tests are pure-mock (no network calls), cover both the happy path and the backward-compatible empty-args path, and previously noted backward-compat weaknesses have been addressed with exact-count list-comprehension assertions. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py | Adds a guard to queue input_json_delta from the trigger chunk when it carries non-empty partial_json, in both sync and async iteration paths; logic is correct and backward-compatible. |
| tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_tool_args.py | Four new mock-only tests covering sync/async × bundled-args/empty-args scenarios; no real network calls, assertions correctly verify event ordering and exact counts. |
Sequence Diagram
sequenceDiagram
participant Provider as Provider (xAI/Gemini)
participant Wrapper as AnthropicStreamWrapper
participant Client as Anthropic Client
Provider->>Wrapper: chunk[text, content="Hello"]
Wrapper->>Client: message_start
Wrapper->>Client: content_block_start (text, index=0)
Wrapper->>Client: content_block_delta (text_delta)
Provider->>Wrapper: chunk[tool_call: name="get_weather", arguments='{"location":"Boston"}']
Note over Wrapper: should_start_new_block=True<br/>_increment_content_block_index()
Note over Wrapper: translate → content_block_delta (input_json_delta)
Wrapper->>Client: content_block_stop (index=0)
Wrapper->>Client: content_block_start (tool_use, index=1)
Note over Wrapper: NEW: partial_json is truthy → queue delta
Wrapper->>Client: content_block_delta (input_json_delta, partial_json='{"location":"Boston"}')
Provider->>Wrapper: chunk[finish_reason="tool_calls"]
Wrapper->>Client: content_block_stop (index=1)
Wrapper->>Client: message_delta (stop_reason="tool_use")
Wrapper->>Client: message_stop
Reviews (4): Last reviewed commit: "test: make no_extra_delta tests assert e..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| assert tool_start_idx is not None | ||
|
|
||
| # The event immediately after content_block_start should NOT be | ||
| # an input_json_delta from the trigger chunk (since arguments were empty). | ||
| # It should be an input_json_delta from the subsequent tool_args_chunk. | ||
| next_event = events[tool_start_idx + 1] | ||
| if ( | ||
| isinstance(next_event, dict) | ||
| and next_event.get("type") == "content_block_delta" | ||
| and isinstance(next_event.get("delta"), dict) | ||
| and next_event["delta"].get("type") == "input_json_delta" | ||
| ): | ||
| # This delta must come from tool_args_chunk, not tool_name_chunk | ||
| assert next_event["delta"].get("partial_json") == '{"location": "NYC"}' |
There was a problem hiding this comment.
The if on line 226 means if next_event is not a content_block_delta (e.g. it's a content_block_stop or some other event), the inner assert never runs and the test passes silently without verifying anything. The test title promises "no_extra_delta_when_tool_args_empty" but doesn't enforce that constraint. Consider asserting directly that no spurious delta was injected:
| assert tool_start_idx is not None | |
| # The event immediately after content_block_start should NOT be | |
| # an input_json_delta from the trigger chunk (since arguments were empty). | |
| # It should be an input_json_delta from the subsequent tool_args_chunk. | |
| next_event = events[tool_start_idx + 1] | |
| if ( | |
| isinstance(next_event, dict) | |
| and next_event.get("type") == "content_block_delta" | |
| and isinstance(next_event.get("delta"), dict) | |
| and next_event["delta"].get("type") == "input_json_delta" | |
| ): | |
| # This delta must come from tool_args_chunk, not tool_name_chunk | |
| assert next_event["delta"].get("partial_json") == '{"location": "NYC"}' | |
| next_event = events[tool_start_idx + 1] | |
| # The event after content_block_start must NOT be an input_json_delta | |
| # originating from the (empty-args) trigger chunk. | |
| if ( | |
| isinstance(next_event, dict) | |
| and next_event.get("type") == "content_block_delta" | |
| and isinstance(next_event.get("delta"), dict) | |
| and next_event["delta"].get("type") == "input_json_delta" | |
| ): | |
| # If there is a delta here it must come from tool_args_chunk, not the empty trigger | |
| assert next_event["delta"].get("partial_json") == '{"location": "NYC"}', ( | |
| "Spurious empty input_json_delta emitted from trigger chunk" | |
| ) | |
| else: | |
| pass | |
| # Separately, assert the args delta from tool_args_chunk is present somewhere | |
| all_input_json_deltas = [ | |
| e for e in events | |
| if isinstance(e, dict) | |
| and e.get("type") == "content_block_delta" | |
| and isinstance(e.get("delta"), dict) | |
| and e["delta"].get("type") == "input_json_delta" | |
| ] | |
| assert any( | |
| d["delta"].get("partial_json") == '{"location": "NYC"}' for d in all_input_json_deltas | |
| ), "Expected tool_args_chunk delta to appear in events" |
|
Tip: Greploop — Automatically fix all review issues by running Use the Greptile plugin for Claude Code to query reviews, search comments, and manage custom context directly from your terminal. |
|
Added Screenshot after the patch to fix the issue 
|
krrish-berri-2
changed the base branch from
main
to
litellm_oss_staging_04_13_2026_p1
krrish-berri-2
merged commit bf3ed8d
into
BerriAI:litellm_oss_staging_04_13_2026_p1
…hunk (#25533) * fix: emit input_json_delta for tool args bundled in first streaming chunk Some providers (xAI, Gemini) include tool_call function arguments in the same streaming chunk as the function name/id. The AnthropicStreamWrapper was discarding the trigger chunk entirely when starting a new content block, which silently dropped the input_json_delta carrying tool arguments. This caused tool_use blocks to arrive with empty input {}. Now queue the processed_chunk after content_block_start when it carries non-empty input_json_delta data. Backward compatible: providers that send empty arguments in the first chunk (OpenAI-style) are unaffected since the condition checks for truthy partial_json. * test: add tests for input_json_delta emission on bundled tool args Covers the fix for providers (xAI, Gemini) that bundle tool_call arguments in the same streaming chunk as the function name/id. Verifies the AnthropicStreamWrapper emits input_json_delta after content_block_start, and that empty-arg chunks (OpenAI-style) are unaffected. * style: apply Black formatting to streaming_iterator.py * fix: mirror input_json_delta fix to sync __next__ and add sync tests * test: make no_extra_delta tests assert explicitly instead of passing silently
…hunk (BerriAI#25533) * fix: emit input_json_delta for tool args bundled in first streaming chunk Some providers (xAI, Gemini) include tool_call function arguments in the same streaming chunk as the function name/id. The AnthropicStreamWrapper was discarding the trigger chunk entirely when starting a new content block, which silently dropped the input_json_delta carrying tool arguments. This caused tool_use blocks to arrive with empty input {}. Now queue the processed_chunk after content_block_start when it carries non-empty input_json_delta data. Backward compatible: providers that send empty arguments in the first chunk (OpenAI-style) are unaffected since the condition checks for truthy partial_json. * test: add tests for input_json_delta emission on bundled tool args Covers the fix for providers (xAI, Gemini) that bundle tool_call arguments in the same streaming chunk as the function name/id. Verifies the AnthropicStreamWrapper emits input_json_delta after content_block_start, and that empty-arg chunks (OpenAI-style) are unaffected. * style: apply Black formatting to streaming_iterator.py * fix: mirror input_json_delta fix to sync __next__ and add sync tests * test: make no_extra_delta tests assert explicitly instead of passing silently
3 participants
Summary
Some providers (xAI, Gemini) include tool_call function arguments in the same streaming chunk as the function name/id. The
AnthropicStreamWrapperwas discarding the trigger chunk entirely when starting a new content block, which silently dropped theinput_json_deltacarrying tool arguments. This causedtool_useblocks to arrive with emptyinput {}.Now queue the
processed_chunkaftercontent_block_startwhen it carries non-emptyinput_json_deltadata. The fix is applied to both the sync__next__and async__anext__iteration paths. Backward compatible: providers that send empty arguments in the first chunk (OpenAI-style) are unaffected since the condition checks for truthypartial_json.Changes
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py: After emittingcontent_block_stop+content_block_startfor a new tool_use block, also emit the trigger chunk'scontent_block_deltawhen it carriesinput_json_deltawith non-emptypartial_json. Applied to both__next__(sync) and__anext__(async).tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_tool_args.py: Added 4 tests covering both sync and async paths for bundled args (xAI/Gemini style) and empty args (OpenAI style)Test plan
test_async_stream_emits_input_json_delta_for_bundled_tool_args- verifies asyncinput_json_deltais emitted aftercontent_block_startwhen tool args are bundledtest_async_stream_no_extra_delta_when_tool_args_empty- verifies async backward compatibility when args are empty (OpenAI-style)test_sync_stream_emits_input_json_delta_for_bundled_tool_args- verifies syncinput_json_deltais emitted aftercontent_block_startwhen tool args are bundledtest_sync_stream_no_extra_delta_when_tool_args_empty- verifies sync backward compatibility when args are empty (OpenAI-style)