fix(bedrock): avoid double-counting cache tokens in Anthropic Messages streaming usage

[Sameerlite]

Problem

For Bedrock Invoke Anthropic Messages streaming (bedrock_sse_wrapper → _promote_message_stop_usage), cached requests could report roughly 2× the real prompt token total and inflated spend.

What went wrong

1. On message_stop, Bedrock sends uncached input_tokens (e.g. 3) plus cache breakdown on the merged message_delta (cache_creation_input_tokens, cache_read_input_tokens).
2. _promote_message_stop_usage was rewriting message_delta.usage.input_tokens as uncached + cache_creation + cache_read (e.g. 3 + 17 + 32651 = 32671).
3. Downstream, AnthropicConfig.calculate_usage treats input_tokens as uncached-only and adds cache read/write again to prompt_tokens.
4. Result: cache tokens were included in input_tokens and added again → double counting (e.g. prompt_tokens 65339 instead of 32671 for the same request).

Fix

Keep message_delta.usage.input_tokens as the uncached-only value from message_stop (raw_input). Still promote cache_creation_input_tokens and cache_read_input_tokens onto message_delta for clients that ignore message_stop. calculate_usage then adds cache to prompt_tokens exactly once.

Tests

• Assert promoted message_delta keeps uncached input_tokens == 3 with cache fields present.
• End-to-end: SSE through bedrock_sse_wrapper → passthrough logging rebuild → prompt_tokens and completion_cost for us.anthropic.claude-sonnet-4-6.
• Updated test_bedrock_sse_wrapper_keeps_usage_in_message_start_and_message_delta expectations for input_tokens after promotion.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 10, 2026 6:35pm

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_bedrock-messages-cache-prompt-double-count (f0d2d26) with main (d0e347a)}

[greptile-apps]

Greptile Summary

This PR fixes a double-counting bug in _promote_message_stop_usage for Bedrock Invoke Anthropic Messages streaming: the old code summed uncached + cache_creation + cache_read into input_tokens, but calculate_usage then added the cache tokens again to prompt_tokens, causing ~2× inflation. The fix keeps input_tokens as the uncached-only count from message_stop, relying on calculate_usage to add cache fields once. The logic change is minimal and correct, and the three new tests (unit, preservation, and end-to-end cost) provide solid coverage.

Confidence Score: 5/5

Safe to merge — the fix is correct, minimal, and well-tested; the only remaining finding is a style nit.

The bug fix is logically correct (remove summation that caused double-counting), the updated test reflects real corrected behavior rather than masking a regression, and three new tests give good coverage including an end-to-end cost check. The sole remaining finding is a P2 import style issue that does not affect correctness or CI.

No files require special attention beyond the minor import style cleanup in the test file.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py	Removes the summation of cache tokens into input_tokens; now passes uncached count through unchanged so calculate_usage adds cache once.
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Updates existing assertion to reflect fixed behavior; adds two new tests, one of which imports proxy code inside the function body (violates module-level import style).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Bedrock Stream] -->|message_delta with cache fields| B[_promote_message_stop_usage]
    B -->|buffer pending_delta| C{message_stop arrives?}
    C -->|No| D[yield pending_delta as-is]
    C -->|Yes| E[Copy cache_creation and cache_read from stop into delta_usage]
    E --> F[Set delta input_tokens to uncached count from message_stop]
    F --> G[Yield merged message_delta]
    G --> H[calculate_usage]
    H --> I[prompt_tokens equals uncached plus cache_creation plus cache_read]
    I --> J[No double-counting]

Loading

_{Reviews (1): Last reviewed commit: "fix(bedrock): avoid double-counting cach..." | Re-trigger Greptile}