fix(logging): preserve provider response headers in StandardLoggingPayload#25807
Conversation
…AdditionalHeaders
…ngPayload get_additional_headers() was only copying the 4 typed fields from StandardLoggingAdditionalHeaders, silently dropping everything else — including llm_provider-x-request-id and other provider-specific headers. Now it copies all remaining headers verbatim after handling the typed fields. Fixes #22341
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis PR fixes Confidence Score: 5/5Safe to merge — clean bug fix with appropriate test coverage and no regressions. All findings are P2 or lower. The core logic is correct: typed fields are populated first with int coercion, and remaining headers are copied verbatim using a case-sensitive key lookup that mirrors the pre-existing behavior. The TypedDict is now aligned with OPENAI_RESPONSE_HEADERS. Tests adequately cover the happy path, None input, and the reset-field case. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/litellm_core_utils/litellm_logging.py | Core fix: get_additional_headers() now builds a typed_keys map then copies all unrecognised headers verbatim; remaining diff is Black reformatting with no semantic changes. |
| litellm/types/utils.py | Adds x_ratelimit_reset_requests: str and x_ratelimit_reset_tokens: str to StandardLoggingAdditionalHeaders, completing alignment with OPENAI_RESPONSE_HEADERS. |
| tests/logging_callback_tests/test_standard_logging_payload.py | Existing test updated from exact-equality assertion (which was encoding the broken behavior) to individual field checks plus new provider-header assertions; test input was already rich but old assertion masked the bug. |
| tests/test_litellm/litellm_core_utils/test_litellm_logging.py | Adds three focused unit tests for get_additional_headers (provider-header preservation, None input, reset fields) alongside Black import-style cleanup; no real network calls. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["get_additional_headers(additiona_headers)"] --> B{"Is None?"}
B -- Yes --> C["return None"]
B -- No --> D["Build typed_keys map from annotations"]
D --> E["Loop over annotated keys"]
E --> F{"_key present in additiona_headers?"}
F -- Yes --> G["Coerce to int, fallback to str"]
G --> H["Store as typed field"]
F -- No --> I["Skip"]
H --> J["Next annotated key"]
I --> J
J -- more --> E
J -- done --> K["Loop over all additiona_headers items"]
K --> L{"k.lower() already in typed_keys?"}
L -- Yes --> M["Skip - already handled"]
L -- No --> N["Copy verbatim e.g. llm_provider-x-request-id"]
M --> O["Next header"]
N --> O
O -- more --> K
O -- done --> P["return merged result"]
Reviews (2): Last reviewed commit: "fix(logging): update test_get_additional..." | Re-trigger Greptile
| except (ValueError, TypeError): | ||
| verbose_logger.debug( | ||
| f"Could not convert {additiona_headers[_key]} to int for key {key}." | ||
| ) | ||
| additional_logging_headers[key] = additiona_headers[_key] # type: ignore |
There was a problem hiding this comment.
Removed debug log reduces int-parse observability
The original code emitted verbose_logger.debug(...) when a typed header value couldn't be cast to int, which helped diagnose unexpected provider response header formats. The new fallback silently stores the raw string instead — correct behavior, but the diagnostic signal is gone. Consider keeping a debug-level log alongside the fallback:
| except (ValueError, TypeError): | |
| verbose_logger.debug( | |
| f"Could not convert {additiona_headers[_key]} to int for key {key}." | |
| ) | |
| additional_logging_headers[key] = additiona_headers[_key] # type: ignore | |
| except (ValueError, TypeError): | |
| verbose_logger.debug( | |
| f"Could not convert {additiona_headers[_key]} to int for key {key}; storing as string." | |
| ) | |
| additional_logging_headers[key] = additiona_headers[_key] # type: ignore |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…header passthrough
7a6b7ad
into
litellm_internal_staging
Relevant issues
Fixes #22341
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
🐛 Bug Fix
Changes
hidden_params.additional_headersinStandardLoggingPayloadwas alwaysnullfor every request. Two root causes:get_additional_headers()only iterated over the 4 annotated fields inStandardLoggingAdditionalHeaders, dropping everything else — includingllm_provider-x-request-idand other provider-specific headers.StandardLoggingAdditionalHeaderswas missingx_ratelimit_reset_requestsandx_ratelimit_reset_tokenswhich are already inOPENAI_RESPONSE_HEADERS.Fix: after populating the typed fields, copy all remaining headers verbatim. Also add the two missing reset fields to the TypedDict.
Screenshots / Proof of Fix
E2E test: local proxy (
openai/fake-openai-endpoint) with a custom logging callback capturingStandardLoggingPayload.Before fix —
additional_headerswas alwaysnull.After fix — callback receives:
{ "additional_headers": { "llm_provider-x-railway-request-id": "hyvzt6mcTS2hjQDs9I3ezw", "llm_provider-server": "railway-edge", "llm_provider-date": "Wed, 15 Apr 2026 18:26:38 GMT", "llm_provider-content-type": "application/json", "llm_provider-x-railway-cdn-edge": "fastly/cache-pao-kpao1770021-PAO", "llm_provider-x-cache": "MISS", "x-litellm-model-group": "fake-model" } }Provider request IDs (e.g. OpenAI's
x-request-id) now persist to S3, SpendLogs, and all other logging backends.