Skip to content

fix(logging): preserve provider response headers in StandardLoggingPayload#25807

Merged
ishaan-berri merged 4 commits intolitellm_internal_stagingfrom
litellm_fix_provider_headers_in_logging
Apr 16, 2026
Merged

fix(logging): preserve provider response headers in StandardLoggingPayload#25807
ishaan-berri merged 4 commits intolitellm_internal_stagingfrom
litellm_fix_provider_headers_in_logging

Conversation

@ishaan-berri
Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #22341

Pre-Submission checklist

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

hidden_params.additional_headers in StandardLoggingPayload was always null for every request. Two root causes:

  1. get_additional_headers() only iterated over the 4 annotated fields in StandardLoggingAdditionalHeaders, dropping everything else — including llm_provider-x-request-id and other provider-specific headers.
  2. StandardLoggingAdditionalHeaders was missing x_ratelimit_reset_requests and x_ratelimit_reset_tokens which are already in OPENAI_RESPONSE_HEADERS.

Fix: after populating the typed fields, copy all remaining headers verbatim. Also add the two missing reset fields to the TypedDict.

Screenshots / Proof of Fix

E2E test: local proxy (openai/fake-openai-endpoint) with a custom logging callback capturing StandardLoggingPayload.

Before fix — additional_headers was always null.

After fix — callback receives:

{
  "additional_headers": {
    "llm_provider-x-railway-request-id": "hyvzt6mcTS2hjQDs9I3ezw",
    "llm_provider-server": "railway-edge",
    "llm_provider-date": "Wed, 15 Apr 2026 18:26:38 GMT",
    "llm_provider-content-type": "application/json",
    "llm_provider-x-railway-cdn-edge": "fastly/cache-pao-kpao1770021-PAO",
    "llm_provider-x-cache": "MISS",
    "x-litellm-model-group": "fake-model"
  }
}

Provider request IDs (e.g. OpenAI's x-request-id) now persist to S3, SpendLogs, and all other logging backends.

…ngPayload

get_additional_headers() was only copying the 4 typed fields from
StandardLoggingAdditionalHeaders, silently dropping everything else —
including llm_provider-x-request-id and other provider-specific headers.

Now it copies all remaining headers verbatim after handling the typed fields.
Fixes #22341
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 15, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 15, 2026 7:25pm

Request Review

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 15, 2026

Greptile Summary

This PR fixes StandardLoggingPayload.hidden_params.additional_headers always being null by overhauling get_additional_headers() in two ways: (1) after populating the annotated typed fields, all remaining headers are now copied verbatim (preserving provider headers like llm_provider-x-request-id), and (2) the two missing reset fields (x_ratelimit_reset_requests, x_ratelimit_reset_tokens) are added to StandardLoggingAdditionalHeaders to align with OPENAI_RESPONSE_HEADERS. The bulk of the diff is Black reformatting with no semantic impact.

Confidence Score: 5/5

Safe to merge — clean bug fix with appropriate test coverage and no regressions.

All findings are P2 or lower. The core logic is correct: typed fields are populated first with int coercion, and remaining headers are copied verbatim using a case-sensitive key lookup that mirrors the pre-existing behavior. The TypedDict is now aligned with OPENAI_RESPONSE_HEADERS. Tests adequately cover the happy path, None input, and the reset-field case.

No files require special attention.

Important Files Changed

Filename Overview
litellm/litellm_core_utils/litellm_logging.py Core fix: get_additional_headers() now builds a typed_keys map then copies all unrecognised headers verbatim; remaining diff is Black reformatting with no semantic changes.
litellm/types/utils.py Adds x_ratelimit_reset_requests: str and x_ratelimit_reset_tokens: str to StandardLoggingAdditionalHeaders, completing alignment with OPENAI_RESPONSE_HEADERS.
tests/logging_callback_tests/test_standard_logging_payload.py Existing test updated from exact-equality assertion (which was encoding the broken behavior) to individual field checks plus new provider-header assertions; test input was already rich but old assertion masked the bug.
tests/test_litellm/litellm_core_utils/test_litellm_logging.py Adds three focused unit tests for get_additional_headers (provider-header preservation, None input, reset fields) alongside Black import-style cleanup; no real network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["get_additional_headers(additiona_headers)"] --> B{"Is None?"}
    B -- Yes --> C["return None"]
    B -- No --> D["Build typed_keys map from annotations"]
    D --> E["Loop over annotated keys"]
    E --> F{"_key present in additiona_headers?"}
    F -- Yes --> G["Coerce to int, fallback to str"]
    G --> H["Store as typed field"]
    F -- No --> I["Skip"]
    H --> J["Next annotated key"]
    I --> J
    J -- more --> E
    J -- done --> K["Loop over all additiona_headers items"]
    K --> L{"k.lower() already in typed_keys?"}
    L -- Yes --> M["Skip - already handled"]
    L -- No --> N["Copy verbatim e.g. llm_provider-x-request-id"]
    M --> O["Next header"]
    N --> O
    O -- more --> K
    O -- done --> P["return merged result"]
Loading

Reviews (2): Last reviewed commit: "fix(logging): update test_get_additional..." | Re-trigger Greptile

Comment on lines 4992 to +4993
except (ValueError, TypeError):
verbose_logger.debug(
f"Could not convert {additiona_headers[_key]} to int for key {key}."
)
additional_logging_headers[key] = additiona_headers[_key] # type: ignore
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Removed debug log reduces int-parse observability

The original code emitted verbose_logger.debug(...) when a typed header value couldn't be cast to int, which helped diagnose unexpected provider response header formats. The new fallback silently stores the raw string instead — correct behavior, but the diagnostic signal is gone. Consider keeping a debug-level log alongside the fallback:

Suggested change
except (ValueError, TypeError):
verbose_logger.debug(
f"Could not convert {additiona_headers[_key]} to int for key {key}."
)
additional_logging_headers[key] = additiona_headers[_key] # type: ignore
except (ValueError, TypeError):
verbose_logger.debug(
f"Could not convert {additiona_headers[_key]} to int for key {key}; storing as string."
)
additional_logging_headers[key] = additiona_headers[_key] # type: ignore

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Collaborator

@ryan-crabbe-berri ryan-crabbe-berri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ishaan-berri ishaan-berri merged commit 7a6b7ad into litellm_internal_staging Apr 16, 2026
60 of 63 checks passed
@ishaan-berri ishaan-berri deleted the litellm_fix_provider_headers_in_logging branch April 16, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Provider response headers (e.g. x-request-id) not persisted in StandardLoggingPayload / S3 / SpendLogs

3 participants