[Infra] Merge dev branch#25871
Merged
yuneng-berri merged 26 commits intolitellm_internal_stagingfrom Apr 16, 2026
Merged
Conversation
Gemini API keys embedded in URLs as ?key= query parameters leak through
httpx error tracebacks, which are then captured by traceback.format_exc()
and forwarded to logging callbacks, Slack/Teams alerts, and HTTP client
responses.
Short-term: all httpx.HTTPStatusError handlers now raise
MaskedHTTPStatusError(...) from None, which masks the URL and breaks
exception chaining so the original error never appears in tracebacks.
Long-term: moved all Gemini/Vertex URL constructions from ?key={api_key}
to x-goog-api-key header (Google's documented auth method), so the key
is never in the URL at all. WebSocket realtime is the only exception
since WS clients cannot use custom headers.
Additionally hardened all outbound credential paths:
- WebSocket close reasons now pass through _redact_string()
- Callback pipeline (failure_handler) redacts traceback_exception and
error_str before forwarding to integrations (Langfuse, Datadog, etc.)
- Slack/Teams alert messages redacted in send_llm_exception_alert,
ProxyLogging.failure_handler, and post_call_failure_hook
- HTTP error responses in proxy SSE and health endpoints redacted
- Exception messages in exception_mapping_utils redacted
- print_verbose() stdout output redacted when set_verbose=True
- HTTPHandler.put() now has MaskedHTTPStatusError (was missing)
Addresses Greptile review — the async streaming path was missing mask_sensitive_info() on the response body, while the sync path had it.
Add 50 tests across 3 files covering the new MaskedHTTPStatusError, safe response helpers, _redact_string in error paths, Gemini interactions x-goog-api-key header auth, and RAG ingestion header usage. Fix missing early-validation for Gemini API key in _get_token_and_url() which caused TypeError when key was None (headers got None value). Harmonize error messages between the two validation sites. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
security: prevent API key leaks in error tracebacks, logs, and alerts
…ictedpython fix(guardrails): replace custom_code sandbox with RestrictedPython
Tighten validation of request body parameters in the proxy routing layer. Use context variables for internal call state management instead of passing flags through request kwargs. Clean up metadata handling at the proxy boundary.
Add safe_join() and safe_filename() in proxy/common_utils/path_utils.py for constructing filesystem paths from user-controlled inputs. Apply to guardrail category YAML endpoint and dotprompt file converter.
Add null byte rejection to safe_join and safe_filename. Normalize backslash separators in safe_filename for cross-platform safety. Include resolved path in ValueError for debugging. Move imports to module level per project conventions.
fix(proxy): harden request parameter handling
fix(proxy): add shared path utilities, prevent directory traversal
- vertex_ai_context_caching.py: add explicit Optional[str] annotation on auth_header so later branches that assign vertex_auth_header (Optional[str]) type-check against the first branch's dict assignment (which already has type: ignore[assignment]). - path_utils.py: remove unused pathlib.Path import (F401). - emulated_handler.py: extract _extract_tool_call_fields, _resolve_queries_from_args, _execute_file_search_tool_calls, and _build_follow_up_input helpers to drop aresponses_with_emulated_file_search below ruff's PLR0915 statement limit. Behavior unchanged.
RestrictedPython (ZPL-2.1, a BSD-style permissive license) was added as a dependency for the custom_code guardrail sandbox, but the license checker didn't recognize it. Add to authorized packages list.
…HTTPStatusError
MaskedHTTPStatusError constructs a new httpx.Response from the original
error. Two bugs surfaced under real HTTP error responses:
1. The new Response was created without request=, so response.request
raised RuntimeError("The .request property has not been set.") for
any downstream caller (e.g. exception_mapping_utils) that inspected it.
2. The decoded response bytes were passed together with the original
Content-Encoding header. On construction httpx tried to decompress
the already-decoded bytes and raised httpx.DecodingError
("Error -3 while decompressing data: incorrect header check").
Set response.request to the masked Request and strip Content-Encoding
(and the now-stale Content-Length) before rebuilding the Response.
URL/message masking is unchanged; the new request carries the already
masked URL.
Also update test_logging_key_masking_gemini: the security commit
25f93be moved Gemini API keys from ?key=... URL params to the
x-goog-api-key header, so api_base no longer contains the key.
…itellm_yj_apr15 # Conflicts: # litellm/litellm_core_utils/litellm_logging.py # uv.lock
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis dev branch merge introduces several security hardening and infrastructure improvements: (1) a RestrictedPython-based sandbox for custom code guardrails replacing the old regex-based validation, (2) path-traversal prevention via Confidence Score: 5/5Safe to merge — all changes are security improvements with no regressions identified. No P0 or P1 issues found. The sandbox migration from regex to RestrictedPython is a clear security improvement with comprehensive test coverage. Path-traversal guards, credential masking, and secret-redaction expansions are all correctly implemented. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/custom_code/sandbox.py | New RestrictedPython sandbox; async allowed via AsyncAwareTransformer; correct guard setup (safer_getattr, full_write_guard, default_guarded_getitem/iter) |
| litellm/proxy/common_utils/path_utils.py | New safe_join/safe_filename helpers; realpath + containment check correctly prevents directory traversal and null-byte injection |
| litellm/_logging.py | SecretRedactionFilter now also redacts exception tracebacks and extra fields; _redact_string respects LITELLM_DISABLE_REDACT_SECRETS flag |
| litellm/llms/custom_httpx/http_handler.py | MaskedHTTPStatusError and raise_masked*_error helpers strip API keys from request URLs on HTTP errors; well-tested |
| litellm/_internal_context.py | New is_internal_call ContextVar for suppressing billing callbacks during emulated file-search sub-calls |
| litellm/proxy/guardrails/guardrail_endpoints.py | get_category_yaml now uses safe_join to prevent path traversal; sandbox helpers applied to test_custom_code_guardrail endpoint |
| litellm/responses/file_search/emulated_handler.py | is_internal_call correctly set and restored in finally blocks for both sub-calls; clean implementation |
| tests/test_litellm/proxy/guardrails/test_custom_code_security.py | Comprehensive sandbox security tests: bytecode rewrite, cr_frame, NFKC homoglyph, dunder access, import forms, runtime rejections, augmented assignment |
| litellm/litellm_core_utils/exception_mapping_utils.py | _redact_string now applied to tracebacks included in mapped exception messages |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[User submits custom guardrail code] --> B[compile_sandboxed]
B --> C{RestrictedPython\nAST transform}
C -->|Import / exec / dunder\nname / bytecode attack| D[SyntaxError raised\n→ CustomCodeCompilationError]
C -->|Clean code| E[Compiled bytecode]
E --> F[exec in build_sandbox_globals]
F --> G{apply_guardrail\ndefined?}
G -->|No| H[CustomCodeCompilationError]
G -->|Yes| I[_compiled_function stored]
I --> J[apply_guardrail called at request time]
J --> K{_getattr_ = safer_getattr\n_write_ = full_write_guard\n__builtins__ = safe_builtins}
K -->|Dunder attr access\nor unsafe write| L[AttributeError / TypeError at runtime]
K -->|Safe operation| M[allow / block / modify result]
Reviews (2): Last reviewed commit: "fix(ci): authorize langgraph-prebuilt in..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+90
to
93
| return value | ||
| return _SECRET_RE.sub(_REDACTED, value) | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
_redact_string now respects the global redaction disable flag
This is a behavioral change: previously _redact_string was unconditional. Now that it's applied in security-sensitive error paths (WebSocket close reasons, error tracebacks) added elsewhere in this PR, disabling the global redaction env var will also disable those new guards. Operators who set this flag for debugging should understand it now covers traceback and WebSocket redaction too — not just message-level redaction. Consider noting this in the env-var documentation.
langgraph-prebuilt 1.0.9 imports ExecutionInfo and ServerInfo from langgraph.runtime, but those symbols are not exported until langgraph 1.1.0. Our pin of langgraph==1.0.10 allows langgraph-prebuilt<1.1.0,>=1.0.8, and uv resolves to 1.0.9 (the latest in range), which breaks at import time in every test that touches langgraph.prebuilt (e.g. tests/pass_through_tests/test_mcp_routes.py): ImportError: cannot import name 'ExecutionInfo' from 'langgraph.runtime' Pinning langgraph-prebuilt to 1.0.8 pairs correctly with langgraph==1.0.10 and restores the import path.
yuneng-berri
temporarily deployed
to
integration-postgres
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
GitHub Actions
Inactive
yuneng-berri
had a problem deploying
to
integration-postgres
GitHub Actions
Error
yuneng-berri
had a problem deploying
to
integration-postgres
GitHub Actions
Error
langgraph-prebuilt was previously pulled in as a transitive of langgraph so PyPI license metadata was reported as unknown. Now that it is explicitly pinned (==1.0.8) to avoid the broken 1.0.9 release, the license checker flags it. It is published under MIT by the same langchain-ai/langgraph repository as langgraph itself.
yuneng-berri
temporarily deployed
to
integration-postgres
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
GitHub Actions
Inactive
yuneng-berri
had a problem deploying
to
integration-postgres
GitHub Actions
Error
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
ryan-crabbe-berri
approved these changes
Apr 16, 2026
yuneng-berri
merged commit 21c0718
into
litellm_internal_staging
98 of 100 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🚄 Infrastructure
Changes