[Infra] Merge dev branch with main#25647
Conversation
[Fix] Align field-level checks in user and key update endpoints
- Reject os.environ/ references supplied via /health/test_connection request params instead of resolving them; config-sourced values are already resolved before reaching the endpoint. - Skip os.environ/ references in dynamic callback params loaded from per-request metadata. - Constrain oidc/file/ to an allowed credential directory allowlist (defaults to /var/run/secrets and /run/secrets, overridable via LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS).
- Log a warning when dropping callback params that carry os.environ/ references so operators notice the misconfiguration. - Require absolute paths in oidc/file/ and correct the documented example to use the leading-slash form. - Drop the unused return value from _reject_os_environ_references.
- Add LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS to the environment variables reference so the documentation test passes. - Annotate the values variable in _reject_os_environ_references so it accepts both dict.values() and list iterables.
…dling [Fix] tighten handling of environment references in request parameters
…ests Brings the date-range branch in line with the non-date-range branch which already hashes sk- prefixed tokens before querying. Adds coverage for filter-combination behavior in view_spend_logs.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
[Fix] /spend/logs: align filter handling with user scoping
Greptile SummaryThis infrastructure PR merges the dev branch into main, delivering a set of security hardening changes: OIDC Confidence Score: 5/5Safe to merge; all new findings are P2 style/cleanup with no impact on correctness or security. Every new finding in this review is P2 (one unreachable litellm/proxy/spend_tracking/spend_management_endpoints.py (dead
|
| Filename | Overview |
|---|---|
| litellm/secret_managers/main.py | Adds OIDC file-provider path-traversal protection via allowlist with os.path.realpath normalization and os.path.commonpath containment check — logic is correct and well-tested. |
| litellm/proxy/spend_tracking/spend_management_endpoints.py | Converts elif-chain filters to combined if-statements (security fix for internal users) and replaces legacy get_data() path with direct Prisma find_many(); contains one unreachable return None at line 2406. |
| litellm/litellm_core_utils/initialize_dynamic_callback_params.py | Replaces silent os.environ/ resolution with an explicit ValueError; adds helper _is_env_reference/_raise_env_reference_error with good test coverage. |
| litellm/proxy/health_endpoints/_health_endpoints.py | Replaces _resolve_os_environ_variables with _reject_os_environ_references (raises HTTP 400 instead of resolving); traversal and HTTPException logic are correct. |
| litellm/proxy/management_endpoints/internal_user_endpoints.py | Adds proxy-admin-only guards for user_role modification (single and bulk) and all_users bulk update path; guards are placed correctly before DB operations. |
| litellm/proxy/management_endpoints/key_management_endpoints.py | Extends max_budget admin guard to cover spend modification; mostly formatting changes; logic correct. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Request arrives] --> B{os.environ/ reference\nin request params?}
B -- Yes --> C[Raise ValueError / HTTP 400\nclear guidance returned to caller]
B -- No --> D{Endpoint type}
D -- /spend/logs --> E{User role?}
E -- INTERNAL_USER --> F[Override user_id with\nauthenticated user_id]
F --> G[Apply ALL filters simultaneously\napi_key AND request_id AND user_id]
E -- PROXY_ADMIN --> G
D -- /user/update --> H{Modifying user_role\nor all_users?}
H -- Yes --> I{Is PROXY_ADMIN?}
I -- No --> J[HTTP 403]
I -- Yes --> K[Proceed]
H -- No --> K
D -- /key/update --> L{Modifying max_budget\nor spend?}
L -- Yes --> M{Is admin role?}
M -- No --> N[HTTP 403]
M -- Yes --> K
D -- oidc/file/path --> O{Path within\nallowlist?}
O -- No --> P[ValueError: outside\nallowed credential dirs]
O -- Yes --> Q[Read token file]
Reviews (3): Last reviewed commit: "raise ValueError on os.environ/ referenc..." | Re-trigger Greptile
| if _is_env_reference(_param_value): | ||
| verbose_logger.warning( | ||
| "Dropping callback param '%s': os.environ/ references " | ||
| "in request-supplied parameters are not resolved. " | ||
| "Configure this value server-side instead.", | ||
| param, | ||
| ) | ||
| continue |
There was a problem hiding this comment.
Silent drop of
os.environ/ references breaks existing integrations
Previously, os.environ/ references in request-supplied callback params were resolved server-side; they are now silently dropped (only a verbose_logger.warning is emitted). Any user who passed {"langfuse_public_key": "os.environ/LANGFUSE_KEY"} in their request body will silently lose their observability integration with no error response and potentially no visible log, depending on their logging configuration. Per the project's backwards-compatibility rule, this should either raise a clear error or be gated behind a feature flag.
Consider raising an HTTPException or propagating an error back to the caller so the behavior change is visible, rather than continuing silently.
Rule Used: What: avoid backwards-incompatible changes without... (source)
| # Only proxy admins can modify user_role | ||
| if ( | ||
| user_request.user_role is not None | ||
| and user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN.value | ||
| ): | ||
| raise HTTPException( | ||
| status_code=403, | ||
| detail="Only proxy admins can modify user roles.", | ||
| ) |
There was a problem hiding this comment.
No unit tests for new user-role authorization guard
The new user_role-modification check (and the corresponding bulk-update guard at line ~1543) is security-sensitive — a regression here could allow privilege escalation. Per the project's CLAUDE.md requirement to always add tests for new features, please add unit tests covering at least: (1) non-admin attempting to modify user_role is rejected with 403, and (2) proxy admin is permitted to change user_role.
| # Admin-only: only proxy admins, team admins, or org admins can modify max_budget or spend | ||
| if ( | ||
| data.max_budget is not None and data.max_budget != existing_key_row.max_budget | ||
| ) or ( | ||
| data.spend is not None | ||
| and data.spend != getattr(existing_key_row, "spend", None) | ||
| ): | ||
| if prisma_client is not None: | ||
| hashed_key = existing_key_row.token | ||
| await _check_key_admin_access( | ||
| user_api_key_dict=user_api_key_dict, | ||
| hashed_token=hashed_key, | ||
| prisma_client=prisma_client, | ||
| user_api_key_cache=user_api_key_cache, | ||
| route="/key/update (max_budget)", | ||
| route="/key/update (max_budget/spend)", | ||
| ) |
There was a problem hiding this comment.
Admin check for
spend modification is skipped when prisma_client is None
The guard that enforces admin-only access for changing spend (and max_budget) is nested inside if prisma_client is not None:. If no database is connected, the check is silently bypassed and the update proceeds. This inherits the same gap that existed for max_budget before this PR, so it is pre-existing; however, extending the guard to cover spend makes the omission more significant. Consider raising early if prisma_client is None when an admin-level field is being modified.
| with open(oidc_aud, "r") as f: | ||
| # Load token from a file within an allowed credential directory. | ||
| safe_path = _resolve_oidc_file_path(oidc_aud) | ||
| with open(safe_path, "r") as f: |
…ck params Previously these were silently dropped with a verbose warning, which could break observability integrations without surfacing a clear error. Now raises ValueError with remediation steps (configure server-side or pass the resolved value) so callers get immediate, actionable feedback.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🚄 Infrastructure
Changes