[Feature] UI - Settings: Claude Code BYOK support#25998
[Feature] UI - Settings: Claude Code BYOK support#25998yuneng-berri merged 9 commits intolitellm_internal_stagingfrom
Conversation
…event recursive-loop misconfiguration
…ions (independent of each other)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…itellm_/wonderful-bouman # Conflicts: # tests/test_litellm/proxy/ui_crud_endpoints/test_proxy_setting_endpoints.py
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 29203065 | Triggered | JSON Web Token | e004876 | tests/test_litellm/proxy/test_litellm_pre_call_utils.py | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Greptile SummaryThis PR exposes the Claude Code BYOK configuration in the admin UI by adding a
Confidence Score: 4/5Safe to merge for Anthropic BYOK (x-api-key path is correct), but anti-replay gap for other provider auth headers should be fixed before extending BYOK to Azure/Google providers. One P1 security finding in the auth layer: the incomplete authenticated_with_header guard leaves api-key and other special headers unprotected when BYOK is enabled. Primary Anthropic use case is correctly implemented; all other changes are clean. litellm/proxy/litellm_pre_call_utils.py — both the clean_headers third elif branch (lines 326–336) and the authenticated_with_header detection block (lines 950–959) need the anti-replay guard extended to all provider auth headers.
|
| Filename | Overview |
|---|---|
| litellm/proxy/litellm_pre_call_utils.py | New clean_headers parameters add BYOK forwarding; x-api-key anti-replay guard is correct, but other provider auth headers lack the same guard and authenticated_with_header detection has an incorrect hardcoded fallback. |
| litellm/proxy/ui_crud_endpoints/proxy_setting_endpoints.py | Adds forward_llm_provider_auth_headers to UISettings model, ALLOWED_UI_SETTINGS_FIELDS, and _RUNTIME_GENERAL_SETTINGS_FLAGS; follows existing pattern exactly. |
| litellm/proxy/public_endpoints/provider_create_fields.json | Makes Anthropic api_key optional with BYOK tooltip, adds api_base field before api_key — both intentional for BYOK. |
| ui/litellm-dashboard/src/components/Settings/AdminSettings/UISettings/UISettings.tsx | Adds Switch row for forward_llm_provider_auth_headers using antd components, following exact existing pattern. |
| tests/test_litellm/proxy/test_litellm_pre_call_utils.py | Three mock-only unit tests covering BYOK x-api-key forwarding, stripping, and anti-replay guard. |
| tests/test_litellm/proxy/ui_crud_endpoints/test_proxy_setting_endpoints.py | Two mock-based tests verifying forward_llm_provider_auth_headers persistence and general_settings sync. |
| docs/my-website/docs/tutorials/claude_code_byok.md | Docs-only addition of UI configuration path for BYOK. |
Comments Outside Diff (1)
-
litellm/proxy/litellm_pre_call_utils.py, line 326-336 (link)Anti-replay guard incomplete for other provider auth headers
This
elifbranch forwardsapi-key(Azure), the Google AI Studio header, and the Azure APIM header to upstream without checkingauthenticated_with_header. The guard that protectsx-api-key(lines 320–325) is not applied here. All three of these headers are also accepted byuser_api_key_authfor proxy authentication — so a client who usesapi-keyto auth with the proxy while BYOK is enabled will have their LiteLLM virtual key forwarded to the upstream LLM provider.There is also a gap at lines 950–959: when neither
x-litellm-api-keynorauthorizationis present, theelsebranch always writes"x-api-key"as the fallback regardless of which header actually carried the proxy credential. This means the guard would still not fire correctly forapi-keyeven after being added here.Two changes are needed:
- Add the auth-header guard to this branch (same pattern used for
x-api-key):
if ( authenticated_with_header is not None and header_lower == authenticated_with_header.lower() ): continue clean_headers[header] = value
- Detect each provider header explicitly at the call site instead of defaulting to a hardcoded string:
if "x-litellm-api-key" in request.headers: authenticated_with_header = "x-litellm-api-key" elif "authorization" in request.headers: authenticated_with_header = "authorization" elif "api-key" in request.headers: authenticated_with_header = "api-key" elif "x-api-key" in request.headers: authenticated_with_header = "x-api-key" else: authenticated_with_header = None
Rule Used: What: Fail any PR which may contains a security in... (source)
- Add the auth-header guard to this branch (same pattern used for
Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
18a5fe5
into
litellm_internal_staging
Relevant issues
(none)
Summary
Exposes Claude Code BYOK (bring-your-own-key) configuration in the admin UI so admins no longer need to edit
config.yamlto enable it.Problem
The Claude Code BYOK tutorial (
docs/my-website/docs/tutorials/claude_code_byok.md) has two proxy-side config requirements that were previously only settable via YAML:api_key(so the client's forwarded key is used).forward_llm_provider_auth_headers: true.Neither was achievable from the UI. The add-model form blocked empty
api_key(Anthropic was markedrequired: true), and the BYOK flag was only readable fromconfig.yaml. Admins without env-var or file access had no way to complete the setup.What this does
forward_llm_provider_auth_headerstoUISettings(Pydantic model + allowlist + runtime sync list) and a toggle row on the UI Settings page, following the existingforward_client_headers_to_llm_apipattern (DB-override-config vialitellm_uisettings).api_keytorequired: falseinprovider_create_fields.jsonwith a BYOK-explaining tooltip.api_basefield to the Anthropic entry (matches AI21 / ANTHROPIC_TEXT), with a tooltip that explicitly warns against setting it to the proxy URL.Changes
UISettings.tsx+proxy_setting_endpoints.pyadd the toggle and its wiring.provider_create_fields.jsonrelaxes the Anthropicapi_keyrequirement and addsapi_base. Tests cover the settings persistence + runtime sync, theclean_headersBYOK forwarding path (including the anti-replay guard wherex-api-keywas used for proxy auth), and the Anthropic provider form shape/ordering.Testing
New unit tests:
test_update_ui_settings_persists_forward_llm_provider_auth_headerstest_update_ui_settings_syncs_forward_llm_provider_auth_headers_to_general_settingstest_clean_headers_preserves_x_api_key_when_byok_enabledtest_clean_headers_strips_x_api_key_when_byok_disabledtest_clean_headers_strips_x_api_key_when_byok_enabled_but_x_api_key_was_auth_headertest_anthropic_provider_fields_support_byokFull test files pass:
test_proxy_setting_endpoints.py,test_public_endpoints.py,test_litellm_pre_call_utils.py.ui/litellm-dashboardbuilds withnpm run build;npx tsc --noEmitclean on the modified file.Verified end-to-end with a local proxy + Docker Postgres:
GET /get/ui_settingsschema exposes the togglePATCH /update/ui_settingspersists and round-trips via DBGET /public/providers/fieldsreturns Anthropic withapi_key.required=falseandapi_basepresent, ordered beforeapi_key/loginusing an Anthropic API key routes through LiteLLM, forwards the client's key to Anthropic, and logs the call under the virtual keyMissing Anthropic API Keyerror as expectedType
🆕 New Feature
✅ Test
📖 Documentation
Screenshots
(to be added: the two toggles on Settings → UI Settings, and the Anthropic Add Model form showing the optional API Key and new Upstream API Base fields)