Merge branch 'main' into litellm_oss_staging_03_11_2026

Author: Chesars

.circleci/config.yml

@@ -4337,7 +4337,8 @@ jobs:
           name: Check for expected error
           command: |
             if grep -q "Error: P1001: Can't reach database server at" docker_output.log && \
-               grep -q "ERROR:    Application startup failed. Exiting." docker_output.log; then
+               (grep -q "Database setup failed after multiple retries" docker_output.log || \
+                grep -q "ERROR:    Application startup failed. Exiting." docker_output.log); then
               echo "Expected error found. Test passed."
             else
               echo "Expected error not found. Test failed."

.github/workflows/create_daily_staging_branch.yml

@@ -41,3 +41,39 @@ jobs:
             git push origin $BRANCH_NAME
             echo "Successfully created and pushed branch: $BRANCH_NAME"
           fi
+
+  create-internal-dev-branch:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: Create internal dev branch
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          # Configure Git user
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+          # Generate branch name with MM_DD_YYYY format
+          BRANCH_NAME="litellm_internal_dev_$(date +'%m_%d_%Y')"
+          echo "Creating branch: $BRANCH_NAME"
+
+          # Fetch all branches
+          git fetch --all
+
+          # Check if the branch already exists
+          if git show-ref --verify --quiet refs/remotes/origin/$BRANCH_NAME; then
+            echo "Branch $BRANCH_NAME already exists. Skipping creation."
+          else
+            echo "Creating new branch: $BRANCH_NAME"
+            # Create the new branch from main
+            git checkout -b $BRANCH_NAME origin/main
+            # Push the new branch
+            git push origin $BRANCH_NAME
+            echo "Successfully created and pushed branch: $BRANCH_NAME"
+          fi

CLAUDE.md

@@ -102,6 +102,22 @@ LiteLLM is a unified interface for 100+ LLM providers with two main components:
 ### UI / Backend Consistency
 - When wiring a new UI entity type to an existing backend endpoint, verify the backend API contract (single value vs. array, required vs. optional params) and ensure the UI controls match — e.g., use a single-select dropdown when the backend accepts a single value, not a multi-select
 
+### MCP OAuth / OpenAPI Transport Mapping
+- `TRANSPORT.OPENAPI` is a UI-only concept. The backend only accepts `"http"`, `"sse"`, or `"stdio"`. Always map it to `"http"` before any API call (including pre-OAuth temp-session calls).
+- FastAPI validation errors return `detail` as an array of `{loc, msg, type}` objects. Error extractors must handle: array (map `.msg`), string, nested `{error: string}`, and fallback.
+- When an MCP server already has `authorization_url` stored, skip OAuth discovery (`_discovery_metadata`) — the server URL for OpenAPI MCPs is the spec file, not the API base, and fetching it causes timeouts.
+- `client_id` should be optional in the `/authorize` endpoint — if the server has a stored `client_id` in credentials, use that. Never require callers to re-supply it.
+
+### MCP Credential Storage
+- OAuth credentials and BYOK credentials share the `litellm_mcpusercredentials` table, distinguished by a `"type"` field in the JSON payload (`"oauth2"` vs plain string).
+- When deleting OAuth credentials, check type before deleting to avoid accidentally deleting a BYOK credential for the same `(user_id, server_id)` pair.
+- Always pass the raw `expires_at` timestamp to the client — never set it to `None` for expired credentials. Let the frontend compute the "Expired" display state from the timestamp.
+- Use `RecordNotFoundError` (not bare `except Exception`) when catching "already deleted" in credential delete endpoints.
+
+### Browser Storage Safety (UI)
+- Never write LiteLLM access tokens or API keys to `localStorage` — use `sessionStorage` only. `localStorage` survives browser close and is readable by any injected script (XSS).
+- Shared utility functions (e.g. `extractErrorMessage`) belong in `src/utils/` — never define them inline in hooks or duplicate them across files.
+
 ### Database Migrations
 - Prisma handles schema migrations
 - Migration files auto-generated with `prisma migrate dev`

docs/my-website/docs/proxy/config_settings.md

@@ -944,7 +944,7 @@ router_settings:
 | QDRANT_URL | Connection URL for Qdrant database
 | QDRANT_VECTOR_SIZE | Vector size for Qdrant operations. Default is 1536
 | REDIS_CONNECTION_POOL_TIMEOUT | Timeout in seconds for Redis connection pool. Default is 5
-| REDIS_CLUSTER_NODES | JSON-formatted list of Redis cluster startup nodes for Redis Cluster mode. Example: '[{"host": "node1", "port": 6379}]'
+| REDIS_CLUSTER_NODES | JSON-formatted list of Redis cluster startup nodes for Redis Cluster mode. Example: `[{"host": "node1", "port": 6379}]`
 | REDIS_HOST | Hostname for Redis server
 | REDIS_PASSWORD | Password for Redis service
 | REDIS_PORT | Port number for Redis server

docs/my-website/docs/proxy/guardrails/guardrail_policies.md

@@ -309,6 +309,10 @@ Response:
 </TabItem>
 </Tabs>
 
+## Policy Flow Builder
+
+For conditional execution (e.g., run a second guardrail only if the first fails), use the [Policy Flow Builder](./policy_flow_builder) to define pipelines with per-step pass/fail actions.
+
 ## Config Reference
 
 ### `policies`
@@ -323,6 +327,7 @@ policies:
       remove: [...]
     condition:
       model: ...
+    pipeline: ...  # optional; see Policy Flow Builder
 ```
 
 | Field | Type | Description |
@@ -332,6 +337,7 @@ policies:
 | `guardrails.add` | `list[string]` | Guardrails to enable. |
 | `guardrails.remove` | `list[string]` | Guardrails to disable (useful with inheritance). |
 | `condition.model` | `string` or `list[string]` | Optional. Only apply when model matches. Supports regex. |
+| `pipeline` | `object` | Optional. Ordered guardrail execution with per-step actions. See [Policy Flow Builder](./policy_flow_builder). |
 
 ### `policy_attachments`
 

docs/my-website/docs/proxy/guardrails/policy_flow_builder.md

@@ -0,0 +1,219 @@
+# Policy Flow Builder
+
+The Policy Flow Builder lets you design guardrail pipelines with **conditional execution**. Instead of running guardrails independently, you chain them into ordered steps and control what happens when each guardrail passes or fails.
+
+Two powerful patterns it enables: **guardrail fallbacks** (try a different guardrail when one fails) and **retrying the same guardrail** (run the same guardrail again if it fails, e.g. to handle transient errors).
+
+## When to use the Flow Builder
+
+| Approach | Use case |
+|----------|----------|
+| **Simple policy** (`guardrails.add`) | All guardrails run in parallel; any failure blocks the request. |
+| **Flow Builder** (pipeline) | Guardrails run in sequence; you choose actions per step (next, block, allow, custom response). |
+
+Use the Flow Builder when you need:
+
+- **Guardrail fallbacks** — use `on_fail: next` to try a different guardrail when one fails (e.g., fast filter → stricter filter)
+- **Retrying the same guardrail** — add the same guardrail as multiple steps; if it fails, `on_fail: next` moves to the next step, which can be the same guardrail again (useful for transient API errors or rate limits)
+- **Conditional routing** — e.g., if a fast guardrail fails, run a more advanced one instead of blocking immediately
+- **Custom responses** — return a specific message when a guardrail fails instead of a generic block
+- **Data chaining** — pass modified data (e.g., PII-masked content) from one step to the next
+- **Fine-grained control** — different actions on pass vs. fail per step
+
+## Concepts
+
+### Pipeline
+
+A pipeline has:
+
+- **Mode**: `pre_call` (before the LLM) or `post_call` (after the LLM)
+- **Steps**: Ordered list of guardrail steps
+
+### Step actions
+
+Each step defines what happens when the guardrail **passes** and when it **fails**:
+
+| Action | Description |
+|--------|-------------|
+| **Next Step** | Continue to the next guardrail in the pipeline |
+| **Allow** | Stop the pipeline and allow the request to proceed |
+| **Block** | Stop the pipeline and block the request |
+| **Custom Response** | Return a custom message instead of the default block |
+
+### Step options
+
+| Field | Type | Description |
+|-------|------|--------------|
+| `guardrail` | `string` | Name of the guardrail to run |
+| `on_pass` | `string` | Action when guardrail passes: `next`, `allow`, `block`, `modify_response` |
+| `on_fail` | `string` | Action when guardrail fails: `next`, `allow`, `block`, `modify_response` |
+| `pass_data` | `boolean` | Forward modified request data (e.g., PII-masked) to the next step |
+| `modify_response_message` | `string` | Custom message when using `modify_response` action |
+
+## Using the Flow Builder (UI)
+
+1. Go to **Policies** in the LiteLLM Admin UI
+2. Click **+ Create New Policy** or **Edit** on an existing policy
+3. Select **Flow Builder** (instead of the simple form)
+4. Design your flow:
+   - **Trigger** — Incoming LLM request (runs when the policy matches)
+   - **Steps** — Add guardrails, set ON PASS and ON FAIL actions per step
+   - **End** — Request proceeds to the LLM
+5. Use the **+** between steps to insert new steps
+6. Use the **Test** panel to run sample messages through the pipeline before saving
+7. Click **Save** to create or update the policy
+
+## Config (YAML)
+
+Define a pipeline in your policy config:
+
+```yaml showLineNumbers title="config.yaml"
+guardrails:
+  - guardrail_name: pii_masking
+    litellm_params:
+      guardrail: presidio
+      mode: pre_call
+
+  - guardrail_name: prompt_injection
+    litellm_params:
+      guardrail: lakera
+      mode: pre_call
+
+policies:
+  my-pipeline-policy:
+    description: "PII mask first, then check for prompt injection"
+    guardrails:
+      add:
+        - pii_masking
+        - prompt_injection
+    pipeline:
+      mode: pre_call
+      steps:
+        - guardrail: pii_masking
+          on_pass: next
+          on_fail: block
+          pass_data: true
+        - guardrail: prompt_injection
+          on_pass: allow
+          on_fail: block
+
+policy_attachments:
+  - policy: my-pipeline-policy
+    scope: "*"
+```
+
+## Fallbacks and retries
+
+### Guardrail fallbacks
+
+Use `on_fail: next` to fall back to another guardrail when one fails. Run a lightweight guardrail first; if it fails, escalate to a stricter or different provider:
+
+```yaml
+policies:
+  fallback-policy:
+    guardrails:
+      add:
+        - fast_content_filter
+        - strict_content_filter
+    pipeline:
+      mode: pre_call
+      steps:
+        - guardrail: fast_content_filter
+          on_pass: allow
+          on_fail: next
+        - guardrail: strict_content_filter
+          on_pass: allow
+          on_fail: block
+```
+
+If `fast_content_filter` passes → allow. If it fails → run `strict_content_filter`; pass → allow, fail → block.
+
+### Retrying the same guardrail
+
+Add the same guardrail as multiple steps to retry on failure. Useful for transient errors (API timeouts, rate limits):
+
+```yaml
+policies:
+  retry-policy:
+    guardrails:
+      add:
+        - lakera_prompt_injection
+    pipeline:
+      mode: pre_call
+      steps:
+        - guardrail: lakera_prompt_injection
+          on_pass: allow
+          on_fail: next
+        - guardrail: lakera_prompt_injection
+          on_pass: allow
+          on_fail: block
+```
+
+First attempt passes → allow. First attempt fails → retry the same guardrail; second pass → allow, second fail → block.
+
+## Example: Custom response on fail
+
+Return a branded message instead of a generic block:
+
+```yaml
+policies:
+  branded-block-policy:
+    guardrails:
+      add:
+        - pii_detector
+    pipeline:
+      mode: pre_call
+      steps:
+        - guardrail: pii_detector
+          on_pass: allow
+          on_fail: modify_response
+          modify_response_message: "Your message contains sensitive information. Please remove PII and try again."
+```
+
+## Test a pipeline (API)
+
+Test a pipeline with sample messages before attaching it:
+
+```bash
+curl -X POST "http://localhost:4000/policies/test-pipeline" \
+  -H "Authorization: Bearer <your_api_key>" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "pipeline": {
+      "mode": "pre_call",
+      "steps": [
+        {
+          "guardrail": "pii_masking",
+          "on_pass": "next",
+          "on_fail": "block",
+          "pass_data": true
+        },
+        {
+          "guardrail": "prompt_injection",
+          "on_pass": "allow",
+          "on_fail": "block"
+        }
+      ]
+    },
+    "test_messages": [
+      {"role": "user", "content": "What is 2+2?"},
+      {"role": "user", "content": "My SSN is 123-45-6789"}
+    ]
+  }'
+```
+
+Response includes per-step outcomes (pass/fail/error), actions taken, and timing.
+
+## Pipeline vs simple policy
+
+When a policy has a `pipeline`, the pipeline defines execution order and actions. The `guardrails.add` list must include all guardrails used in the pipeline steps.
+
+| Policy type | Execution |
+|-------------|-----------|
+| Simple (`guardrails.add` only) | All guardrails run; any failure blocks |
+| Pipeline (`pipeline` present) | Steps run in order; actions control flow |
+
+## Related docs
+
+- [Guardrail Policies](./guardrail_policies) — Policy basics, attachments, inheritance
+- [Policy Templates](./policy_templates) — Pre-built policy templates