Litellm oss staging 04 11 2026

[krrish-berri-2]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 14, 2026 3:37pm

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_oss_staging_04_11_2026 (a0e61a9) with main (e64d98f)}

[greptile-apps]

Greptile Summary

This staging PR bundles several bug fixes: aligning model_max_budget spend-tracking cache keys to use model_group (the user-facing alias), fixing silent cache-invalidation failures caused by double-hashing token IDs in bulk-update and key-rotation endpoints, resetting spend counters for budget-tier-linked keys and team members when a budget period expires, and adding model_max_budget validation to the /budget/new and /budget/update endpoints. New model pricing entries (baseten, wandb) and accompanying unit/integration tests are included.

Confidence Score: 5/5

• Safe to merge; all remaining findings are P2 style and test-coverage suggestions that do not block production correctness.
• All three issues are P2: an inline import that should be module-level, a missing mock method that silently skips (but does not break) a test, and a missing assertion on a spend > 0 filter. None affect runtime behaviour or data integrity.
• tests/test_litellm/proxy/common_utils/test_reset_budget_job.py — add find_many stub to MockLiteLLMTeamMembership so the new team-member cache-reset path is exercised.

Important Files Changed

Filename	Overview
litellm/proxy/common_utils/reset_budget_job.py	Adds reset_budget_for_keys_linked_to_budgets (resets spend on keys tied to a budget tier with no own reset schedule) and reset_budget_for_team_members (resets team-member spend counters in Redis/in-memory + DB). Also extends _reset_budget_common to flush Redis spend counters for keys and teams on budget reset. Logic is sound; imports from proxy_server are inline due to the circular-import constraint.
litellm/proxy/hooks/model_max_budget_limiter.py	Uses model_group (user-facing alias) instead of the deployment-level model field as the spend-tracking cache key, aligning the log path with the enforcement path in is_key_within_model_budget. No issues found.
litellm/proxy/management_endpoints/budget_management_endpoints.py	Adds validate_model_max_budget calls to /budget/new and /budget/update to enforce the enterprise license check and schema validation on model_max_budget. Minor: the import is placed inline inside the handler functions instead of at the top of the file, violating the project style guide.
tests/test_litellm/proxy/common_utils/test_reset_budget_job.py	Comprehensive new test suite for ResetBudgetJob covering keys, users, teams, end-users, budget-table resets, and the new reset_budget_for_keys_linked_to_budgets helper. Minor gap: MockLiteLLMTeamMembership is missing find_many, silently skipping the new team-member cache-reset path; and spend: {gt: 0} filter is not verified.
tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py	Adds tests for cache-invalidation correctness in _process_single_key_update and _execute_virtual_key_regeneration, verifying that pre-hashed tokens are not double-hashed. Tests are isolated and use appropriate mocking.
tests/test_litellm/test_cost_calculator.py	Adds pricing-verification tests for baseten and wandb model entries, a dynamic-routing cost-fallback test, and an image token cost test with and without input_cost_per_image_token. All tests are self-contained mock-based unit tests.
tests/test_budget_management.py	Integration test verifying that /budget/new with a budget_duration sets budget_reset_at to the next calendar-aligned reset time. Lives in the tests/ root (integration tier) and makes real HTTP calls, which is appropriate for this location.
litellm/proxy/management_endpoints/key_management_endpoints.py	Replaces raw cache-key lookup with _hash_token_if_needed in bulk-update and key-regeneration paths to prevent double-hashing when the input is already a pre-hashed token ID. No functional regressions found.

Sequence Diagram

sequenceDiagram
    participant BudgetResetJob
    participant DB as PrismaClient / DB
    participant Cache as Redis / InMemoryCache

    BudgetResetJob->>DB: "find budgets where reset_at <= now"
    DB-->>BudgetResetJob: budgets_to_reset[]

    BudgetResetJob->>BudgetResetJob: _reset_budget_reset_at_date() for each budget
    BudgetResetJob->>DB: update_many budgets (new reset_at)

    BudgetResetJob->>DB: find team memberships by budget_id
    DB-->>BudgetResetJob: memberships[]
    loop each membership
        BudgetResetJob->>Cache: "set spend:team_member:{uid}:{tid} = 0"
    end
    BudgetResetJob->>DB: "update_many litellm_teammembership spend=0"

    BudgetResetJob->>DB: "update_many litellm_verificationtoken<br/>(budget_id IN ids, budget_duration IS NULL, spend > 0) spend=0"

    BudgetResetJob->>DB: find end-users by budget_id
    DB-->>BudgetResetJob: endusers[]
    BudgetResetJob->>BudgetResetJob: "_reset_budget_for_enduser() spend=0"
    BudgetResetJob->>DB: "update_many endusers spend=0"

Loading

_{Reviews (4): Last reviewed commit: "Fix code qa" | Re-trigger Greptile}

[greptile-apps]

DictLikeResult workaround is fragile

The inner DictLikeResult class works today because _execute_virtual_key_regeneration calls dict(updated_token), and __iter__ yields (key, value) pairs. However, if the function is later updated to call any Prisma model method (e.g., updated_token.model_dump(), updated_token.token, attribute access), the test would raise an AttributeError without a clear failure message.

A more resilient alternative is using a MagicMock with __iter__ set, or using a real LiteLLM_VerificationToken instance as the mock return value. This keeps the test aligned with the actual Prisma return type shape.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
29203053	Triggered	Generic Password	ee40da5	.circleci/config.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}