cherry-pick: tag query fix + MCP metadata support#25145
Conversation
* added support for metadata * fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests Made-with: Cursor * pyproject to .25 * feat(teams): resolve access group models/MCPs/agents in team endpoints Add access_group_models, access_group_mcp_server_ids, and access_group_agent_ids to /team/info and /v2/team/list responses. These fields contain resources inherited from access groups, kept separate from direct assignments so the UI can distinguish the source. Backend: _resolve_access_group_resources() helper resolves access group resources via existing _get_*_from_access_groups() functions. UI: Teams table and detail view show direct models as blue badges and access-group-sourced models as green badges. * perf(teams): single-pass access group resolution + asyncio.gather in list endpoint - Fetch each access group object once and extract all 3 resource fields in a single pass instead of 3 separate calls (3N → N lookups) - Use asyncio.gather to resolve access groups across teams concurrently in list_team_v2 instead of sequential awaits - Add 5 unit tests for _resolve_access_group_resources * docs: add default_team_params to config reference and update examples - Add default_team_params to litellm_settings reference table in config_settings.md with all sub-fields documented - Update self_serve.md and msft_sso.md examples to include team_member_permissions, tpm_limit, and rpm_limit - Fix misleading comment that implied default_team_params only applies to SSO auto-created teams — it applies to all /team/new calls * docs: clarify that models sub-field only applies to SSO auto-created teams * fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display - Remove get_access_object from module-level import in team_endpoints.py and use a lazy _get_access_object wrapper to avoid cyclic dependency - Add _prisma_client is None early-exit guard in _resolve_access_group_resources - Short-circuit UI to show "All Proxy Models" when team.models is empty or contains "all-proxy-models", skipping access group model resolution * add: making organizations a select instead of read only badges * fix(ui): only send organization_id when changed and use raw initial value * fix(ui): add paginated team search to usage page filter Replace the static team dropdown on the usage page with a new TeamMultiSelect component that uses the paginated v2/team/list endpoint with debounced server-side search and infinite scroll. * fix(ui): fix imports and update placeholder for team multi select * fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab The Key Alias dropdown on the Virtual Keys page was showing aliases from all teams regardless of which team was selected. The team_id was never passed through the frontend chain to the backend /key/aliases endpoint. - Backend: add optional team_id query param to /key/aliases endpoint - networking.tsx: add team_id param to keyAliasesCall - useKeyAliases: accept and forward team_id to API call and query key - filter.tsx: pass allFilters context to custom filter components - PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook * fix(tests): correct mock targets in TestResolveAccessGroupResources Three tests were patching the non-existent `get_access_object` instead of `_get_access_object` (the lazy-import wrapper), causing AttributeError. Also added missing `prisma_client` mock so tests get past the early-exit guard and actually exercise the resolution logic. * fix: use direct attribute access with or [] fallback in _resolve_access_group_resources Replace getattr(ag, "field", []) with ag.field or [] for cleaner access and safe handling if a field is None. * fix(ui): remove model source legend from team detail view The blue/green color distinction is self-explanatory; the legend added visual clutter without providing enough value. * fix(ui): add missing access_group fields to TeamData.team_info type The TeamData interface was missing access_group_models, access_group_mcp_server_ids, and access_group_agent_ids fields, causing a TypeScript build failure. * perf(teams): batch-fetch access groups in single DB query Replace per-ID _resolve_access_group_resources loop with a single find_many call that deduplicates IDs across all teams. Removes the N+1 query pattern on cold cache for the team list endpoint. * refactor(proxy): extract helpers to fix PLR0915 violations Extract `_apply_non_admin_alias_scope` from `key_aliases`, `_resolve_team_access_group_resources` from `team_info`, and `_enforce_list_team_v2_access` from `list_team_v2` to bring each function under ruff's 50-statement limit. No behavior changes. * test(ui): update tests to match new team_id / access-group signatures - useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to spy matchers for the new `team_id` param on `useInfiniteKeyAliases` and `keyAliasesCall`. - EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider is not required for team-entity tests. - ModelsCell: replace the overflow-accordion test with one that verifies the new collapse-on-`all-proxy-models` behavior (no accordion, single badge). * fix(ui): send null (not '') for cleared organization_id on team update AntD <Select allowClear> returns undefined when the user clears the selection. Coalescing to "" caused the team-update payload to carry organization_id: "" instead of null, relying on the backend to coerce it. Send null directly so the intent is explicit at the source. * poetry * chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump * chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0) --------- Co-authored-by: shivam <shivam@uni.minerva.edu> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* feat(tag-spend): implement separate scheduler job for daily tag spend updates * fix(docker): add g++ to build dependencies in Dockerfile * initial test cases. TODO: check scheduler init and test cases in proxy_server related to it * resolved QPS issue when redis transaction buffer is enabled * resolving circular import error flagged by greptile
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis cherry-pick brings two independent fixes onto
Minor findings (all P2):
All previously flagged items (inline import in Confidence Score: 5/5Safe to merge — no P0 or P1 defects found; all findings are minor coverage and style suggestions. All three remaining findings are P2: a test file placed outside No files require special attention. The
|
| Filename | Overview |
|---|---|
| litellm/constants.py | Adds DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME lock key and DAILY_TAG_SPEND_BATCH_MULTIPLIER = 2.3 for the new scheduler; pattern consistent with existing constants. |
| litellm/proxy/_experimental/mcp_server/server.py | Adds _normalize_resource_contents() to propagate MCP 1.26.0 meta field; correctly uses is None guard so empty-dict meta is preserved. |
| litellm/proxy/db/db_spend_update_writer.py | Extracts tag spend into _commit_daily_tag_spend_to_db() / _commit_daily_tag_spend_to_db_with_redis(); cleanly removes it from the shared no-Redis path with an explanatory comment. |
| litellm/proxy/db/db_transaction_queue/redis_update_buffer.py | Adds store_in_memory_daily_tag_spend_updates_in_redis and get_all_daily_tag_spend_update_transactions_from_redis_buffer for the tag-only Redis path, correctly separate from the main 6-queue pipeline. |
| litellm/proxy/proxy_server.py | Registers the new update_daily_tag_spend APScheduler job at the correct multiplied interval. |
| litellm/proxy/utils.py | Adds update_daily_tag_spend() entry point that dispatches to Redis or non-Redis path; error is caught and logged without re-raising. |
| tests/proxy_unit_tests/test_update_daily_tag_spend.py | Good test coverage for update_daily_tag_spend branches, but placed in tests/proxy_unit_tests/ and will not be executed by make test-unit. |
| tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py | Adds well-structured tests for _normalize_resource_contents covering text, blob, empty-dict meta, and no-meta backward-compatibility; in the correct tests/test_litellm/ location. |
| tests/test_litellm/proxy/db/db_transaction_queue/test_redis_update_buffer.py | Tests the main pipeline path adequately but lacks coverage for the new store_in_memory_daily_tag_spend_updates_in_redis method. |
| docker/Dockerfile.dev | Minor, unrelated change; no issues. |
Sequence Diagram
sequenceDiagram
participant R as Request Handler
participant Q as daily_tag_spend_update_queue
participant S as APScheduler
participant U as update_daily_tag_spend()
participant W as DBSpendUpdateWriter
participant Redis as Redis Buffer
participant DB as LiteLLM_DailyTagSpend
R->>Q: add_spend_log_transaction_to_daily_tag_transaction()
Note over S: Every batch_interval x 2.3 seconds
S->>U: trigger update_daily_tag_spend()
alt Redis buffer enabled
U->>W: _commit_daily_tag_spend_to_db_with_redis()
W->>Redis: store_in_memory_daily_tag_spend_updates_in_redis()
W->>W: acquire pod lock (DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME)
W->>Redis: get_all_daily_tag_spend_update_transactions_from_redis_buffer()
W->>DB: update_daily_tag_spend() upsert
W->>W: release pod lock
else No Redis buffer
U->>W: _commit_daily_tag_spend_to_db()
W->>Q: flush_and_get_aggregated_daily_spend_update_transactions()
W->>DB: update_daily_tag_spend() upsert
end
Reviews (3): Last reviewed commit: "Merge branch 'main' into litellm_cherry_..." | Re-trigger Greptile
| ### UPDATE DAILY TAG SPEND (separate scheduler job with longer interval) ### | ||
| ## Reduces QPS as there are more tags for a single request | ||
| tag_spend_update_interval = int(batch_writing_interval * DAILY_TAG_SPEND_BATCH_MULTIPLIER) | ||
| from litellm.proxy.utils import update_daily_tag_spend |
There was a problem hiding this comment.
Inline import inside function body
Per the project style guide, imports should be at the top of the file (module-level), not inside functions/methods. The only accepted exception is to break circular imports. This import can be moved to the top of proxy_server.py alongside the other litellm.proxy.utils imports.
Move from litellm.proxy.utils import update_daily_tag_spend to the top-level imports section of proxy_server.py where the other utils symbols are imported.
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| # The number of tag entries are higher than number of user, team entries. This leads to a higher QPS. | ||
| # This will run tag spcific tasks at a later time to smooth QPS |
There was a problem hiding this comment.
"spcific" should be "specific".
| # The number of tag entries are higher than number of user, team entries. This leads to a higher QPS. | |
| # This will run tag spcific tasks at a later time to smooth QPS | |
| # The number of tag entries are higher than number of user, team entries. This leads to a higher QPS. | |
| # This will run tag specific tasks at a later time to smooth QPS |
| from litellm._logging import verbose_proxy_logger | ||
| from litellm.caching import DualCache, RedisCache | ||
| from litellm.constants import DB_SPEND_UPDATE_JOB_NAME | ||
| from litellm.constants import DB_SPEND_UPDATE_JOB_NAME,DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME |
There was a problem hiding this comment.
Missing space after comma in import
Black and standard Python style require a space after commas in import lists.
| from litellm.constants import DB_SPEND_UPDATE_JOB_NAME,DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME | |
| from litellm.constants import DB_SPEND_UPDATE_JOB_NAME, DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…atch superclass signature
* added support for metadata (BerriAI#24261) * added support for metadata * fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests Made-with: Cursor * pyproject to .25 * feat(teams): resolve access group models/MCPs/agents in team endpoints Add access_group_models, access_group_mcp_server_ids, and access_group_agent_ids to /team/info and /v2/team/list responses. These fields contain resources inherited from access groups, kept separate from direct assignments so the UI can distinguish the source. Backend: _resolve_access_group_resources() helper resolves access group resources via existing _get_*_from_access_groups() functions. UI: Teams table and detail view show direct models as blue badges and access-group-sourced models as green badges. * perf(teams): single-pass access group resolution + asyncio.gather in list endpoint - Fetch each access group object once and extract all 3 resource fields in a single pass instead of 3 separate calls (3N → N lookups) - Use asyncio.gather to resolve access groups across teams concurrently in list_team_v2 instead of sequential awaits - Add 5 unit tests for _resolve_access_group_resources * docs: add default_team_params to config reference and update examples - Add default_team_params to litellm_settings reference table in config_settings.md with all sub-fields documented - Update self_serve.md and msft_sso.md examples to include team_member_permissions, tpm_limit, and rpm_limit - Fix misleading comment that implied default_team_params only applies to SSO auto-created teams — it applies to all /team/new calls * docs: clarify that models sub-field only applies to SSO auto-created teams * fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display - Remove get_access_object from module-level import in team_endpoints.py and use a lazy _get_access_object wrapper to avoid cyclic dependency - Add _prisma_client is None early-exit guard in _resolve_access_group_resources - Short-circuit UI to show "All Proxy Models" when team.models is empty or contains "all-proxy-models", skipping access group model resolution * add: making organizations a select instead of read only badges * fix(ui): only send organization_id when changed and use raw initial value * fix(ui): add paginated team search to usage page filter Replace the static team dropdown on the usage page with a new TeamMultiSelect component that uses the paginated v2/team/list endpoint with debounced server-side search and infinite scroll. * fix(ui): fix imports and update placeholder for team multi select * fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab The Key Alias dropdown on the Virtual Keys page was showing aliases from all teams regardless of which team was selected. The team_id was never passed through the frontend chain to the backend /key/aliases endpoint. - Backend: add optional team_id query param to /key/aliases endpoint - networking.tsx: add team_id param to keyAliasesCall - useKeyAliases: accept and forward team_id to API call and query key - filter.tsx: pass allFilters context to custom filter components - PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook * fix(tests): correct mock targets in TestResolveAccessGroupResources Three tests were patching the non-existent `get_access_object` instead of `_get_access_object` (the lazy-import wrapper), causing AttributeError. Also added missing `prisma_client` mock so tests get past the early-exit guard and actually exercise the resolution logic. * fix: use direct attribute access with or [] fallback in _resolve_access_group_resources Replace getattr(ag, "field", []) with ag.field or [] for cleaner access and safe handling if a field is None. * fix(ui): remove model source legend from team detail view The blue/green color distinction is self-explanatory; the legend added visual clutter without providing enough value. * fix(ui): add missing access_group fields to TeamData.team_info type The TeamData interface was missing access_group_models, access_group_mcp_server_ids, and access_group_agent_ids fields, causing a TypeScript build failure. * perf(teams): batch-fetch access groups in single DB query Replace per-ID _resolve_access_group_resources loop with a single find_many call that deduplicates IDs across all teams. Removes the N+1 query pattern on cold cache for the team list endpoint. * refactor(proxy): extract helpers to fix PLR0915 violations Extract `_apply_non_admin_alias_scope` from `key_aliases`, `_resolve_team_access_group_resources` from `team_info`, and `_enforce_list_team_v2_access` from `list_team_v2` to bring each function under ruff's 50-statement limit. No behavior changes. * test(ui): update tests to match new team_id / access-group signatures - useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to spy matchers for the new `team_id` param on `useInfiniteKeyAliases` and `keyAliasesCall`. - EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider is not required for team-entity tests. - ModelsCell: replace the overflow-accordion test with one that verifies the new collapse-on-`all-proxy-models` behavior (no accordion, single badge). * fix(ui): send null (not '') for cleared organization_id on team update AntD <Select allowClear> returns undefined when the user clears the selection. Coalescing to "" caused the team-update payload to carry organization_id: "" instead of null, relying on the backend to coerce it. Send null directly so the intent is explicit at the source. * poetry * chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump * chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0) --------- Co-authored-by: shivam <shivam@uni.minerva.edu> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * Tag query fix (BerriAI#25094) * feat(tag-spend): implement separate scheduler job for daily tag spend updates * fix(docker): add g++ to build dependencies in Dockerfile * initial test cases. TODO: check scheduler init and test cases in proxy_server related to it * resolved QPS issue when redis transaction buffer is enabled * resolving circular import error flagged by greptile * fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature --------- Co-authored-by: Shivam Rawat <shivam@berri.ai> Co-authored-by: shivam <shivam@uni.minerva.edu> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Harish <harishgokul01@gmail.com> Co-authored-by: Ishaan Jaffer <ishaan@berri.ai>
* added support for metadata (BerriAI#24261) * added support for metadata * fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests Made-with: Cursor * pyproject to .25 * feat(teams): resolve access group models/MCPs/agents in team endpoints Add access_group_models, access_group_mcp_server_ids, and access_group_agent_ids to /team/info and /v2/team/list responses. These fields contain resources inherited from access groups, kept separate from direct assignments so the UI can distinguish the source. Backend: _resolve_access_group_resources() helper resolves access group resources via existing _get_*_from_access_groups() functions. UI: Teams table and detail view show direct models as blue badges and access-group-sourced models as green badges. * perf(teams): single-pass access group resolution + asyncio.gather in list endpoint - Fetch each access group object once and extract all 3 resource fields in a single pass instead of 3 separate calls (3N → N lookups) - Use asyncio.gather to resolve access groups across teams concurrently in list_team_v2 instead of sequential awaits - Add 5 unit tests for _resolve_access_group_resources * docs: add default_team_params to config reference and update examples - Add default_team_params to litellm_settings reference table in config_settings.md with all sub-fields documented - Update self_serve.md and msft_sso.md examples to include team_member_permissions, tpm_limit, and rpm_limit - Fix misleading comment that implied default_team_params only applies to SSO auto-created teams — it applies to all /team/new calls * docs: clarify that models sub-field only applies to SSO auto-created teams * fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display - Remove get_access_object from module-level import in team_endpoints.py and use a lazy _get_access_object wrapper to avoid cyclic dependency - Add _prisma_client is None early-exit guard in _resolve_access_group_resources - Short-circuit UI to show "All Proxy Models" when team.models is empty or contains "all-proxy-models", skipping access group model resolution * add: making organizations a select instead of read only badges * fix(ui): only send organization_id when changed and use raw initial value * fix(ui): add paginated team search to usage page filter Replace the static team dropdown on the usage page with a new TeamMultiSelect component that uses the paginated v2/team/list endpoint with debounced server-side search and infinite scroll. * fix(ui): fix imports and update placeholder for team multi select * fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab The Key Alias dropdown on the Virtual Keys page was showing aliases from all teams regardless of which team was selected. The team_id was never passed through the frontend chain to the backend /key/aliases endpoint. - Backend: add optional team_id query param to /key/aliases endpoint - networking.tsx: add team_id param to keyAliasesCall - useKeyAliases: accept and forward team_id to API call and query key - filter.tsx: pass allFilters context to custom filter components - PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook * fix(tests): correct mock targets in TestResolveAccessGroupResources Three tests were patching the non-existent `get_access_object` instead of `_get_access_object` (the lazy-import wrapper), causing AttributeError. Also added missing `prisma_client` mock so tests get past the early-exit guard and actually exercise the resolution logic. * fix: use direct attribute access with or [] fallback in _resolve_access_group_resources Replace getattr(ag, "field", []) with ag.field or [] for cleaner access and safe handling if a field is None. * fix(ui): remove model source legend from team detail view The blue/green color distinction is self-explanatory; the legend added visual clutter without providing enough value. * fix(ui): add missing access_group fields to TeamData.team_info type The TeamData interface was missing access_group_models, access_group_mcp_server_ids, and access_group_agent_ids fields, causing a TypeScript build failure. * perf(teams): batch-fetch access groups in single DB query Replace per-ID _resolve_access_group_resources loop with a single find_many call that deduplicates IDs across all teams. Removes the N+1 query pattern on cold cache for the team list endpoint. * refactor(proxy): extract helpers to fix PLR0915 violations Extract `_apply_non_admin_alias_scope` from `key_aliases`, `_resolve_team_access_group_resources` from `team_info`, and `_enforce_list_team_v2_access` from `list_team_v2` to bring each function under ruff's 50-statement limit. No behavior changes. * test(ui): update tests to match new team_id / access-group signatures - useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to spy matchers for the new `team_id` param on `useInfiniteKeyAliases` and `keyAliasesCall`. - EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider is not required for team-entity tests. - ModelsCell: replace the overflow-accordion test with one that verifies the new collapse-on-`all-proxy-models` behavior (no accordion, single badge). * fix(ui): send null (not '') for cleared organization_id on team update AntD <Select allowClear> returns undefined when the user clears the selection. Coalescing to "" caused the team-update payload to carry organization_id: "" instead of null, relying on the backend to coerce it. Send null directly so the intent is explicit at the source. * poetry * chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump * chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0) --------- Co-authored-by: shivam <shivam@uni.minerva.edu> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * Tag query fix (BerriAI#25094) * feat(tag-spend): implement separate scheduler job for daily tag spend updates * fix(docker): add g++ to build dependencies in Dockerfile * initial test cases. TODO: check scheduler init and test cases in proxy_server related to it * resolved QPS issue when redis transaction buffer is enabled * resolving circular import error flagged by greptile * fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature --------- Co-authored-by: Shivam Rawat <shivam@berri.ai> Co-authored-by: shivam <shivam@uni.minerva.edu> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Harish <harishgokul01@gmail.com> Co-authored-by: Ishaan Jaffer <ishaan@berri.ai>
Cherry-picks two commits onto main:
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
Changes
Cherry-pick of commits f4e69f4 and fca672f onto clean main.