Skip to content

litellm ryan march 31#25119

Merged
yuneng-berri merged 26 commits intomainfrom
litellm_ryan-march-31
Apr 4, 2026
Merged

litellm ryan march 31#25119
yuneng-berri merged 26 commits intomainfrom
litellm_ryan-march-31

Conversation

Add access_group_models, access_group_mcp_server_ids, and
access_group_agent_ids to /team/info and /v2/team/list responses.
These fields contain resources inherited from access groups, kept
separate from direct assignments so the UI can distinguish the source.

Backend: _resolve_access_group_resources() helper resolves access
group resources via existing _get_*_from_access_groups() functions.

UI: Teams table and detail view show direct models as blue badges
and access-group-sourced models as green badges.
…list endpoint

- Fetch each access group object once and extract all 3 resource fields
  in a single pass instead of 3 separate calls (3N → N lookups)
- Use asyncio.gather to resolve access groups across teams concurrently
  in list_team_v2 instead of sequential awaits
- Add 5 unit tests for _resolve_access_group_resources
- Add default_team_params to litellm_settings reference table in
  config_settings.md with all sub-fields documented
- Update self_serve.md and msft_sso.md examples to include
  team_member_permissions, tpm_limit, and rpm_limit
- Fix misleading comment that implied default_team_params only applies
  to SSO auto-created teams — it applies to all /team/new calls
…cuit all-proxy-models display

- Remove get_access_object from module-level import in team_endpoints.py
  and use a lazy _get_access_object wrapper to avoid cyclic dependency
- Add _prisma_client is None early-exit guard in _resolve_access_group_resources
- Short-circuit UI to show "All Proxy Models" when team.models is empty
  or contains "all-proxy-models", skipping access group model resolution
Replace the static team dropdown on the usage page with a new
TeamMultiSelect component that uses the paginated v2/team/list
endpoint with debounced server-side search and infinite scroll.
The Key Alias dropdown on the Virtual Keys page was showing aliases from
all teams regardless of which team was selected. The team_id was never
passed through the frontend chain to the backend /key/aliases endpoint.

- Backend: add optional team_id query param to /key/aliases endpoint
- networking.tsx: add team_id param to keyAliasesCall
- useKeyAliases: accept and forward team_id to API call and query key
- filter.tsx: pass allFilters context to custom filter components
- PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook
…filter-alias-dropdown

fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab
fix(ui): add paginated team search to usage page filter
fix(ui): allow changing team organization from team settings
docs: document default_team_params in config reference
Three tests were patching the non-existent `get_access_object` instead
of `_get_access_object` (the lazy-import wrapper), causing AttributeError.
Also added missing `prisma_client` mock so tests get past the early-exit
guard and actually exercise the resolution logic.
…ss_group_resources

Replace getattr(ag, "field", []) with ag.field or [] for cleaner
access and safe handling if a field is None.
The blue/green color distinction is self-explanatory; the legend added
visual clutter without providing enough value.
The TeamData interface was missing access_group_models,
access_group_mcp_server_ids, and access_group_agent_ids fields,
causing a TypeScript build failure.
Replace per-ID _resolve_access_group_resources loop with a single
find_many call that deduplicates IDs across all teams. Removes the
N+1 query pattern on cold cache for the team list endpoint.
feat(teams): resolve access group resources in team endpoints
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 4, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 4, 2026 5:10pm

Request Review

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 4, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_ryan-march-31 (76c0591) with main (a5322c6)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 4, 2026

Greptile Summary

This PR bundles five improvements: (1) resolving access-group models/MCP-servers/agents on team list and team-info endpoints via a batch DB fetch; (2) a new team_id filter on /key/aliases with corresponding UI wiring through PaginatedKeyAliasSelect; (3) a paginated team multi-select filter on the Usage Page; (4) allowing org reassignment from Team Settings via a proper <Select> (fixes the previous empty-string-instead-of-null regression); and (5) several PLR0915 refactors that extract long inline blocks into named helpers.

Key points:

  • The batch-fetch pattern in _batch_resolve_access_group_resources is correct and avoids N+1 queries.
  • The organization_id ?? null fix in TeamInfo.tsx resolves the prior concern about writing empty strings to the DB.
  • The team_id security scoping for non-admin users is correct: the scope conditions ensure a user who requests team_id=X but isn't a member of team X receives an empty result.
  • Access-group model display for isAllModels (when direct models list is empty but access-group models exist) still shows "All Proxy Models" in both ModelsCell.tsx and TeamInfo.tsx; this was noted in prior review threads and has not been resolved in this PR.
  • The hardcoded \"Team ID\" string in PaginatedKeyAliasSelect still couples the component to the filter label — also noted in prior threads.

Confidence Score: 4/5

Safe to merge after addressing prior review threads; no new critical issues introduced.

All new backend logic (batch access-group fetch, team_id filter, access-control helpers) is correct and well-tested. The organization_id null fix from a prior review thread is confirmed present. The remaining open concerns (isAllModels ignoring access_group_models in ModelsCell and TeamInfo, hardcoded "Team ID" filter key) were carried over from previous threads and are not newly introduced by this PR. Score is 4 rather than 5 because those prior threads are still unresolved in the merged code.

ui/litellm-dashboard/src/app/(dashboard)/teams/components/TeamsTable/ModelsCell.tsx and ui/litellm-dashboard/src/components/team/TeamInfo.tsx — the isAllModels / access_group_models display bug from prior threads remains open.

Important Files Changed

Filename Overview
litellm/proxy/management_endpoints/team_endpoints.py Adds batch access-group resource resolution helpers and extracts two large inline blocks into named functions; logic is correct but _resolve_team_access_group_resources is defined before its dependency _batch_resolve_access_group_resources (valid Python, minor readability concern).
litellm/proxy/management_endpoints/key_management_endpoints.py Extracts non-admin alias scope logic into _apply_non_admin_alias_scope and adds team_id filter; security scoping is correct — non-admin users querying a team they're not a member of receive an empty result.
litellm/types/proxy/management_endpoints/team_endpoints.py Adds three Optional[List[str]] fields to TeamListItem for access-group-resolved models, MCP server IDs, and agent IDs; clean schema extension.
ui/litellm-dashboard/src/components/team/TeamInfo.tsx Fixes organization_id from empty-string to null on clear, replaces read-only Input with Select dropdown; models section now shows access-group badges in green but still shows "All proxy models" when direct models is empty regardless of access_group_models (pre-existing, in prior review thread).
ui/litellm-dashboard/src/app/(dashboard)/teams/components/TeamsTable/ModelsCell.tsx Refactored to use unified ModelEntry[] combining direct and access-group models with colour-coded badges; isAllModels still ignores access_group_models when direct models is empty (pre-existing, in prior review thread).
ui/litellm-dashboard/src/components/common_components/team_multi_select.tsx New paginated team multi-select component using useInfiniteTeams with debounced search and scroll-based pagination; implementation is correct.
ui/litellm-dashboard/src/components/KeyAliasSelect/PaginatedKeyAliasSelect/PaginatedKeyAliasSelect.tsx Passes allFilters prop through and extracts team_id via hardcoded "Team ID" key; functionally correct but tightly coupled to filter label naming convention (noted in prior review thread).
ui/litellm-dashboard/src/components/UsagePage/components/EntityUsage/EntityUsage.tsx Adds TeamMultiSelect as the filter control for entityType="team" and hides the legacy UsageExportHeader filter for this case; correctly reuses selectedTags state.
tests/test_litellm/proxy/management_endpoints/test_team_endpoints.py Adds six async unit tests for _batch_resolve_access_group_resources covering empty input, single group, multi-group, missing group, null prisma client, and deduplication; all use mocks, no real network calls.
ui/litellm-dashboard/src/components/molecules/filter.tsx Adds allFilters prop to FilterOptionCustomComponentProps and threads it through to custom filter components; minimal, correct change.

Sequence Diagram

sequenceDiagram
    participant UI as UI (Virtual Keys Tab)
    participant FilterComponent as FilterComponent
    participant PaginatedKeyAliasSelect as PaginatedKeyAliasSelect
    participant useInfiniteKeyAliases as useInfiniteKeyAliases
    participant keyAliasesCall as keyAliasesCall (networking.tsx)
    participant Backend as /key/aliases (FastAPI)
    participant DB as PostgreSQL

    UI->>FilterComponent: Render with "Team ID" filter option
    FilterComponent->>PaginatedKeyAliasSelect: allFilters={"Team ID": "team-xyz"}
    PaginatedKeyAliasSelect->>useInfiniteKeyAliases: (pageSize, search, teamId="team-xyz")
    useInfiniteKeyAliases->>keyAliasesCall: (token, page, size, search, team_id="team-xyz")
    keyAliasesCall->>Backend: GET /key/aliases?team_id=team-xyz
    Backend->>Backend: _apply_non_admin_alias_scope (scope WHERE)
    Backend->>DB: SELECT key_alias FROM LiteLLM_VerificationToken WHERE ... AND team_id=$N
    DB-->>Backend: Filtered aliases
    Backend-->>UI: PaginatedKeyAliasResponse

    Note over UI,Backend: Access group resolution path
    UI->>Backend: GET /team/info?team_id=abc
    Backend->>Backend: _resolve_team_access_group_resources
    Backend->>Backend: _batch_resolve_access_group_resources
    Backend->>DB: SELECT FROM litellm_accessgrouptable WHERE access_group_id IN (...)
    DB-->>Backend: Access group rows (models, mcp_servers, agents)
    Backend-->>UI: TeamInfoResponseObject with access_group_models populated
Loading

Reviews (4): Last reviewed commit: "chore: regen poetry.lock for litellm-pro..." | Re-trigger Greptile

Comment on lines +4476 to +4478
if team_id:
query_params.append(team_id)
where_parts.append(f"team_id = ${len(query_params)}")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Raw SQL for new team_id filter

The team_id filter is added using the existing raw-SQL pattern in this function. The comment at the top of the function explains why raw SQL is used here (Prisma doesn't support column-level SELECT projection on find_many), and the value is correctly parameterized, so there is no injection risk. However, CLAUDE.md explicitly states: "Do not write raw SQL for proxy DB operations. Use Prisma model methods instead of execute_raw/query_raw." The existing code already violates this rule, but extending it amplifies the concern. If a Prisma-level approach is ever adopted (e.g. returning only the key_alias column), the team_id filter should migrate with it.

Comment on lines +494 to +498
...(secretManagerSettings !== undefined ? { secret_manager_settings: secretManagerSettings } : {}),
},
...(values.policies?.length > 0 ? { policies: values.policies } : {}),
organization_id: values.organization_id,
...(values.organization_id !== info.organization_id
? { organization_id: values.organization_id ?? "" }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Empty string sent as organization_id when field is cleared

Ant Design's <Select allowClear> returns undefined when the user clears the selection. The expression values.organization_id ?? "" converts undefined to "", so the API call receives { organization_id: "" }. Prisma will write an empty string to the organization_id column rather than null, which is inconsistent with the rest of the codebase (organization IDs are UUIDs or null). This can corrupt the team record.

Use null as the fallback so the backend receives a proper null value to clear the association:

Suggested change
...(secretManagerSettings !== undefined ? { secret_manager_settings: secretManagerSettings } : {}),
},
...(values.policies?.length > 0 ? { policies: values.policies } : {}),
organization_id: values.organization_id,
...(values.organization_id !== info.organization_id
? { organization_id: values.organization_id ?? "" }
...(values.organization_id !== info.organization_id
? { organization_id: values.organization_id ?? null }
: {}),

Comment on lines 660 to 679
<Card>
<Text>Models</Text>
<div className="mt-2 flex flex-wrap gap-2">
{info.models.length === 0 ? (
{info.models.length === 0 || info.models.includes("all-proxy-models") ? (
<Badge color="red">All proxy models</Badge>
) : (
info.models.map((model, index) => (
<Badge key={index} color="red">
{model}
</Badge>
))
<>
{info.models.map((model: string, index: number) => (
<Badge key={`direct-${index}`} color="blue">
{model}
</Badge>
))}
{(info.access_group_models || []).map((model: string, index: number) => (
<Badge key={`ag-${index}`} color="green" title="From access group">
{model}
</Badge>
))}
</>
)}
</div>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 "All proxy models" badge hides access-group models when direct models list is empty

The condition info.models.length === 0 || info.models.includes("all-proxy-models") short-circuits to the red "All proxy models" badge even when info.access_group_models is non-empty. A team that has no directly-assigned models but inherits models from access groups will therefore display "All Proxy Models" — which is the opposite of the access restriction that is actually in place.

The same issue exists in ModelsCell.tsx where isAllModels is computed purely from team.models without considering team.access_group_models.

A straightforward fix is to treat the team as having restricted models when access_group_models are present:

// TeamInfo.tsx
const hasNoModels =
  (info.models.length === 0 || info.models.includes("all-proxy-models")) &&
  !(info.access_group_models?.length);

Then branch on hasNoModels instead of the inline condition.

Comment on lines 17 to +24
const [expandedAccordion, setExpandedAccordion] = useState<boolean>(false);

const isAllModels = !team.models || team.models.length === 0 || team.models.includes("all-proxy-models");

const modelEntries: ModelEntry[] = useMemo(() => {
if (isAllModels) return [];
const entries: ModelEntry[] = team.models.map((m) => ({
name: m,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 isAllModels ignores access-group models — same bug as in TeamInfo.tsx

isAllModels is set to true whenever team.models is empty, even if team.access_group_models has entries. Because isAllModels === true causes modelEntries to be [], the component then renders the red "All Proxy Models" badge rather than the inherited access-group models. A team restricted to, say, ["gpt-4"] via an access group would incorrectly appear to have unrestricted model access in the table.

Fix by also checking whether access-group models exist:

const isAllModels =
  (!team.models || team.models.length === 0 || team.models.includes("all-proxy-models")) &&
  !(team.access_group_models?.length);

}: PaginatedKeyAliasSelectProps) => {
const [searchInput, setSearchInput] = useState("");
const [debouncedSearch, setDebouncedSearch] = useDebouncedState("", {
wait: DEBOUNCE_MS,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hardcoded "Team ID" key couples component to filter naming convention

allFilters?.["Team ID"] relies on the filter option being named exactly "Team ID" in the parent FilterComponent. If that name ever changes (e.g. to "team_id" or "Team"), this silently returns undefined and the filter stops working with no error. Consider accepting team_id as an explicit prop rather than reading it from the opaque allFilters bag, or at least exporting the key string as a shared constant.

Extract `_apply_non_admin_alias_scope` from `key_aliases`,
`_resolve_team_access_group_resources` from `team_info`, and
`_enforce_list_team_v2_access` from `list_team_v2` to bring each
function under ruff's 50-statement limit. No behavior changes.
- useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to
  spy matchers for the new `team_id` param on `useInfiniteKeyAliases`
  and `keyAliasesCall`.
- EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider
  is not required for team-entity tests.
- ModelsCell: replace the overflow-accordion test with one that
  verifies the new collapse-on-`all-proxy-models` behavior (no
  accordion, single badge).
AntD <Select allowClear> returns undefined when the user clears the
selection. Coalescing to "" caused the team-update payload to carry
organization_id: "" instead of null, relying on the backend to coerce
it. Send null directly so the intent is explicit at the source.
@ryan-crabbe-berri ryan-crabbe-berri temporarily deployed to integration-postgres April 4, 2026 16:47 — with GitHub Actions Inactive
@ryan-crabbe-berri ryan-crabbe-berri temporarily deployed to integration-redis-postgres April 4, 2026 16:47 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri enabled auto-merge April 4, 2026 17:22
@yuneng-berri yuneng-berri merged commit fa9e3d8 into main Apr 4, 2026
109 of 116 checks passed
@yuneng-berri yuneng-berri deleted the litellm_ryan-march-31 branch April 4, 2026 17:24
fede-kamel pushed a commit to fede-kamel/litellm that referenced this pull request Apr 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants