feat(teams): per-member model scope + team default_team_member_models#24950
feat(teams): per-member model scope + team default_team_member_models#24950ishaan-berri merged 22 commits intolitellm_ishaan_april6from
Conversation
…quest, UpdateTeamRequest
…update and team/update
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR introduces per-member model scope for LiteLLM teams: a Key findings:
Confidence Score: 3/5Not safe to merge — two P0 merge conflicts break Python test parsing and TypeScript compilation. Score of 3 reflects two P0 blockers (unresolved conflict markers in a test file and in a UI interface definition) plus a P1 for missing unit tests covering the new auth-enforcement path. The underlying backend logic is sound. tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py (merge conflict + wrong assertion) and ui/litellm-dashboard/src/components/team/TeamInfo.tsx (merge conflict in TeamData interface)
|
| Filename | Overview |
|---|---|
| tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py | Contains unresolved git merge conflict (lines 37–43) causing SyntaxError; worktree-side assertion is also logically incorrect for the given input model. |
| ui/litellm-dashboard/src/components/team/TeamInfo.tsx | Contains unresolved git merge conflict in the TeamData interface definition, preventing TypeScript compilation. |
| litellm/proxy/auth/auth_checks.py | Adds _check_team_member_model_access with correct early-return guard and llm_router threading; no unit tests provided for the new enforcement path. |
| litellm/proxy/management_endpoints/common_utils.py | _upsert_budget_and_membership updated to update-in-place when an existing budget is present; allowed_models=None vs [] semantics are documented. |
| litellm/proxy/management_endpoints/team_endpoints.py | _process_team_members seeds allowed_models from default_team_member_models when omitted; team_member_update propagates allowed_models to both upsert paths. |
| litellm/proxy/management_helpers/utils.py | add_new_member creates a budget entry when allowed_models is set, even without max_budget_in_team. |
| litellm-proxy-extras/litellm_proxy_extras/migrations/20260401000000_add_team_member_model_scope/migration.sql | Idempotent migration using IF NOT EXISTS; both columns default to empty TEXT array. |
| litellm/proxy/_types.py | default_team_member_models added to TeamBase and UpdateTeamRequest; allowed_models added to LiteLLM_BudgetTable and membership update types. |
| litellm/llms/bedrock/common_utils.py | Adds safe context-window bracket suffix stripping via regex; formatting-only change to CommonBatchFilesUtils.prepare_request. |
| ui/litellm-dashboard/src/components/team/TeamMemberTab.tsx | Adds Model Scope column; initializes allowed_models as [] so the !== undefined guard in teamMemberUpdateCall always fires. |
| ui/litellm-dashboard/src/components/team/EditMembership.tsx | Adds multi-select field type using Ant Design Select mode=multiple; wired to allowed_models form field. |
| ui/litellm-dashboard/src/components/networking.tsx | Member interface gains allowed_models; teamMemberUpdateCall includes it when !== undefined. |
| ui/litellm-dashboard/src/app/(dashboard)/teams/components/modals/CreateTeamModal.tsx | Adds default_team_member_models multi-select scoped to team models with sensible fallback to full model list. |
| schema.prisma | New fields default_team_member_models and allowed_models added consistently. |
| litellm/proxy/schema.prisma | New fields added to match proxy-extras schema. |
| litellm-proxy-extras/litellm_proxy_extras/schema.prisma | New fields with @default([]) annotations on BudgetTable and TeamTable. |
Sequence Diagram
sequenceDiagram
participant Client
participant CommonChecks as common_checks()
participant TeamCheck as can_team_access_model()
participant MemberCheck as _check_team_member_model_access()
participant Cache as user_api_key_cache / DB
Client->>CommonChecks: request(model, team_object, valid_token)
CommonChecks->>TeamCheck: check team-level model access
TeamCheck-->>CommonChecks: OK / raise HTTP 401
CommonChecks->>MemberCheck: check per-member model scope
MemberCheck->>Cache: get_team_membership(user_id, team_id)
Cache-->>MemberCheck: team_membership or None
alt allowed_models is empty or None
MemberCheck-->>CommonChecks: return (no restriction)
else allowed_models is non-empty
MemberCheck->>MemberCheck: _can_object_call_model(model, allowed_models, llm_router)
alt model in allowed_models
MemberCheck-->>CommonChecks: return OK
else model not in allowed_models
MemberCheck-->>CommonChecks: raise ProxyException HTTP 401
end
end
Comments Outside Diff (2)
-
tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py, line 37-43 (link)Unresolved git merge conflict — file will not parse
This file contains raw conflict markers (
<<<<<<< worktree-rustling-wishing-kite,=======,>>>>>>> main) that cause a PythonSyntaxErroron import, making every test in this file fail to run.Additionally, the worktree-side assertion is logically wrong: the input model is
bedrock/us-gov.anthropic.claude-haiku-4-5-20251001-v1:0, but the conflict-side assertion expects"anthropic.claude-3-5-sonnet-20240620-v1:0". Resolve the conflict by accepting themain-side value: -
ui/litellm-dashboard/src/components/team/TeamInfo.tsx, line 96-103 (link)Unresolved git merge conflict in
TeamDatainterface — TypeScript will not compileThe
TeamDatainterface definition contains raw conflict markers (<<<<<<< worktree-rustling-wishing-kite/>>>>>>> main). TypeScript will fail to compile this file, breaking the entire dashboard build.The correct resolution is to include all fields from both sides:
Reviews (5): Last reviewed commit: "Merge branch 'main' into worktree-rustli..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # 2.2. If team member has per-member model scope, enforce it | ||
| if _model and team_object and valid_token and valid_token.user_id: | ||
| with tracer.trace( | ||
| "litellm.proxy.auth.common_checks.check_team_member_model_access" | ||
| ): | ||
| await _check_team_member_model_access( | ||
| model=_model, | ||
| team_object=team_object, | ||
| valid_token=valid_token, | ||
| prisma_client=prisma_client, | ||
| user_api_key_cache=user_api_key_cache, | ||
| proxy_logging_obj=proxy_logging_obj, | ||
| ) |
There was a problem hiding this comment.
get_team_membership cache lookup on every team+model request
_check_team_member_model_access is invoked on every request where _model, team_object, and valid_token.user_id are set — i.e. the vast majority of team key requests. Internally it calls get_team_membership, which hits the cache (or DB on miss) for every one of those requests, even when no member has allowed_models set.
For the common case (no per-member restrictions), this adds a cache round-trip per request with zero benefit. Consider guarding the entire block against a lightweight signal on the team object (e.g. a boolean flag has_member_model_restrictions on the cached team, or checking if team_object.default_team_member_models is non-empty as a heuristic), so that deployments not using this feature pay no cost.
Rule Used: What: Avoid creating new database requests or Rout... (source)
| if (formValues.rpm_limit !== undefined && formValues.rpm_limit !== null) { | ||
| requestBody.rpm_limit = formValues.rpm_limit; | ||
| } | ||
| if (formValues.allowed_models !== undefined) { |
There was a problem hiding this comment.
allowed_models is always sent, even when the user has not changed it
In TeamMemberTab, the member object is pre-populated with allowed_models: membership?.litellm_budget_table?.allowed_models || []. Because [] is not undefined, the check formValues.allowed_models !== undefined will always be true, so allowed_models is included in every teamMemberUpdateCall payload — even when the admin only edits max_budget_in_team.
Until the server-side _upsert_budget_and_membership properly updates rather than replaces the budget row, this guarantees that a round-trip from the Edit Member modal will lose any server-side allowed_models that were set to a value the front-end hadn't fetched yet. More importantly, once the server side is fixed, the client should only send fields the user actually changed.
| if (formValues.rpm_limit !== undefined && formValues.rpm_limit !== null) { | |
| requestBody.rpm_limit = formValues.rpm_limit; | |
| } | |
| if (formValues.allowed_models !== undefined) { | |
| if (formValues.allowed_models !== undefined && formValues.allowed_models !== null) { | |
| requestBody.allowed_models = formValues.allowed_models; | |
| } |
| _can_object_call_model( | ||
| model=model, | ||
| llm_router=None, | ||
| models=member_allowed_models, | ||
| object_type="team", | ||
| ) |
There was a problem hiding this comment.
_can_object_call_model is called with llm_router=None. Inside that function, the llm_router is needed to resolve named access groups (e.g. a group called "gpt-4-group"). The team-level check (can_team_access_model, a few lines above at line 414) correctly passes the router, but the new member-level check does not.
In practice, if a team admin sets a member's allowed_models=["gpt-4-access-group"] and a request comes in for gpt-4, the group will never be expanded, the check will always fail, and the member will be permanently locked out even though they have valid access.
The llm_router argument is already available in common_checks via the outer function scope — it should be threaded into _check_team_member_model_access and forwarded to _can_object_call_model the same way can_team_access_model uses it.
…reating new one When a member already has a budget_id, patch only the fields the caller provided rather than always creating a fresh budget record. The old code ignored existing_budget_id entirely, so updating only allowed_models silently dropped the stored max_budget / tpm_limit / rpm_limit values.
Without the router, _can_object_call_model cannot resolve wildcard model names (e.g. openai/*) or access-group names in allowed_models, causing legitimate requests to be denied. Thread the existing llm_router from _run_common_checks through to the new member-scope check.
|
|
Groups default_team_member_models, member budget/key duration, and tpm/rpm defaults into a single collapsible section. The model picker is filtered to only show the models selected for the team, and the copy distinguishes it from the team-level Models field.
…m form Moves default_team_member_models + per-member budget/key/tpm/rpm fields into a collapsible "Team Member Settings" panel. Keeps the top-level form focused on team-wide settings (team models, team budget, tpm/rpm).
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
4 participants
Relevant issues
What this does
Two new team capabilities:
default_team_member_modelson a team — when a member is added without an explicitallowed_models, they automatically inherit this list. Set viaPOST /team/neworPOST /team/update.allowed_modelsper team member — enforced at auth time. If set (non-empty), only those models are accessible. Empty = inherit all team models (existing behavior).API changes
POST /team/new+POST /team/updateacceptdefault_team_member_models: string[]POST /team/member_addaccepts top-levelallowed_models: string[](falls back to team default if omitted)POST /team/member_updateacceptsallowed_models: string[]GET /team/inforeturnsallowed_modelsper member inteam_memberships[].litellm_budget_tableSchema changes
LiteLLM_TeamTable.default_team_member_models TEXT[] DEFAULT ARRAY[]LiteLLM_BudgetTable.allowed_models TEXT[] DEFAULT ARRAY[]UI changes
Auth enforcement
After team-level model check passes, a new check fires if the member's
allowed_modelsis non-empty — returns 401 with"Team member not allowed to access model"if the requested model isn't in the list.Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
Changes
schema.prisma/litellm-proxy-extras/schema.prisma: new fields on TeamTable and BudgetTablelitellm-proxy-extras/migrations/: migration SQL for the new columnslitellm/proxy/_types.py:allowed_modelson request typeslitellm/proxy/management_helpers/utils.py:add_new_memberaccepts + persistsallowed_modelslitellm/proxy/management_endpoints/common_utils.py:_upsert_budget_and_membershipacceptsallowed_modelslitellm/proxy/management_endpoints/team_endpoints.py: member_add seeding, member_update + team/update persistencelitellm/proxy/auth/auth_checks.py:_check_team_member_model_accessenforced in_run_common_checksTeamMemberTab,EditMembership,TeamInfo,networking.tsx