Skip to content

test(e2e): add edit team model TPM/RPM limits test#25658

Merged
ryan-crabbe-berri merged 2 commits intomainfrom
litellm_e2e-edit-team-model-test
Apr 14, 2026
Merged

test(e2e): add edit team model TPM/RPM limits test#25658
ryan-crabbe-berri merged 2 commits intomainfrom
litellm_e2e-edit-team-model-test

Conversation

@ryan-crabbe-berri
Copy link
Copy Markdown
Collaborator

Summary

Adds an end-to-end UI test covering the full write-path flow for team-scoped models on the Models + Endpoints page.

  • Creates a team-scoped model via POST /model/new with an initial TPM/RPM
  • Navigates to Models + Endpoints, clicks the new row to open the detail view
  • Clicks Edit Settings, changes TPM to 999 and RPM to 888, clicks Save Changes
  • Asserts the new values render back in view mode (consistent with the suite's existing "value-reflection" assertion style used by the key-edit test)
  • Cleans up via POST /model/delete in a finally block so Playwright retries stay deterministic

Fixture change

Enables store_model_in_db: true in ui/litellm-dashboard/e2e_tests/fixtures/config.yml. The proxy's /model/new and /model/delete endpoints require this flag — without it they return 500 with "Set 'STORE_MODEL_IN_DB='True'' in your env to enable this feature."

No existing test in the suite reads the all-models table or hits the model CRUD endpoints, so enabling the flag has no cross-test impact — verified by inspection of teams.spec.ts, keys.spec.ts, and addModel.spec.ts (the only other test on the models page, which reads the provider dropdown).

Test plan

  • Runs locally via ./ui/litellm-dashboard/e2e_tests/run_e2e.sh --headed -g "Edit team model" — passes in 19.4s
  • Proxy log confirms the full flow: POST /model/newGET /v2/model/info → row click → PATCH /model/{id}/update → cleanup via POST /model/delete
  • CircleCI e2e_ui_testing job passes

Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.

Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 14, 2026 0:07am

Request Review

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 13, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_e2e-edit-team-model-test (152d689) with main (c08fb82)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 13, 2026

Greptile Summary

Adds an end-to-end Playwright test that creates a team-scoped model via POST /model/new, navigates to the Models + Endpoints page, edits the TPM/RPM limits, and asserts the updated values are reflected in view mode. The store_model_in_db: true flag is added to the shared fixture config to satisfy the /model/new endpoint's requirement.

Note: the PR description states the test "cleans up via POST /model/delete in a finally block," but the code explicitly has no cleanup step and relies on the ephemeral Postgres container being discarded at run end. The description should be updated to avoid confusion about the test's cleanup strategy.

Confidence Score: 5/5

Safe to merge; the only finding is a P2 style suggestion about using an env-var for the mock LLM URL.

All findings are P2 or lower. The test logic is sound, the fixture change is scoped to e2e infra, and existing tests are unaffected. The api_base hard-coding is a minor consistency nit, not a correctness issue.

No files require special attention.

Important Files Changed

Filename Overview
ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts Adds 'Edit team model TPM and RPM limits' test; creates a team-scoped model via API, navigates to the UI, edits TPM/RPM, and asserts the new values are rendered; minor: api_base is hard-coded instead of using process.env.MOCK_LLM_URL
ui/litellm-dashboard/e2e_tests/fixtures/config.yml Adds store_model_in_db: true to the shared e2e fixture config, required by /model/new and /model/delete endpoints used by the new test

Sequence Diagram

sequenceDiagram
    participant T as Playwright Test
    participant P as LiteLLM Proxy
    participant UI as Browser UI

    T->>P: POST /model/new (team_id, tpm=100, rpm=200)
    P-->>T: 200 OK

    T->>UI: page.goto("/ui")
    T->>UI: click "Models + Endpoints"
    UI->>P: GET /v2/model/info
    P-->>UI: model list (includes new model)

    T->>UI: click model row
    UI-->>T: detail view shown ("Back to Models")

    T->>UI: click "Edit Settings"
    T->>UI: fill TPM → "999", RPM → "888"
    T->>UI: click "Save Changes"
    UI->>P: PATCH /model/{id}/update
    P-->>UI: 200 OK

    T->>UI: assert getByText("999") visible
    T->>UI: assert getByText("888") visible
Loading

Reviews (2): Last reviewed commit: "test(e2e): drop cleanup from edit team m..." | Re-trigger Greptile

Comment thread ui/litellm-dashboard/e2e_tests/fixtures/config.yml
Comment thread ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts Outdated
Comment thread ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts Outdated
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.

The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.

Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
@ryan-crabbe-berri ryan-crabbe-berri merged commit 87b6b51 into main Apr 14, 2026
97 of 106 checks passed
@ryan-crabbe-berri ryan-crabbe-berri deleted the litellm_e2e-edit-team-model-test branch April 14, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants