test(e2e): add edit team model TPM/RPM limits test#25658
test(e2e): add edit team model TPM/RPM limits test#25658ryan-crabbe-berri merged 2 commits intomainfrom
Conversation
Covers the full write-path flow for team-scoped models on the Models + Endpoints page: create via /model/new, click the row to open the detail view, click Edit Settings, change TPM/RPM, click Save Changes, assert the new values render back. Cleans up via /model/delete in finally so reruns stay deterministic. Requires store_model_in_db: true in the fixture general_settings so the proxy accepts /model/new and /model/delete. No existing test in the dashboard e2e suite reads the all-models table or hits the model CRUD endpoints, so enabling the flag has no cross-test impact.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryAdds an end-to-end Playwright test that creates a team-scoped model via Note: the PR description states the test "cleans up via Confidence Score: 5/5Safe to merge; the only finding is a P2 style suggestion about using an env-var for the mock LLM URL. All findings are P2 or lower. The test logic is sound, the fixture change is scoped to e2e infra, and existing tests are unaffected. The api_base hard-coding is a minor consistency nit, not a correctness issue. No files require special attention.
|
| Filename | Overview |
|---|---|
| ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts | Adds 'Edit team model TPM and RPM limits' test; creates a team-scoped model via API, navigates to the UI, edits TPM/RPM, and asserts the new values are rendered; minor: api_base is hard-coded instead of using process.env.MOCK_LLM_URL |
| ui/litellm-dashboard/e2e_tests/fixtures/config.yml | Adds store_model_in_db: true to the shared e2e fixture config, required by /model/new and /model/delete endpoints used by the new test |
Sequence Diagram
sequenceDiagram
participant T as Playwright Test
participant P as LiteLLM Proxy
participant UI as Browser UI
T->>P: POST /model/new (team_id, tpm=100, rpm=200)
P-->>T: 200 OK
T->>UI: page.goto("/ui")
T->>UI: click "Models + Endpoints"
UI->>P: GET /v2/model/info
P-->>UI: model list (includes new model)
T->>UI: click model row
UI-->>T: detail view shown ("Back to Models")
T->>UI: click "Edit Settings"
T->>UI: fill TPM → "999", RPM → "888"
T->>UI: click "Save Changes"
UI->>P: PATCH /model/{id}/update
P-->>UI: 200 OK
T->>UI: assert getByText("999") visible
T->>UI: assert getByText("888") visible
Reviews (2): Last reviewed commit: "test(e2e): drop cleanup from edit team m..." | Re-trigger Greptile
Reviewer flagged that cleanup failures were silently swallowed and suggested asserting `delete.ok()`. While thinking through the fix, the actual question turned out to be "does the cleanup matter at all?" — and the answer is no. The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per invocation and tears it down at the end, so every local and CI run starts with an empty DB. Playwright retries share the same DB but each attempt creates a new model with a unique `Date.now()` name and only queries its own model, so orphans from failed attempts never collide with later attempts or other tests. Nothing else in the suite reads the all-models table. Keeping the cleanup would also turn every write test into an implicit delete test, coupling responsibilities and inflating runtime — which is probably why `teams.spec.ts` (create a team), `keys.spec.ts` (update key limits), etc. all leave their entities in place. Matching that convention, drop the try/finally block and the `createdModelId` tracking. 12 lines removed, no behavior change.
Summary
Adds an end-to-end UI test covering the full write-path flow for team-scoped models on the Models + Endpoints page.
POST /model/newwith an initial TPM/RPMPOST /model/deletein afinallyblock so Playwright retries stay deterministicFixture change
Enables
store_model_in_db: trueinui/litellm-dashboard/e2e_tests/fixtures/config.yml. The proxy's/model/newand/model/deleteendpoints require this flag — without it they return 500 with"Set 'STORE_MODEL_IN_DB='True'' in your env to enable this feature."No existing test in the suite reads the all-models table or hits the model CRUD endpoints, so enabling the flag has no cross-test impact — verified by inspection of
teams.spec.ts,keys.spec.ts, andaddModel.spec.ts(the only other test on the models page, which reads the provider dropdown).Test plan
./ui/litellm-dashboard/e2e_tests/run_e2e.sh --headed -g "Edit team model"— passes in 19.4sPOST /model/new→GET /v2/model/info→ row click →PATCH /model/{id}/update→ cleanup viaPOST /model/deletee2e_ui_testingjob passes