[Test] UI - E2E: Add Playwright tests with local PostgreSQL

[yuneng-berri]

Summary

• Adds a self-contained Playwright E2E test suite that uses a local PostgreSQL container instead of Neon
• Covers role-based access for all 5 user roles (proxy admin, admin viewer, internal user, internal viewer, team admin)
• Tests authentication flows (login, redirect, per-role visibility)
• Includes GHA workflow, mock LLM server, DB seed, and a local runner script (run_e2e.sh)

Changes

• New tests/ui_e2e_tests/ directory with 10 tests across 6 spec files
• run_e2e.sh auto-detects CI vs local: spins up Docker postgres locally, uses GHA service container in CI
• DB seed uses scrypt-hashed passwords matching the proxy's verify_password() format
• Selectors use getByRole("menuitem", ...) to avoid strict-mode violations and brittleness
• GHA workflow follows existing patterns from test-proxy-e2e-azure-batches.yml
• Added e2e:psql script to ui/litellm-dashboard/package.json

Testing

• All 10 tests pass consistently across 4+ consecutive local runs with 0 flakiness
• Verified on macOS with Docker Desktop

Type

✅ Test
🚄 Infrastructure

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 7, 2026 5:28pm

[CLAassistant]

All committers have signed the CLA.

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_ui_e2e_psql_pr (184050e) with main (730ba0f)}

[greptile-apps]

Greptile Summary

This PR adds a self-contained Playwright E2E test suite (tests/ui_e2e_tests/) that validates role-based UI access for all five LiteLLM user roles against a local PostgreSQL database. It includes a mock LLM server, a DB seed with correctly formatted scrypt-hashed passwords matching verify_password() in litellm/proxy/utils.py, a shell-script orchestrator (run_e2e.sh), and an e2e:psql npm script in the dashboard package.

Key changes:

• 10 Playwright tests across 6 spec files covering proxy admin, admin viewer, internal user, internal viewer, team admin, and auth flows
• run_e2e.sh auto-detects CI vs local and handles Docker PostgreSQL, Prisma schema push, proxy startup, and test execution
• fixtures/seed.sql seeds five test users with correct scrypt:-prefixed password hashes (16-byte salt + 32-byte dk, base64-encoded) validated against the actual hash_password() implementation
• Mock LLM server (FastAPI, port 8090) provides canned OpenAI-format responses for streaming and non-streaming calls

Issues found:

• The mock LLM server health-check loop in run_e2e.sh has no failure guard — if the server never starts, the script silently continues and produces confusing downstream errors (P1)
• globalSetup.ts hardcodes http://localhost:4000 instead of reading from an env var or constant, creating drift risk if the port changes (P2)
• No .github/workflows/ file was added despite the PR description listing a GHA workflow as a deliverable (P2)

Confidence Score: 4/5

Safe to merge after addressing the missing mock server failure guard; remaining issues are P2 style and completeness concerns.

One P1 finding: the mock LLM server health-check loop in run_e2e.sh has no exit guard, producing silent hard-to-diagnose failures when the mock server fails to start. The fix is a handful of lines and follows the exact pattern already used for the proxy check in the same file. Two P2 findings (hardcoded URL in globalSetup.ts, missing GHA workflow) do not affect test correctness but represent incomplete work relative to the PR's stated scope. All test logic, the seed SQL scrypt hash format, and the proxy integration are verified correct.

tests/ui_e2e_tests/run_e2e.sh (mock server health-check loop, lines 109–113) and tests/ui_e2e_tests/globalSetup.ts (hardcoded URL, line 9)

Important Files Changed

Filename	Overview
tests/ui_e2e_tests/run_e2e.sh	Orchestration script for E2E tests; missing failure guard after mock LLM server health-check loop, letting the run continue silently when the mock never starts
tests/ui_e2e_tests/globalSetup.ts	Pre-test admin login and storage-state capture; hardcodes absolute URL instead of an env var or shared constant
tests/ui_e2e_tests/playwright.config.ts	Playwright configuration is clean; correctly defines baseURL, single worker, CI/local retry and reporter branching
tests/ui_e2e_tests/fixtures/seed.sql	Database seed with correctly formatted scrypt-hashed test users and idempotent ON CONFLICT DO NOTHING inserts
tests/ui_e2e_tests/fixtures/mock_llm_server/server.py	Minimal FastAPI OpenAI-compatible mock server; clean and correct
tests/ui_e2e_tests/constants.ts	Test user credentials and page/role enums; clean
tests/ui_e2e_tests/tests/roles/proxy-admin.spec.ts	Reuses global admin storage state; includes an API call test against /team/list using the master key
tests/ui_e2e_tests/tests/roles/internal-viewer.spec.ts	Verifies absence of write-action buttons for the internal viewer role; correct role-isolation pattern
tests/ui_e2e_tests/tests/security/login-logout.spec.ts	Validates login with valid credentials and unauthenticated redirect to login page
ui/litellm-dashboard/package.json	Added e2e:psql script entry point pointing to run_e2e.sh

Sequence Diagram

sequenceDiagram
    participant S as run_e2e.sh
    participant DB as PostgreSQL
    participant ML as Mock LLM :8090
    participant PX as LiteLLM Proxy :4000
    participant PW as Playwright / globalSetup

    S->>DB: prisma db push (schema)
    S->>ML: start background process
    Note over S,ML: health poll 15 s max<br/>⚠️ no exit guard if never healthy
    S->>PX: start background process
    Note over S,PX: health poll 180 s max<br/>✅ exits with error if never healthy
    S->>DB: psql seed.sql (users, teams)
    S->>S: npm install
    S->>PW: npx playwright test
    PW->>PX: goto http://localhost:4000/ui/login (hardcoded)
    PX-->>PW: login page
    PW->>PX: fill credentials + click Login
    PX-->>PW: redirect to /ui dashboard
    PW->>PW: saveStorageState → admin.storageState.json
    loop Each spec (10 tests)
        PW->>PX: navigate / assert UI
        PX->>ML: LLM calls (if any)
        ML-->>PX: canned responses
        PX-->>PW: HTML / API response
    end

Loading

_{Reviews (5): Last reviewed commit: "Merge remote main into litellm_ui_e2e_ps..." | Re-trigger Greptile}

[greptile-apps]

Team admin test does not verify team-admin-specific behavior

The only assertion here is toBeVisible() for the "Virtual Keys" menu item, which is visible to every authenticated user regardless of role. This test would pass even if the team admin's scoping to their own teams was completely broken, since it only validates that a login succeeded.

A more meaningful test would verify something specific to the team admin role — e.g. that the user sees keys or teams scoped to their administered teams, but not teams/keys outside their scope. For example, checking that e2e-team-crud is visible but system-level admin controls (like "Admin Settings") are not.

[greptile-apps]

Fallback master key "sk-1234" duplicated inline

constants.ts already reads process.env.LITELLM_MASTER_KEY || "sk-1234" for the proxy admin password. Duplicating the same expression (with the same fallback) inside proxy-admin.spec.ts creates two places to keep in sync. More importantly, the run_e2e.sh script always sets LITELLM_MASTER_KEY to a freshly generated random value (sk-e2e-<hex32>), so the "sk-1234" fallback will only ever fire outside the test harness.

Prefer importing users[Role.ProxyAdmin].password from constants.ts instead of re-reading the env var inline:

Suggested change

	test("Can list teams via API", async ({ page }) => {
	const response = await page.request.get("/team/list", {
	headers: {
	Authorization: `Bearer ${users[Role.ProxyAdmin].password}`,
	},

(Add the required import for users and Role at the top of the file.)

[greptile-apps]

Missing playwright install step before running tests

npm install --silent only installs the Node packages — it does not download Playwright's browser binaries. Without an explicit npx playwright install call, any fresh CI agent or new developer machine will fail immediately with:

Error: browserType.launch: Executable doesn't exist at /path/to/chromium

The fix is to add the install step between npm install and npx playwright test:

Suggested change

	# --- Playwright ---
	echo "=== Installing Playwright dependencies ==="
	cd "$SCRIPT_DIR"
	npm install --silent
	echo "=== Running Playwright tests ==="
	npx playwright test "$@"
	echo "=== Installing Playwright dependencies ==="
	cd "$SCRIPT_DIR"
	npm install --silent
	npx playwright install chromium --with-deps
	echo "=== Running Playwright tests ==="
	npx playwright test "$@"
	EXIT_CODE=$?

The --with-deps flag also installs the OS-level shared libraries Chromium requires, which is commonly the missing piece in Linux CI environments.