Conversation
Set up Playwright E2E testing for the web frontend using _scripted_echo provider for deterministic, LLM-free test execution. - Add playwright.config.ts (chromium, baseURL 127.0.0.1:5494) - Add shared fixtures: createSession, apiTracker, consoleErrors - Add 6 spec files covering 16 test cases: - session-lifecycle: create, archive, delete - websocket-streaming: progressive response, multi-turn order - session-switching: cross-session isolation - refresh-resilience: recovery + API call storm detection - dynamic-bugs: console.error guard, slow network, rapid send - navigation: URL ?session= sync - Add echo-scripts.txt (50 rounds of scripted responses) - Add web-e2e CI job to ci-kimi-cli.yml - Add npm scripts: test:e2e, test:e2e:ui, test:e2e:install
| forbidOnly: !!process.env.CI, | ||
| retries: process.env.CI ? 2 : 0, | ||
| workers: 1, | ||
| reporter: process.env.CI ? "github" : "html", |
There was a problem hiding this comment.
🟡 CI Playwright reporter "github" does not produce the playwright-report/ directory that the upload step expects
In web/playwright.config.ts:9, the reporter is set to "github" when process.env.CI is truthy. The "github" reporter only emits inline annotations to stdout—it does not generate a playwright-report/ directory. However, the CI workflow at .github/workflows/ci-kimi-cli.yml:335-338 uploads web/playwright-report/ with if: always(), clearly intending to preserve the HTML report for debugging failures. Because the directory is never created in CI, the artifact upload is effectively a no-op (the default if-no-files-found is warn), so no report will ever be available when tests fail in CI—defeating the purpose of the if: always() guard.
| reporter: process.env.CI ? "github" : "html", | |
| reporter: process.env.CI ? [["github"], ["html"]] : "html", |
Was this helpful? React with 👍 or 👎 to provide feedback.
| KIMI_SCRIPTED_ECHO_TRACE = "1" | ||
| TOML | ||
| uv run kimi web --no-open --dangerously-omit-auth & | ||
| for i in $(seq 1 30); do curl -sf http://127.0.0.1:5494/healthz && break; sleep 1; done |
There was a problem hiding this comment.
🟡 Health-check loop exits successfully even when the backend never starts
At .github/workflows/ci-kimi-cli.yml:326, the health-check loop for i in $(seq 1 30); do curl -sf … && break; sleep 1; done silently succeeds even if all 30 curl attempts fail, because the loop's last command is sleep 1 (exit code 0). The workflow then proceeds to run Playwright tests against a server that isn't running, producing confusing "connection refused" errors instead of a clear "backend failed to start" message. Adding a final check (e.g. curl -sf http://127.0.0.1:5494/healthz after the loop) would make the step fail fast with a clear diagnosis.
| for i in $(seq 1 30); do curl -sf http://127.0.0.1:5494/healthz && break; sleep 1; done | |
| for i in $(seq 1 30); do curl -sf http://127.0.0.1:5494/healthz && break; sleep 1; done | |
| curl -sf http://127.0.0.1:5494/healthz || { echo "Backend failed to start after 30s"; exit 1; } |
Was this helpful? React with 👍 or 👎 to provide feedback.
Related Issue
N/A — new test infrastructure, no prior issue.
Description
Establish end-to-end testing infrastructure for the Kimi Web frontend to catch regression bugs early when frontend code changes. Tests run against a real backend using the
_scripted_echoprovider for deterministic, LLM-free execution — no mocks, no flaky network calls.What's included
web/playwright.config.ts— Chromium project, baseURL127.0.0.1:5494, on-first-retry tracesweb/tests/e2e/helpers/fixtures.ts):createSession()— creates a session via the UI dialogapiTracker— intercepts/api/**calls for request countingconsoleErrors— capturesconsole.errorandpageerroreventssendMessage()/waitForResponse()— helper functionssession-lifecycle— create → send message → archive → deletewebsocket-streaming— progressive response, multi-turn message ordersession-switching— cross-session content isolation, event leak guardrefresh-resilience— page refresh recovery, API call storm detectiondynamic-bugs— console.error guard, slow network resilience, rapid sendnavigation— URL?session=sync on create/navigate/reloadecho-scripts.txt— 50 rounds oftext:DSL responses for_scripted_echoweb-e2einci-kimi-cli.yml) — starts backend with_scripted_echo+config.toml, runs Playwright, uploads report/traces as artifactstest:e2e,test:e2e:ui,test:e2e:installHow to run locally
No backend changes
All tests use the existing
_scripted_echoprovider andtext:DSL. No modifications topackages/kosong/or the Python backend.Checklist
make gen-changelogto update the changelog.make gen-docsto update the user documentation.