diff --git a/docs/index.html b/docs/index.html index 01ff5125..618456a0 100644 --- a/docs/index.html +++ b/docs/index.html @@ -354,6 +354,19 @@ color: hsl(var(--muted-foreground)); } .picker-option:hover .picker-shortcut { opacity: .4; } +.picker-note { + margin-top: 20px; padding: 14px 16px; max-width: 420px; + border: 1px solid hsl(var(--border) / .6); border-radius: 10px; + background: hsl(var(--muted) / .35); color: hsl(var(--foreground)); +} +.picker-note-title { + font-size: 12px; font-weight: 600; letter-spacing: .02em; + text-transform: uppercase; color: hsl(var(--muted-foreground)); + margin-bottom: 8px; +} +.picker-note-line { + font-size: 13px; line-height: 1.5; color: hsl(var(--foreground)); +} /* ========== Projects View ========== */ .sessions-view { flex: 1; overflow-y: auto; background: hsl(var(--background)); transition: background-color .2s; } @@ -493,12 +506,13 @@
New Pane
-
-
- - -
-
-
+ + +
+
Freshclaude settings
+
Model: Provider default tracks the latest Opus release.
+
Effort: Only live capability-supported levels appear for the selected model.
+
Pinned legacy model IDs stay visible as unavailable until you change them.
+
+ + + diff --git a/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls-test-plan.md b/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls-test-plan.md new file mode 100644 index 00000000..4d31cb31 --- /dev/null +++ b/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls-test-plan.md @@ -0,0 +1,211 @@ +# Freshclaude Capability-Driven Model Controls Test Plan + +Reconciliation result: the approved testing strategy still holds. The implementation plan and current worktree do not add paid services, external infrastructure, or a larger user-facing surface than the user already approved. The main adjustment is prioritization, not scope: the capability registry, HTTP route, client capability state, persistence migration path, and browser/jsdom harnesses already exist here, so the highest-value red checks are now the unresolved effort-provenance cleanup cases and the contract that tracked saved selections remain visible instead of being silently healed. + +## Source-of-Truth Legend + +- `SoT1`: the user request and approved trycycle strategy in the transcript. Freshclaude must stop hardcoding stale model and thinking options, pick up newer Anthropic improvements automatically, and avoid silent fallback or silent migration behavior. +- `SoT2`: the implementation plan at `docs/plans/2026-04-18-freshclaude-capability-driven-model-controls.md`, especially `User-Visible Behavior`, `Contracts And Invariants`, and Task 5/6. This is the authoritative contract for provider-default `opus`, tracked vs exact selections, dynamic effort strings, stale-capability refresh, saved-selection rows, unavailable exact rows, and provenance-sensitive effort cleanup. +- `SoT3`: the concrete shared and server contracts already present in this worktree: `shared/agent-chat-capabilities.ts`, `shared/settings.ts`, `shared/ws-protocol.ts`, `server/agent-chat-capability-registry.ts`, `server/agent-chat-capabilities-router.ts`, `server/settings-router.ts`, `server/config-store.ts`, and `server/ws-handler.ts`. +- `SoT4`: the current user-facing entry points and persistence surfaces already wired in this worktree: `src/components/agent-chat/AgentChatView.tsx`, `src/components/agent-chat/AgentChatSettings.tsx`, `src/components/panes/PaneContainer.tsx`, `src/components/TabsView.tsx`, `src/store/persistMiddleware.ts`, `src/store/persistedState.ts`, `src/lib/tab-registry-snapshot.ts`, `/api/settings`, `/api/agent-chat/capabilities/:provider`, and the pane picker flow used in Playwright. +- `SoT5`: the installed Claude SDK `supportedModels()` shape described in the approved strategy. This is a differential reference for a conditional non-CI contract probe only, not the primary acceptance gate. + +## Harness Requirements + +No new harnesses need to be built. The existing harnesses already cover the required proof surfaces: + +1. **Direct API harness** + What it does: exercises the real Express routers, shared schemas, config-store persistence, registry normalization, and websocket input validation. + Exposes: `supertest`, isolated temp config directories, and real schema parsing. + Estimated complexity: none beyond extending existing tests. + Tests depending on it: 9, 10, 11. + +2. **Programmatic state and jsdom interaction harness** + What it does: mounts `AgentChatView`, `AgentChatSettings`, `PaneContainer`, and `TabsView` against the real Redux store so tests can inspect pane state, persisted settings intent, and sent websocket payloads without mocking the whole app. + Exposes: `window.__FRESHELL_TEST_HARNESS__` in browser tests, store dispatch/state inspection in jsdom, mocked `ws.send`, and mocked API routes. + Estimated complexity: none beyond extending existing tests. + Tests depending on it: 2, 3, 12, 13, 14, 16. + +3. **Browser interaction and artifact harness** + What it does: runs the real UI through Playwright with route interception, sent-websocket capture, on-disk `config.json` inspection via `TestServerInfo.homeDir`, and screenshot assertions. + Exposes: pane-picker interactions, reload and multi-page flows, sent `sdk.create` payload capture, capability-route interception, and screenshot comparison. + Estimated complexity: none beyond extending existing tests. + Tests depending on it: 1, 4, 5, 6, 7, 8, 15. + +## Test Plan + +1. **Name:** Unsupported provider-default effort is cleared from persisted Freshclaude settings after create-time validation + **Type:** scenario + **Disposition:** extend `test/e2e-browser/specs/settings-persistence-split.spec.ts` + **Harness:** browser interaction + programmatic state + output capture + **Preconditions:** Seed server settings with `agentChat.providers.freshclaude.effort = 'turbo'` and no `modelSelection`. Intercept `GET /api/agent-chat/capabilities/freshclaude` so provider-default `opus` no longer supports effort. Enable Freshclaude in the pane picker. + **Actions:** Create a Freshclaude pane from the picker, allow the automatic `sdk.create`, inspect the sent websocket payload, inspect resolved settings and on-disk `config.json`, reload, and create a second Freshclaude pane. + **Expected outcome:** The first create sends `model: 'opus'` and omits `effort`. The stale saved effort is cleared from resolved settings and from `config.json`, and the reloaded second pane also creates with `model: 'opus'` and no `effort`. Freshell does not silently substitute a different model or keep replaying the invalid effort on later panes. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT4`. + **Interactions:** pane picker, directory confirmation, capability GET, settings persistence, config-store write path, websocket `sdk.create`. + +2. **Name:** Create-time cleanup drops an unsupported pane-local effort snapshot without rewriting provider defaults + **Type:** regression + **Disposition:** extend `test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx` + **Harness:** programmatic state + jsdom interaction + **Preconditions:** Seed provider settings with a still-valid saved default effort for Freshclaude. Render a creating agent-chat pane whose restored `modelSelection` and `effort` no longer match current provider defaults and whose selected model no longer supports effort. + **Actions:** Mount `AgentChatView`, let the create path resolve and send `sdk.create`, then inspect pane state and saved-settings dispatches. + **Expected outcome:** Pane state is sanitized so the outgoing `sdk.create` omits `effort`, but no `saveServerSettingsPatch` is dispatched to clear the provider default because the stale effort belonged only to the restored pane snapshot. Sources: `SoT2`, `SoT3`, `SoT4`. + **Interactions:** create-time capability validation, pane-state mutation, settings save thunk, websocket `sdk.create`. + +3. **Name:** Passive cleanup of a restored pane snapshot clears only pane state when provider defaults still remain valid + **Type:** regression + **Disposition:** extend `test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx` + **Harness:** programmatic state + jsdom interaction + **Preconditions:** Seed provider settings with a valid saved effort override for the current provider defaults. Render a running or restored agent-chat pane whose local snapshot carries an unsupported effort for a different selected model. + **Actions:** Mount `AgentChatView` so the passive cleanup effect runs after capability resolution, then inspect pane state and saved-settings dispatches. + **Expected outcome:** The pane-local `effort` is cleared from pane state, but provider settings remain untouched and no persisted-clear request is sent. This proves Freshell distinguishes pane-local cleanup from global-default cleanup. Sources: `SoT2`, `SoT3`, `SoT4`. + **Interactions:** passive cleanup effect, capability resolution, pane merge/update flow, settings thunk. + +4. **Name:** A saved tracked model that drops out of the catalog stays visible as a synthetic saved-selection row and still launches as that tracked id + **Type:** scenario + **Disposition:** extend `test/e2e-browser/specs/settings-persistence-split.spec.ts` + **Harness:** browser interaction + programmatic state + output capture + **Preconditions:** Persist `agentChat.providers.freshclaude.modelSelection = { kind: 'tracked', modelId: 'haiku' }`. Intercept capabilities so the live catalog excludes `haiku` and contains only other live rows. Enable Freshclaude. + **Actions:** Reload, create a Freshclaude pane, open settings, inspect the selected model row, and inspect the sent `sdk.create` payload. + **Expected outcome:** Settings show `haiku (Saved selection)` with the explanatory saved-selection message. Freshell does not rewrite the saved tracked selection to provider-default or the nearest live row, and `sdk.create` still uses `model: 'haiku'` when no extra validation is required. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT4`. + **Interactions:** settings rendering, pane creation, capability GET, websocket `sdk.create`, persisted provider settings. + +5. **Name:** Provider-default tracking and tracked live-model overrides persist across reload, and switching back to provider-default clears the override + **Type:** scenario + **Disposition:** existing `test/e2e-browser/specs/settings-persistence-split.spec.ts` + **Harness:** browser interaction + programmatic state + output capture + **Preconditions:** Intercept capabilities with provider-default `opus` and at least one tracked live option such as `opus[1m]`. Enable Freshclaude. + **Actions:** Create a Freshclaude pane, switch the model from provider-default to a tracked live option, reload, create a new pane, then switch back to `Provider default (track latest Opus)`, reload again, and create another pane. + **Expected outcome:** The tracked selection persists across reload and the corresponding `sdk.create` uses the tracked model id. Switching back to provider-default removes the saved override and future creates send `model: 'opus'` with no persisted tracked selection left behind. Sources: `SoT1`, `SoT2`, `SoT4`. + **Interactions:** model select, settings persistence, reload/rehydration, pane creation, websocket `sdk.create`. + +6. **Name:** A migrated legacy exact model stays visible, blocks create, and never silently migrates to a live alias + **Type:** scenario + **Disposition:** existing `test/e2e-browser/specs/settings-persistence-split.spec.ts` plus existing `test/e2e/agent-chat-capability-settings-flow.test.tsx` + **Harness:** browser interaction + programmatic state + output capture + **Preconditions:** Persist `modelSelection = { kind: 'exact', modelId: 'claude-opus-4-6' }`. Intercept capabilities so that exact id is absent from the live catalog. + **Actions:** Reload, create a Freshclaude pane, inspect the visible failure state, open settings, inspect the selected option, then switch to provider-default and retry. + **Expected outcome:** Freshell surfaces the unavailable exact row and explanatory message, blocks create while the unavailable exact model remains selected, sends no `sdk.create`, and only proceeds after the user explicitly chooses a launchable selection. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT4`. + **Interactions:** create-time validation, settings UI, unavailable exact row rendering, retry flow, websocket `sdk.create`. + +7. **Name:** Opening settings refreshes stale capability cache and updates visible live options instead of trusting a session-lifetime snapshot + **Type:** scenario + **Disposition:** extend `test/e2e-browser/specs/agent-chat.spec.ts` and keep `test/e2e/agent-chat-capability-settings-flow.test.tsx` + **Harness:** browser interaction + output capture + **Preconditions:** Seed stale cached capabilities in client state or via fixture, then intercept the next `GET /api/agent-chat/capabilities/freshclaude` with a newer catalog that changes model rows and effort levels. + **Actions:** Open Freshclaude settings and wait for the refresh to complete, then inspect the visible model list, effort list, and screenshot artifact. + **Expected outcome:** Freshell revalidates stale capabilities when settings open, updates the visible model and effort options from the new catalog, and the settings surface screenshot reflects provider-default plus current live rows only. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT4`. + **Interactions:** settings button, stale-capability freshness check, capability GET, screenshot baseline, settings model/effort rendering. + +8. **Name:** Capability fetch failures show an explicit retryable settings error, allow safe tracked creates, and block validation-dependent creates until retry succeeds + **Type:** scenario + **Disposition:** existing `test/e2e-browser/specs/settings.spec.ts` plus existing `test/e2e/agent-chat-capability-settings-flow.test.tsx` + **Harness:** browser interaction + programmatic state + output capture + **Preconditions:** Intercept initial capability fetch to fail with a typed error, and intercept refresh to succeed. Prepare one pane that can launch safely with a tracked model id and another that requires capability validation because it has an explicit effort or exact selection. + **Actions:** Open settings, inspect the alert and retry button, create the safe tracked pane, create the validation-dependent pane, retry the capability load, and retry the blocked create. + **Expected outcome:** The settings surface shows an explicit retryable error. Safe tracked creates continue to send the tracked `model` without waiting for live validation. Validation-dependent creates are blocked until a successful refresh provides the required capability data, after which retry succeeds with the validated payload. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT4`. + **Interactions:** capability GET/refresh, error alert, retry button, create retry button, websocket `sdk.create`, capability-validation gate. + +9. **Name:** Shared settings and settings API round-trip tracked and exact selections, dynamic effort strings, clear sentinels, and legacy migration without coercion + **Type:** integration + **Disposition:** existing `test/unit/shared/settings.test.ts` plus existing `test/integration/server/settings-api.test.ts` + **Harness:** direct API harness + **Preconditions:** Real settings schema, settings router, and temp config-store directory. + **Actions:** Parse or patch representative tracked selections, exact selections, unfamiliar effort strings, `null` and empty-string clears, and legacy `defaultModel/defaultEffort` inputs. + **Expected outcome:** The shared schema accepts tracked and exact selections and non-empty dynamic effort strings, the API accepts explicit clears, clears stored selection and effort when requested, and migrates legacy `defaultModel/defaultEffort` into `exact` plus explicit `effort` without rewriting to `opus`. Sources: `SoT1`, `SoT2`, `SoT3`. + **Interactions:** shared Zod schemas, settings router normalization, config-store merge path, `/api/settings`. + +10. **Name:** Capability registry and capability router normalize runtime payloads, coalesce refreshes, honor TTL, and keep the last good catalog after failure + **Type:** integration + **Disposition:** existing `test/unit/server/agent-chat-capability-registry.test.ts` plus existing `test/integration/server/agent-chat-capabilities-router.test.ts` + **Harness:** direct API harness + **Preconditions:** Mocked SDK query factory and the real registry/router code. + **Actions:** Feed the registry mixed runtime payload shapes, concurrent refresh calls, stale-cache lookups, malformed payloads, timeout cases, and router GET/refresh requests. + **Expected outcome:** Runtime capability payloads normalize into the shared contract, one in-flight probe services concurrent refreshes, cached successes are reused within TTL, failed refreshes do not poison the last good catalog, malformed payloads yield typed errors, and the router returns typed success or failure payloads on the documented endpoints. Sources: `SoT2`, `SoT3`, `SoT5`. + **Interactions:** SDK probe query lifecycle, shared capability schema, registry cache, Express router, HTTP status mapping. + +11. **Name:** WebSocket and SDK bridge accept unfamiliar effort strings, stop broadcasting `sdk.models`, and avoid session-init capability discovery + **Type:** integration + **Disposition:** existing `test/unit/server/sdk-bridge.test.ts`, existing `test/unit/server/sdk-bridge-types.test.ts`, and existing `test/unit/server/ws-handler-sdk.test.ts` + **Harness:** direct API harness + **Preconditions:** Real websocket schema parsing and mocked SDK bridge/session plumbing. + **Actions:** Validate `sdk.create` with a non-enum effort string, route `sdk.create` and `sdk.set-model` through the websocket handler, and replay session-init flows. + **Expected outcome:** `sdk.create` accepts any non-empty effort string at the transport boundary, the handler forwards unfamiliar strings unchanged to the SDK bridge, `sdk.set-model` passes tracked ids through unchanged, and no code path emits `sdk.models` or calls `supportedModels()` during session init. Sources: `SoT1`, `SoT2`, `SoT3`. + **Interactions:** shared websocket schema, ws-handler ownership checks, sdk-bridge create/set-model, session-init replay. + +12. **Name:** Persisted layouts, pane payloads, and tab snapshots migrate legacy `model` and `effort` fields into selection strategies without losing reload or sync behavior + **Type:** integration + **Disposition:** extend `test/unit/client/store/persistedState.test.ts`, `test/unit/client/store/panesPersistence.test.ts`, `test/unit/client/lib/tab-registry-snapshot.test.ts`, `test/unit/client/components/TabsView.test.tsx`, and `test/unit/client/components/panes/PaneContainer.createContent.test.tsx` + **Harness:** programmatic state + jsdom interaction + **Preconditions:** Seed legacy persisted pane payloads and tab snapshots that still contain raw `model` and `effort` fields, plus current provider settings for new pane creation. + **Actions:** Load persisted panes, rehydrate tabs from snapshots, create a new Freshclaude pane from the pane container, and inspect normalized pane content after migration. + **Expected outcome:** Legacy `model` fields become exact selections, legacy non-empty `effort` strings survive as explicit overrides, current new-pane creation uses provider-default or saved strategy fields instead of stale dated ids, and reload or cross-device sync continues to hydrate the new strategy shape. Sources: `SoT2`, `SoT3`, `SoT4`. + **Interactions:** persisted layout parser, pane-tree validation, tab-registry snapshot serialization, TabsView rehydration, PaneContainer defaults. + +13. **Name:** The settings surface renders only provider-default, live capability rows, saved tracked rows, unavailable exact rows, and capability-derived effort controls on desktop and mobile + **Type:** scenario + **Disposition:** existing `test/unit/client/components/agent-chat/AgentChatSettings.test.tsx`, existing `test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx`, and existing `test/e2e/agent-chat-capability-settings-flow.test.tsx` + **Harness:** programmatic state + jsdom interaction + output capture + **Preconditions:** Capability states covering success, stale cache, failure, saved tracked selection missing from catalog, and unavailable exact selection. + **Actions:** Render the settings popover on desktop and mobile, open it, switch models between effort-supporting and non-effort-supporting capabilities, and trigger capability retry. + **Expected outcome:** The surface shows provider-default plus live rows only, adds a synthetic saved tracked row only when needed, adds an unavailable exact row only when needed, derives effort options exclusively from `supportedEffortLevels`, hides or replaces effort UI when the selected model has no effort support, and exposes accessible loading/error states and retry controls on both layouts. Sources: `SoT1`, `SoT2`, `SoT4`. + **Interactions:** AgentChatSettings rendering, AgentChatView on-open refresh, capability retry, responsive/mobile dialog layout. + +14. **Name:** Freshclaude runtime metadata and activity indicators remain correct while provider-default creates and tracked mid-session model changes occur + **Type:** regression + **Disposition:** existing `test/e2e/pane-header-runtime-meta-flow.test.tsx` and existing `test/e2e/pane-activity-indicator-flow.test.tsx` + **Harness:** programmatic state + jsdom interaction + **Preconditions:** Seed runtime metadata, capability catalogs, and active Freshclaude panes with visible header and activity indicators. + **Actions:** Let a provider-default Freshclaude pane create, then change the model mid-session to another tracked live model and inspect header/runtime metadata and tab activity styling. + **Expected outcome:** Provider-default create still launches with `model: 'opus'`, tracked mid-session changes send `sdk.set-model` with the tracked id, and header/runtime metadata plus activity indicator styling stay correct while settings persistence updates only the intended provider defaults. Sources: `SoT2`, `SoT3`, `SoT4`. + **Interactions:** AgentChatView, pane header/runtime metadata selectors, tab activity rendering, websocket `sdk.create`, websocket `sdk.set-model`. + +15. **Name:** The Playwright settings surface snapshot stays stable while showing capability-driven Freshclaude controls + **Type:** regression + **Disposition:** existing `test/e2e-browser/specs/agent-chat.spec.ts` + **Harness:** browser interaction + screenshot comparison + **Preconditions:** Intercept a representative capability catalog for Freshclaude and enable the provider in the picker. + **Actions:** Create a Freshclaude pane, open settings, and capture the existing screenshot baseline. + **Expected outcome:** The screenshot still shows the capability-driven surface with provider-default, live model rows, and dynamic effort rows, catching accidental UI regressions while the behavior-focused assertions cover the semantics. Sources: `SoT1`, `SoT2`, `SoT4`. + **Interactions:** pane picker, settings popover, screenshot baseline tooling. + +16. **Name:** A conditional live SDK probe still fits Freshell’s normalization and validation contract + **Type:** differential + **Disposition:** new non-CI probe adjacent to `test/unit/server/agent-chat-capability-registry.test.ts` + **Harness:** reference comparison harness + **Preconditions:** The installed Claude SDK is available in the test environment and the probe is marked opt-in so it does not block normal CI. + **Actions:** Invoke a short-lived live `supportedModels()` probe, pass the raw payload through the same normalization path the registry uses, and compare the result against Freshell’s shared capability schema and the helper assumptions used by the settings surface. + **Expected outcome:** The current live SDK payload still parses into Freshell’s capability contract without needing a new hardcoded model or effort table. If it does not, the differential probe fails with the raw payload shape preserved for diagnosis. Sources: `SoT1`, `SoT2`, `SoT3`, `SoT5`. + **Interactions:** live SDK query, capability normalization, shared schema parsing, optional diagnostics. + +17. **Name:** Large capability catalogs remain buildable without catastrophic option-building regressions + **Type:** invariant + **Disposition:** existing `test/unit/client/lib/agent-chat-capabilities.test.ts` + **Harness:** unit harness + **Preconditions:** A synthetic capability catalog large enough to represent worst-case upstream growth. + **Actions:** Build settings model options from the large catalog and measure the elapsed time with a generous threshold. + **Expected outcome:** Option building stays comfortably under the loose threshold, so any failure indicates a severe regression rather than normal test variance. Sources: `SoT1`, `SoT2`, `SoT4`. + **Interactions:** client capability helper logic, option-building path used by the settings surface. + +## Coverage Summary + +Covered action space: + +- Opening Freshclaude settings from the pane header. +- Triggering stale-capability refresh on settings open. +- Clicking `Retry model load` after capability fetch failure. +- Selecting provider-default, tracked live, saved tracked, and unavailable exact model rows. +- Selecting and clearing effort overrides, including unfamiliar dynamic strings. +- Creating a Freshclaude pane from the pane picker and confirming the starting directory. +- Retrying a failed Freshclaude create after changing settings. +- Reloading the app and rehydrating persisted settings, pane payloads, and tab snapshots. +- Persisting and clearing `agentChat.providers..modelSelection` and `effort` through `/api/settings`. +- Fetching and refreshing `/api/agent-chat/capabilities/:provider`. +- Sending `sdk.create` and `sdk.set-model` with resolved tracked or provider-default model ids. +- Preserving header metadata and activity indicators while the model-selection contract changes underneath. + +Explicitly excluded per the approved strategy: + +- No paid or external Anthropic API calls in the primary acceptance suite. Risk: upstream capability metadata could change in a way only the optional differential probe sees. +- No manual QA or human screenshot review. Risk: if a visual regression slips past the existing Playwright screenshot and DOM assertions, it will be caught later rather than by subjective inspection here. +- No broad agent-chat coverage unrelated to this feature, such as session-lost recovery, split-pane choreography, or resume-history hydration outside the touched settings and create/model-change paths. Risk: an unrelated agent-chat regression would be caught by adjacent suites, not by this feature plan’s primary gates. diff --git a/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls.md b/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls.md new file mode 100644 index 00000000..005a7d60 --- /dev/null +++ b/docs/plans/2026-04-18-freshclaude-capability-driven-model-controls.md @@ -0,0 +1,775 @@ +# Freshclaude Capability-Driven Model Controls Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use trycycle-executing to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace Freshclaude’s stale hardcoded Claude model and thinking controls with capability-driven tracked selections that automatically pick up new Anthropic model improvements without breaking persisted settings, pane restore, or tab sync. + +**Architecture:** Treat live Claude runtime capabilities as the only source of truth for the live model catalog and effort levels. Persist provider settings and pane state as selection strategies: provider-default means Freshell’s stable `opus` track, tracked selections store opaque model IDs chosen from the live capability catalog, and legacy dated IDs migrate into explicit exact selections. If a persisted tracked or exact selection later disappears from the live catalog, render a synthetic saved/unavailable row so the UI preserves the current value instead of silently healing it. Add a refreshable server-side capability registry with a typed HTTP route. The registry must obtain capabilities by creating a short-lived Claude SDK query with the same sanitized environment and Claude executable settings that `server/sdk-bridge.ts` uses for real sessions, calling `query.supportedModels()`, normalizing the result, and immediately closing that probe query. Use the registry for UI/options/validation, and require the client to revalidate stale cached capabilities when the settings UI opens or when create-time validation needs fresh data. Keep create-time resolution simple: provider-default resolves to `opus`, tracked catalog IDs can launch directly, while explicit effort overrides and unavailable exact selections are validated against the live catalog before Freshell sends them. + +**Tech Stack:** TypeScript, React 18, Redux Toolkit, Express, Claude Agent SDK, Zod, Vitest, Playwright + +--- + +## User-Visible Behavior + +Freshclaude and Kilroy must default to Freshell’s provider-default model track, which resolves to the Claude SDK alias `opus`. New Opus releases should take effect for new sessions without changing Freshell code again. + +The settings UI must show: +- a provider-default option that clearly means “track latest Opus” +- live model options from Claude runtime capabilities +- a synthetic saved-selection row when the currently persisted tracked selection is missing from the latest capability catalog +- effort options from the currently selected model capability only +- migrated legacy exact model IDs as unavailable rows when they no longer exist in the live capability catalog + +Freshell must not render stale hardcoded dated Claude model IDs or a fixed `[low, medium, high, max]` effort table anywhere in the UI, shared schema, pane validation, or WebSocket contract. + +Freshclaude/Kilroy must stop hardcoding a provider-default effort override. When the user has not explicitly chosen an effort, Freshell should omit `effort` and let the selected model use its own default behavior. + +`sdk.create` must send `model: 'opus'` for provider-default Freshclaude/Kilroy behavior. It must not omit `model`, because omission would delegate to Claude’s global default rather than Freshell’s “latest Opus” product contract. + +Tracked model IDs chosen from the live capability catalog may be sent directly to `sdk.create` and `sdk.set-model`. Today those catalog IDs are alias-style values such as `opus`, `opus[1m]`, `haiku`, or `default`, but Freshell must treat them as opaque runtime data rather than a second hardcoded allowlist. Capability fetch is required for rendering options and validating explicit effort overrides or unavailable exact selections, not for blocking a plain tracked catalog ID that Freshell already persisted from the live catalog. + +Opening the settings UI must revalidate stale cached capabilities instead of treating the first fetched catalog as session-lifetime truth. Automatic model improvements should appear after TTL expiry without requiring a page reload or server restart. + +If a selected model does not support effort, Freshell must clear any stale explicit effort override and omit `effort` from `sdk.create`. When the stale effort still matches the current provider defaults that seed new panes, Freshell must also clear the persisted provider setting; when the stale effort only exists on a restored pane snapshot that no longer matches current provider defaults, Freshell must clear only pane state. + +Because persisted legacy settings do not record whether a saved `defaultEffort` came from an old Freshclaude default or from an explicit user choice, migration must preserve any stored legacy effort value as an explicit override string. Removing provider-default effort applies only to new unsaved defaults, not to already-persisted user config. + +Legacy saved exact model IDs such as `claude-opus-4-6` must remain visible and clearly marked unavailable when they are absent from the live capability catalog. Freshell must not silently rewrite them to `opus`, `default`, or a “closest” live option. + +If capability discovery fails, the settings UI must show an explicit error with retry. Create-time behavior must only block when Freshell cannot safely validate what it is about to send, for example an unavailable exact selection or an explicit effort override whose support is unknown. + +## Contracts And Invariants + +- Runtime-selectable model IDs and effort levels come only from normalized Claude runtime capabilities. Synthetic saved/unavailable rows exist only to faithfully represent persisted selections that are absent from the current catalog; they are not a second model catalog. +- Freshell’s product default is a stable track alias, not a dated model ID. For this feature, Freshclaude and Kilroy both default to `opus`. +- Provider defaults and pane state distinguish selection from resolution: + - no stored selection means provider default track + - tracked selection means an opaque model string chosen from the live capability catalog; Freshell does not maintain its own hardcoded allowlist of alias names + - exact selection means an explicit preserved model string that is not currently represented by the live capability catalog, used first for migrated legacy values and unavailable pins +- If a tracked selection is persisted but absent from the current capability catalog, Freshell keeps treating it as a tracked opaque ID for create/set-model, and the settings UI renders a synthetic “Saved selection” row so the value does not disappear or get silently rewritten. +- Pane state must carry selection strategy, not just a resolved string. Persisted layouts, tab snapshots, and restore flows must survive reload and cross-device sync with the new shape. +- `sdk.create` and `sdk.set-model` operate on resolved strings. Provider-default resolves to `opus` for both create-time and mid-session model changes; tracked selections resolve to themselves; exact selections resolve to themselves only when still available. +- `opus` is the only model ID Freshell hardcodes on purpose, because it is the product-defined provider-default track. Every other selectable model ID must come from the live capability catalog. +- Explicit effort overrides are free-form non-empty strings at the storage/protocol layer and are validated only against the selected model’s live `supportedEffortLevels`. The Claude SDK currently types those levels as a closed union, so Freshell must re-declare the shared/storage schema as `string`, then narrow or cast only at the final SDK call boundary after capability validation. Do not hardcode the current level names into shared Zod schemas, pane validation, TypeScript unions, or transport payloads. +- Capability discovery is an explicit probe, not an ambient side effect: the registry creates a short-lived SDK query, calls `supportedModels()`, caches only successful normalized results for a bounded TTL, and serializes concurrent refreshes onto a single in-flight probe. +- The capability cache must be refreshable. Do not cache the catalog for the entire server lifetime with no invalidation path. +- Client capability state must also honor staleness. Opening settings or performing validation with stale cached capabilities must trigger a refresh instead of trusting the stale client snapshot indefinitely. +- No code path silently “heals” an unavailable exact selection to a tracked alias. +- Clearing a saved override that still matches current provider defaults must remove it from persisted settings after the round trip; in-memory `undefined` alone is not sufficient. Cleanup for a stale pane-local snapshot that no longer matches provider defaults must not rewrite provider settings. + +## File Structure + +### Create + +- `shared/agent-chat-capabilities.ts` + Shared Zod schemas and TypeScript types for normalized Claude capabilities, tracked/exact selection strategies, capability fetch responses, and capability fetch errors. +- `server/agent-chat-capability-registry.ts` + Server-side capability probe/cache abstraction around Claude SDK capability discovery, including the short-lived probe-query mechanism, normalization, in-flight refresh coalescing, TTL/refresh behavior, and explicit errors. +- `server/agent-chat-capabilities-router.ts` + Typed HTTP route for reading and refreshing capabilities. +- `src/lib/agent-chat-capabilities.ts` + Client helpers for provider-default resolution, tracked/exact selection handling, effort derivation, create-time validation, and UI option building. +- `test/unit/server/agent-chat-capability-registry.test.ts` + Unit tests for normalization, refresh behavior, TTL caching, and failure handling. +- `test/integration/server/agent-chat-capabilities-router.test.ts` + Contract coverage for the capability route. +- `test/unit/client/lib/agent-chat-capabilities.test.ts` + Unit tests for selection resolution, effort derivation, and unavailable exact modeling. +- `test/e2e/agent-chat-capability-settings-flow.test.tsx` + High-fidelity jsdom flow for capability fetch, settings rendering, persistence, and create preflight behavior. + +### Modify + +- `shared/settings.ts` + Persist model-selection strategies and explicit effort overrides instead of raw `defaultModel` plus fixed effort enums. +- `shared/ws-protocol.ts` + Remove the obsolete `sdk.models` server message and stop hardcoding effort levels in SDK message schemas. +- `server/config-store.ts` + Preserve the new settings shape through load/save/migration. +- `server/index.ts` + Mount the capability router. +- `server/settings-router.ts` + Normalize explicit clear sentinels for agent-chat selection/effort patches on the HTTP boundary. +- `server/sdk-bridge.ts` + Remove session-scoped `sdk.models` broadcast ownership and accept non-enum effort strings. +- `server/ws-handler.ts` + Keep create/set-model behavior aligned with resolved string semantics. +- `src/lib/api.ts` + Add typed capability fetch and refresh helpers. +- `src/lib/agent-chat-types.ts` + Change provider config from hardcoded `defaultModel`/`defaultEffort` to provider-default track metadata plus settings visibility. +- `src/lib/agent-chat-utils.ts` + Express Freshclaude/Kilroy provider intent as “track `opus`” with no baked-in effort override. +- `src/lib/session-type-utils.ts` + Resume/new-pane constructors must carry model-selection strategies and optional explicit effort overrides. +- `src/lib/tab-registry-snapshot.ts` + Serialize the new selection-strategy pane payload for tab sync. +- `src/lib/sdk-message-handler.ts` + Remove the obsolete `sdk.models` reducer path. +- `src/store/agentChatTypes.ts` + Replace flat `availableModels` with provider-scoped capability state and selection types. +- `src/store/agentChatSlice.ts` + Store capabilities, fetch status, and capability errors by provider. +- `src/store/agentChatThunks.ts` + Fetch/retry capabilities through the new HTTP route. +- `src/store/settingsThunks.ts` + Preserve explicit clear operations for model selection and effort overrides through serialization. +- `src/store/persistedState.ts` + Version and migrate persisted pane payloads into the new selection-strategy shape. +- `src/store/persistMiddleware.ts` + Persist the new pane payload shape with the updated schema version. +- `src/store/paneTypes.ts` + Replace pane-local raw `model?: string` assumptions with selection-strategy semantics plus an optional explicit effort override string. +- `src/store/panesSlice.ts` + Normalize/hydrate the new agent-chat pane shape. +- `src/store/paneTreeValidation.ts` + Accept the new persisted pane-content shape without hardcoded effort enums. +- `src/store/types.ts` + Remove the stale shared `AgentChatEffort` dependency from app-wide settings typings. +- `src/components/panes/PaneContainer.tsx` + New agent-chat panes should inherit provider-default track or saved overrides, not inject dated model IDs or a hardcoded effort. +- `src/components/TabsView.tsx` + Rehydrate agent-chat panes from registry snapshots using the new selection-strategy payload. +- `src/components/agent-chat/AgentChatSettings.tsx` + Render provider-default track, live capabilities, unavailable exact selections, and capability-derived effort controls. +- `src/components/agent-chat/AgentChatView.tsx` + Resolve create/model-change payloads correctly, fetch capabilities when validation is required, clear invalid effort overrides, and persist strategy changes. +- `docs/index.html` + Update the mock if the visible Freshclaude settings affordances materially change. + +### Modify Tests + +- `test/unit/shared/settings.test.ts` +- `test/integration/server/settings-api.test.ts` +- `test/unit/server/config-store.test.ts` +- `test/unit/server/sdk-bridge.test.ts` +- `test/unit/server/sdk-bridge-types.test.ts` +- `test/unit/server/ws-handler-sdk.test.ts` +- `test/unit/client/agentChatSlice.test.ts` +- `test/unit/client/store/persistedState.test.ts` +- `test/unit/client/store/panesPersistence.test.ts` +- `test/unit/client/store/agentChatThunks.test.ts` +- `test/unit/client/store/settingsThunks.test.ts` +- `test/unit/client/sdk-message-handler.test.ts` +- `test/unit/client/lib/agent-chat-utils.test.ts` +- `test/unit/client/lib/session-type-utils.test.ts` +- `test/unit/client/lib/tab-registry-snapshot.test.ts` +- `test/unit/client/components/panes/PaneContainer.createContent.test.tsx` +- `test/unit/client/components/panes/PaneContainer.test.tsx` +- `test/unit/client/components/TabsView.test.tsx` +- `test/unit/client/components/agent-chat/AgentChatSettings.test.tsx` +- `test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx` +- `test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx` +- `test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx` +- `test/e2e/agent-chat-capability-settings-flow.test.tsx` +- `test/e2e/pane-header-runtime-meta-flow.test.tsx` +- `test/e2e/pane-activity-indicator-flow.test.tsx` +- `test/e2e-browser/specs/agent-chat.spec.ts` +- `test/e2e-browser/specs/settings.spec.ts` +- `test/e2e-browser/specs/settings-persistence-split.spec.ts` + +## Strategy Gate + +The previous plan had seven real failure modes: + +1. It removed hardcoded effort choices from the UI but still left hardcoded effort enums in `shared/settings.ts`, `shared/ws-protocol.ts`, pane validation, and multiple client/server types. That would still break automatic adoption if Anthropic adds or renames effort levels. +2. It used `defaultEffort` wording inside the new settings contract even though the intended behavior is “no provider-default effort; only explicit per-user overrides.” That naming would push the implementation back toward the stale design we are replacing. +3. It changed settings persistence semantics without explicitly covering the existing `/api/settings` integration tests and client-side patch normalization that are responsible for null-sentinel clears. +4. It removed `sdk.models` transport behavior without updating the dedicated schema tests that currently encode the old message and effort enum assumptions. +5. It blocked exact effort-level future-proofing because the plan still let several transport/storage boundaries assume `'low' | 'medium' | 'high' | 'max'`. +6. It was otherwise on the right architectural path, so the correct response is a focused rewrite, not a directional reset. +7. It changed pane payload semantics without explicitly updating persisted-layout schema/versioning and migration coverage, which would make reload/localStorage restoration the likeliest silent regression path. + +The correct direction is: + +1. Persist tracked/exact/default selection strategy, not dated model IDs. +2. Treat effort overrides as dynamic strings validated against live capabilities, not as a hardcoded enum. +3. Use a refreshable capability probe as the source of truth for UI and validation. +4. Keep provider-default and tracked aliases launchable without unnecessary capability fetch blocking. +5. Remove hardcoded provider-default effort so thinking behavior can evolve with the selected model. + +## Task 1: Define Shared Selection, Dynamic Effort, And Settings Contracts + +**Files:** +- Create: `shared/agent-chat-capabilities.ts` +- Modify: `shared/settings.ts` +- Modify: `server/settings-router.ts` +- Modify: `test/unit/shared/settings.test.ts` +- Modify: `test/integration/server/settings-api.test.ts` + +- [ ] **Step 1: Identify or write the failing tests** + +Add shared-contract and settings-API tests that prove: +- agent-chat provider settings can store no selection, a tracked selection, or an exact selection +- explicit effort override is optional and separate from model selection +- effort overrides are stored as non-empty strings, not a fixed enum +- settings patches can explicitly clear model selection and effort override through the `/api/settings` contract +- the HTTP settings boundary normalizes null/empty clear sentinels for agent-chat selection and effort before patching config +- legacy `defaultModel` values migrate into exact selections without coercion to `opus` + +Run: + +```bash +npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts +``` + +Expected: FAIL because the shared selection schema, dynamic effort contract, and clear semantics do not exist yet. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts +``` + +Expected: FAIL with stale `defaultModel` and fixed-effort assumptions. + +- [ ] **Step 3: Write the minimal implementation** + +Implement: +- `shared/agent-chat-capabilities.ts` with normalized capability schemas/types and tracked/exact selection schemas/types +- `shared/settings.ts` storing `modelSelection` plus optional explicit `effort` override as a validated non-empty string +- `server/settings-router.ts` normalization for explicit null/empty clear sentinels on agent-chat provider patches +- migration from legacy `defaultModel` to exact selection without silent alias rewrite +- `/api/settings` patch acceptance for explicit null clears of the new fields + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts test/unit/server/config-store.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add shared/agent-chat-capabilities.ts shared/settings.ts server/settings-router.ts test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts test/unit/server/config-store.test.ts +git commit -m "feat: add dynamic agent chat selection contracts" +``` + +## Task 2: Build A Refreshable Claude Capability Registry And HTTP Contract + +**Files:** +- Create: `server/agent-chat-capability-registry.ts` +- Create: `server/agent-chat-capabilities-router.ts` +- Modify: `server/index.ts` +- Modify: `server/sdk-bridge.ts` +- Modify: `shared/ws-protocol.ts` +- Create: `test/unit/server/agent-chat-capability-registry.test.ts` +- Create: `test/integration/server/agent-chat-capabilities-router.test.ts` +- Modify: `test/unit/server/sdk-bridge.test.ts` +- Modify: `test/unit/server/sdk-bridge-types.test.ts` +- Modify: `test/unit/server/ws-handler-sdk.test.ts` + +- [ ] **Step 1: Identify or write the failing tests** + +Add server tests that prove: +- capability discovery normalizes full model info including effort/adaptive-thinking flags +- capability discovery creates a short-lived SDK query probe, closes it after `supportedModels()`, and reuses one in-flight probe for concurrent refreshes +- the registry caches successful results only for a bounded TTL and supports explicit refresh +- malformed or incomplete SDK payloads fail clearly +- the capability route returns a typed error payload on probe failure +- websocket flows no longer depend on `sdk.models` +- SDK transport/input schemas accept dynamic effort strings instead of a fixed enum + +Run: + +```bash +npm run test:vitest -- test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: FAIL because the registry, route, websocket cleanup, and dynamic effort transport contract do not exist yet. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Implement the server capability path: +- build the probe abstraction around a short-lived Claude SDK query created solely for capability discovery; share the same env sanitization, `pathToClaudeCodeExecutable`, and MCP wiring rules that `server/sdk-bridge.ts` uses, then call `supportedModels()` and immediately close the probe query +- coalesce concurrent refresh requests onto one in-flight probe and cache only successful normalized results; do not let a failed refresh poison the last known-good catalog +- normalize and validate the SDK capability payload against shared schemas +- expose GET and refresh semantics through the HTTP route +- remove session-scoped `sdk.models` broadcast behavior +- stop hardcoding current effort strings in `shared/ws-protocol.ts` and `server/sdk-bridge.ts` + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts test/unit/server/config-store.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add server/agent-chat-capability-registry.ts server/agent-chat-capabilities-router.ts server/index.ts server/sdk-bridge.ts shared/ws-protocol.ts test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts test/unit/server/config-store.test.ts +git commit -m "feat: add refreshable claude capability registry" +``` + +## Task 3: Migrate Settings, Pane State, And Sync Payloads To Selection Strategies + +**Files:** +- Modify: `server/config-store.ts` +- Modify: `src/store/settingsThunks.ts` +- Modify: `src/store/persistedState.ts` +- Modify: `src/store/persistMiddleware.ts` +- Modify: `src/lib/agent-chat-types.ts` +- Modify: `src/lib/agent-chat-utils.ts` +- Modify: `src/lib/session-type-utils.ts` +- Modify: `src/lib/tab-registry-snapshot.ts` +- Modify: `src/store/paneTypes.ts` +- Modify: `src/store/panesSlice.ts` +- Modify: `src/store/paneTreeValidation.ts` +- Modify: `src/store/types.ts` +- Modify: `src/components/panes/PaneContainer.tsx` +- Modify: `src/components/TabsView.tsx` +- Modify: `test/unit/server/config-store.test.ts` +- Modify: `test/unit/client/store/persistedState.test.ts` +- Modify: `test/unit/client/store/panesPersistence.test.ts` +- Modify: `test/unit/client/store/settingsThunks.test.ts` +- Modify: `test/unit/client/lib/agent-chat-utils.test.ts` +- Modify: `test/unit/client/lib/session-type-utils.test.ts` +- Modify: `test/unit/client/lib/tab-registry-snapshot.test.ts` +- Modify: `test/unit/client/components/panes/PaneContainer.createContent.test.tsx` +- Modify: `test/unit/client/components/panes/PaneContainer.test.tsx` +- Modify: `test/unit/client/components/TabsView.test.tsx` + +- [ ] **Step 1: Identify or write the failing tests** + +Extend tests so they prove: +- Freshclaude/Kilroy provider config expresses provider-default track `opus` with no baked-in effort override +- settings thunk normalization sends null sentinels for cleared selection and cleared effort override +- settings clears actually remove stored model selection and explicit effort override +- persisted legacy `defaultEffort` values migrate into explicit overrides because their provenance is unknowable +- persisted layout parsing and persistence migrate legacy `model`/`effort` pane fields into selection strategy plus optional explicit effort override +- pane-schema versioning changes are explicit and old localStorage payloads still load into the new shape +- pane creation, layout hydration, tab snapshot serialization, and TabsView rehydration preserve selection strategy +- no path injects `claude-opus-4-6` or a hardcoded effort into new agent-chat panes +- pane validation accepts any non-empty effort override string and rejects empty ones + +Run: + +```bash +npm run test:vitest -- test/unit/server/config-store.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx +``` + +Expected: FAIL because provider config and pane payloads still assume raw model strings and fixed effort semantics. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/server/config-store.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Implement the strategy cutover across: +- settings persistence and clear serialization +- migration that preserves any persisted legacy `defaultEffort` as an explicit override while removing provider-level hardcoded defaults for new panes +- persisted-layout schema/version bump plus legacy pane migration in localStorage +- provider metadata +- pane types and pane normalization +- persisted pane-tree validation +- tab snapshot serialization and rehydration + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/server/config-store.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts test/unit/server/config-store.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add server/config-store.ts src/store/settingsThunks.ts src/store/persistedState.ts src/store/persistMiddleware.ts src/lib/agent-chat-types.ts src/lib/agent-chat-utils.ts src/lib/session-type-utils.ts src/lib/tab-registry-snapshot.ts src/store/paneTypes.ts src/store/panesSlice.ts src/store/paneTreeValidation.ts src/store/types.ts src/components/panes/PaneContainer.tsx src/components/TabsView.tsx test/unit/server/config-store.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx +git commit -m "refactor: persist agent chat selections as strategies" +``` + +## Task 4: Add Client Capability State, Fetch Lifecycle, And Resolution Helpers + +**Files:** +- Create: `src/lib/agent-chat-capabilities.ts` +- Modify: `src/lib/api.ts` +- Modify: `src/store/agentChatTypes.ts` +- Modify: `src/store/agentChatSlice.ts` +- Modify: `src/store/agentChatThunks.ts` +- Modify: `src/lib/sdk-message-handler.ts` +- Modify: `test/unit/client/lib/agent-chat-capabilities.test.ts` +- Modify: `test/unit/client/agentChatSlice.test.ts` +- Modify: `test/unit/client/store/agentChatThunks.test.ts` +- Modify: `test/unit/client/sdk-message-handler.test.ts` + +- [ ] **Step 1: Identify or write the failing tests** + +Add client tests that prove: +- capabilities are stored per provider with `idle/loading/succeeded/failed` status +- provider-default resolves to tracked alias `opus` +- tracked aliases resolve without dated-ID remapping +- exact legacy selections become explicit unavailable state when absent from the live catalog +- effort options come only from the resolved capability +- the client never assumes a fixed set of effort strings +- `sdk.models` reducer handling is gone + +Run: + +```bash +npm run test:vitest -- test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/sdk-message-handler.test.ts +``` + +Expected: FAIL because the client still stores one flat `availableModels` array. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/sdk-message-handler.test.ts +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Implement: +- typed capability fetch helpers +- provider-scoped capability state and retry behavior +- shared selection-resolution and effort-validation helpers +- removal of websocket `sdk.models` + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/sdk-message-handler.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/sdk-message-handler.test.ts test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/lib/agent-chat-capabilities.ts src/lib/api.ts src/store/agentChatTypes.ts src/store/agentChatSlice.ts src/store/agentChatThunks.ts src/lib/sdk-message-handler.ts test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/sdk-message-handler.test.ts test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx +git commit -m "feat: add capability-driven agent chat client state" +``` + +## Task 5: Resolve Create-Time And Mid-Session Model Changes Correctly + +**Files:** +- Modify: `src/components/agent-chat/AgentChatView.tsx` +- Modify: `server/ws-handler.ts` +- Modify: `test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx` +- Modify: `test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx` +- Modify: `test/e2e/pane-header-runtime-meta-flow.test.tsx` +- Modify: `test/e2e/pane-activity-indicator-flow.test.tsx` +- Modify: `test/unit/server/ws-handler-sdk.test.ts` + +- [ ] **Step 1: Identify or write the failing tests** + +Add tests that prove: +- provider-default Freshclaude/Kilroy create sends `model: 'opus'` +- tracked aliases can create without a blocking capability fetch when no effort validation is needed +- explicit effort overrides wait for capability validation and are cleared when unsupported +- unavailable exact selections block create with a clear error +- mid-session model changes send resolved strings and clear invalid effort overrides from persisted defaults only when the pane still matches current provider defaults +- create-time and passive cleanup drop unsupported pane-local effort snapshots without rewriting provider defaults when the pane no longer matches current defaults +- create and set-model paths pass through non-enum effort strings when the live capability list allows them + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: FAIL because create/model-change flow still assumes raw pane `model` strings and hardcoded defaults. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Implement runtime resolution in `AgentChatView`: +- send provider-default/tracked aliases directly when safe +- fetch capabilities when required for effort validation or unavailable exact detection +- omit `effort` when no explicit override is active +- when invalid effort is cleared, compare the pane’s current selection/effort against current provider settings before deciding whether to persist a settings clear +- keep `sdk.create` and `sdk.set-model` payloads consistent with resolved strings + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx test/unit/server/ws-handler-sdk.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx test/unit/server/ws-handler-sdk.test.ts test/unit/client/lib/agent-chat-capabilities.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/components/agent-chat/AgentChatView.tsx server/ws-handler.ts test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx test/unit/server/ws-handler-sdk.test.ts test/unit/client/lib/agent-chat-capabilities.test.ts +git commit -m "feat: resolve freshclaude creates from tracked selections" +``` + +## Task 6: Rebuild The Settings UI Around Live Capabilities + +**Files:** +- Modify: `src/components/agent-chat/AgentChatSettings.tsx` +- Modify: `src/components/agent-chat/AgentChatView.tsx` +- Modify: `src/lib/agent-chat-capabilities.ts` +- Modify: `test/unit/client/components/agent-chat/AgentChatSettings.test.tsx` +- Modify: `test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx` +- Modify: `test/e2e/agent-chat-capability-settings-flow.test.tsx` + +- [ ] **Step 1: Identify or write the failing tests** + +Add UI tests that prove: +- the model control shows provider-default track plus live capability rows, and adds a synthetic saved-selection row only when the currently selected tracked model is missing from the latest catalog +- provider-default explains “track latest Opus” +- opening settings revalidates stale cached capabilities instead of treating them as session-lifetime truth +- a migrated unavailable exact model renders clearly and stays selected until the user changes it +- effort options come from `supportedEffortLevels` +- choosing a model without effort support hides/disables effort and clears any stale saved override +- loading and error states are explicit and accessible on desktop and mobile +- the rendered effort choices are whatever the capability payload says, not a locally defined canonical list + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx +``` + +Expected: FAIL because the UI still renders hardcoded dated models and static effort options. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Rebuild the settings UI so it renders: +- provider-default track row +- live capability rows +- synthetic saved-selection row when the selected tracked model is missing from the latest catalog +- unavailable exact row when needed +- effort UI only from the selected capability +- stale-cache refresh on settings open +- explicit retryable load/error states + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx +``` + +Expected: PASS. + +- [ ] **Step 5: Refactor and verify** + +Run: + +```bash +npm run test:vitest -- test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/components/agent-chat/AgentChatSettings.tsx src/components/agent-chat/AgentChatView.tsx src/lib/agent-chat-capabilities.ts test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx +git commit -m "feat: render freshclaude settings from live capabilities" +``` + +## Task 7: Browser Coverage, Docs, And Final Verification + +**Files:** +- Modify: `test/e2e-browser/specs/agent-chat.spec.ts` +- Modify: `test/e2e-browser/specs/settings.spec.ts` +- Modify: `test/e2e-browser/specs/settings-persistence-split.spec.ts` +- Modify: `docs/index.html` +- Modify: any touched files if final cleanup is required + +- [ ] **Step 1: Identify or write the failing browser tests** + +Extend browser coverage to prove: +- a new Freshclaude pane defaults to provider-default latest-Opus tracking +- default create sends `model: 'opus'` +- switching to another tracked live model persists and survives reload +- switching a saved override back to provider-default clears persistence and survives reload +- a saved tracked selection that disappears from the latest catalog remains visible as a saved row instead of being silently rewritten +- a saved legacy exact model is surfaced clearly after reload instead of being silently migrated +- opening settings after the client-side capability cache goes stale refreshes and shows the latest catalog +- capability fetch failure shows a visible settings error and only blocks create when validation is actually required +- a capability payload with unfamiliar effort strings still renders and round-trips correctly + +Run: + +```bash +npm run test:e2e -- test/e2e-browser/specs/agent-chat.spec.ts test/e2e-browser/specs/settings.spec.ts test/e2e-browser/specs/settings-persistence-split.spec.ts +``` + +Expected: FAIL because browser fixtures still assume hardcoded model tables, raw `defaultModel`, or fixed effort strings. `npm run test:e2e` is the correct Playwright runner for `test/e2e-browser/specs/*` in this repo. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: + +```bash +npm run test:e2e -- test/e2e-browser/specs/agent-chat.spec.ts test/e2e-browser/specs/settings.spec.ts test/e2e-browser/specs/settings-persistence-split.spec.ts +``` + +Expected: FAIL. + +- [ ] **Step 3: Write the minimal implementation** + +Finish the cutover: +- update browser fixtures/mocks to use the capability HTTP route +- update `docs/index.html` if the mock includes Freshclaude settings + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: + +```bash +npm run test:e2e -- test/e2e-browser/specs/agent-chat.spec.ts test/e2e-browser/specs/settings.spec.ts test/e2e-browser/specs/settings-persistence-split.spec.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Run the full required verification** + +Run: + +```bash +FRESHELL_TEST_SUMMARY="freshclaude capability tracks targeted" npm run test:vitest -- test/unit/shared/settings.test.ts test/integration/server/settings-api.test.ts test/unit/server/agent-chat-capability-registry.test.ts test/integration/server/agent-chat-capabilities-router.test.ts test/unit/server/config-store.test.ts test/unit/server/sdk-bridge.test.ts test/unit/server/sdk-bridge-types.test.ts test/unit/server/ws-handler-sdk.test.ts test/unit/client/lib/agent-chat-capabilities.test.ts test/unit/client/lib/agent-chat-utils.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/tab-registry-snapshot.test.ts test/unit/client/agentChatSlice.test.ts test/unit/client/store/persistedState.test.ts test/unit/client/store/panesPersistence.test.ts test/unit/client/store/agentChatThunks.test.ts test/unit/client/store/settingsThunks.test.ts test/unit/client/sdk-message-handler.test.ts test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/TabsView.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.test.tsx test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx test/e2e/agent-chat-capability-settings-flow.test.tsx test/e2e/pane-header-runtime-meta-flow.test.tsx test/e2e/pane-activity-indicator-flow.test.tsx +npm run typecheck +npm run lint +npm run test:e2e -- test/e2e-browser/specs/agent-chat.spec.ts test/e2e-browser/specs/settings.spec.ts test/e2e-browser/specs/settings-persistence-split.spec.ts +FRESHELL_TEST_SUMMARY="freshclaude capability tracks full suite" npm test +``` + +Expected: all PASS. + +- [ ] **Step 6: Final refactor pass** + +Remove dead code: +- stale hardcoded model and effort arrays +- websocket `sdk.models` types, mocks, and reducers +- comments that still describe omission-based or dated-ID defaults +- any remaining fixed effort unions or validators + +Re-run the smallest relevant checks if cleanup changes behavior. + +- [ ] **Step 7: Commit** + +```bash +git status --short +# Review the touched list, then stage explicit paths rather than using git add -A. +# At minimum this task should stage the updated browser specs, docs/index.html if touched, +# and any final cleanup edits from previously-touched Freshclaude capability files. +git commit -m "chore: finalize capability-driven freshclaude model controls" +``` + +## Notes For Execution + +- Use only the implementation worktree: `/home/user/code/freshell/.worktrees/trycycle-freshclaude-capabilities`. +- Keep commits small and aligned with the tasks above. +- Do not reintroduce stale dated model tables, fixed effort option tables, or “closest model” migration heuristics. +- Do not hardcode a provider-default effort override; explicit effort should remain opt-in and capability-validated. +- Do not invent a second hardcoded alias allowlist. Aside from the deliberate provider-default `opus` contract, tracked model IDs must flow from the runtime capability catalog as opaque strings. +- Do not silently drop or rewrite a tracked selection that is absent from the current catalog; represent it as a saved selection row and keep treating it as an opaque tracked ID unless validation specifically requires a live capability. +- Do not hardcode the current effort level names into shared schemas, pane validators, or TypeScript unions; future upstream effort strings must round-trip without another Freshell code change. +- Keep capability fetches refreshable and retryable; a server restart must not be the only way to see new capability metadata. +- When invalid effort cleanup runs, rewrite provider defaults only if the pane still matches the current provider settings; stale pane-local snapshots should be sanitized locally without clobbering global defaults. +- Do not block provider-default or tracked-alias creates unless Freshell genuinely lacks the information required to validate an explicit override. diff --git a/server/agent-chat-capabilities-router.ts b/server/agent-chat-capabilities-router.ts new file mode 100644 index 00000000..a08328c2 --- /dev/null +++ b/server/agent-chat-capabilities-router.ts @@ -0,0 +1,45 @@ +import { Router } from 'express' +import { z } from 'zod' + +import { AgentChatCapabilitiesResponseSchema } from '../shared/agent-chat-capabilities.js' + +const ProviderSchema = z.string().trim().min(1) + +type AgentChatCapabilityRegistryLike = { + getCapabilities: (provider: string) => Promise + refreshCapabilities: (provider: string) => Promise +} + +export function createAgentChatCapabilitiesRouter( + deps: { registry: AgentChatCapabilityRegistryLike }, +): Router { + const router = Router() + + router.get('/:provider', async (req, res) => { + const parsedProvider = ProviderSchema.safeParse(req.params.provider) + if (!parsedProvider.success) { + return res.status(400).json({ error: 'Invalid provider' }) + } + + const result = AgentChatCapabilitiesResponseSchema.parse( + await deps.registry.getCapabilities(parsedProvider.data), + ) + + return res.status(result.ok ? 200 : 503).json(result) + }) + + router.post('/:provider/refresh', async (req, res) => { + const parsedProvider = ProviderSchema.safeParse(req.params.provider) + if (!parsedProvider.success) { + return res.status(400).json({ error: 'Invalid provider' }) + } + + const result = AgentChatCapabilitiesResponseSchema.parse( + await deps.registry.refreshCapabilities(parsedProvider.data), + ) + + return res.status(result.ok ? 200 : 503).json(result) + }) + + return router +} diff --git a/server/agent-chat-capability-registry.ts b/server/agent-chat-capability-registry.ts new file mode 100644 index 00000000..bcd34379 --- /dev/null +++ b/server/agent-chat-capability-registry.ts @@ -0,0 +1,302 @@ +import { + query, + type Query as SdkQuery, +} from '@anthropic-ai/claude-agent-sdk' + +import { + AGENT_CHAT_CAPABILITY_CACHE_TTL_MS, + AgentChatCapabilitiesResponseSchema, + AgentChatCapabilitiesSchema, + AgentChatCapabilityErrorSchema, + AgentChatModelCapabilitySchema, + type AgentChatCapabilitiesResponse, + type AgentChatModelCapability, +} from '../shared/agent-chat-capabilities.js' +import { formatModelDisplayName } from '../shared/format-model-name.js' +import { createClaudeSdkOptions } from './sdk-bridge.js' +import { logger } from './logger.js' + +const log = logger.child({ component: 'agent-chat-capability-registry' }) +const DEFAULT_PROBE_TIMEOUT_MS = 10_000 + +type ProbeQuery = Pick + +type RegistryOptions = { + queryFactory?: typeof query + now?: () => number + ttlMs?: number + probeTimeoutMs?: number +} + +type CachedCatalog = { + fetchedAt: number + models: AgentChatModelCapability[] +} + +function invalidCapabilityPayload(message: string): Error & { + code: string + retryable: boolean +} { + return AgentChatCapabilityRegistry.createError( + 'CAPABILITY_PAYLOAD_INVALID', + message, + false, + ) +} + +function normalizeStringList(value: unknown, fieldLabel: string, modelId: string): string[] { + if (value === undefined || value === null) { + return [] + } + + if (!Array.isArray(value)) { + throw invalidCapabilityPayload( + `Capability payload has invalid ${fieldLabel} for ${modelId}`, + ) + } + + return value.map((entry) => { + if (typeof entry !== 'string') { + throw invalidCapabilityPayload( + `Capability payload has invalid ${fieldLabel} for ${modelId}`, + ) + } + + const trimmed = entry.trim() + if (trimmed.length === 0) { + throw invalidCapabilityPayload( + `Capability payload has invalid ${fieldLabel} for ${modelId}`, + ) + } + + return trimmed + }) +} + +function formatCapabilityDisplayName(rawName: string): string { + const formatted = formatModelDisplayName(rawName) + if (formatted !== rawName) { + return formatted + } + + const aliasMatch = rawName.match(/^([a-z]+)(\[[^\]]+\])?$/) + if (!aliasMatch) { + return rawName + } + + const base = aliasMatch[1].charAt(0).toUpperCase() + aliasMatch[1].slice(1) + return aliasMatch[2] ? `${base} ${aliasMatch[2]}` : base +} + +function readModelId(model: Record): string { + const value = typeof model.value === 'string' ? model.value.trim() : '' + if (value.length > 0) { + return value + } + + const id = typeof model.id === 'string' ? model.id.trim() : '' + if (id.length > 0) { + return id + } + + throw AgentChatCapabilityRegistry.createError( + 'CAPABILITY_PAYLOAD_INVALID', + 'Capability payload is missing a model id', + false, + ) +} + +function normalizeModelCapability(rawModel: unknown): AgentChatModelCapability { + if (!rawModel || typeof rawModel !== 'object' || Array.isArray(rawModel)) { + throw AgentChatCapabilityRegistry.createError( + 'CAPABILITY_PAYLOAD_INVALID', + 'Capability payload must contain model objects', + false, + ) + } + + const model = rawModel as Record + const id = readModelId(model) + const rawDisplayName = typeof model.displayName === 'string' + ? model.displayName.trim() + : typeof model.display_name === 'string' + ? model.display_name.trim() + : id + const supportedEffortLevels = normalizeStringList( + model.supportedEffortLevels + ?? model.supported_effort_levels + ?? model.effortLevels + ?? model.effort_levels, + 'supported effort levels', + id, + ) + // The settings surface renders effort controls from the live levels list, so + // normalize the boolean to that same source of truth and avoid UI/runtime drift. + const supportsEffort = supportedEffortLevels.length > 0 + const supportsAdaptiveThinking = typeof model.supportsAdaptiveThinking === 'boolean' + ? model.supportsAdaptiveThinking + : typeof model.supports_adaptive_thinking === 'boolean' + ? model.supports_adaptive_thinking + : false + + return AgentChatModelCapabilitySchema.parse({ + id, + displayName: formatCapabilityDisplayName(rawDisplayName || id), + description: typeof model.description === 'string' ? model.description : undefined, + supportsEffort, + supportedEffortLevels, + supportsAdaptiveThinking, + }) +} + +export function normalizeAgentChatCapabilityCatalog(rawModels: unknown): AgentChatModelCapability[] { + if (!Array.isArray(rawModels)) { + throw invalidCapabilityPayload('Capability payload must be an array of models') + } + + return rawModels.map((rawModel) => normalizeModelCapability(rawModel)) +} + +export class AgentChatCapabilityRegistry { + private readonly queryFactory: typeof query + private readonly now: () => number + private readonly ttlMs: number + private readonly probeTimeoutMs: number + private cachedCatalog: CachedCatalog | null = null + private inFlightRefresh: Promise | null = null + + constructor(options: RegistryOptions = {}) { + this.queryFactory = options.queryFactory ?? query + this.now = options.now ?? Date.now + this.ttlMs = options.ttlMs ?? AGENT_CHAT_CAPABILITY_CACHE_TTL_MS + this.probeTimeoutMs = options.probeTimeoutMs ?? DEFAULT_PROBE_TIMEOUT_MS + } + + static createError(code: string, message: string, retryable: boolean): Error & { + code: string + retryable: boolean + } { + return Object.assign(new Error(message), { code, retryable }) + } + + async getCapabilities(provider: string): Promise { + const cached = this.cachedCatalog + if (cached && this.now() - cached.fetchedAt <= this.ttlMs) { + return this.createSuccess(provider, cached) + } + + try { + const catalog = await this.refreshCatalog() + return this.createSuccess(provider, catalog) + } catch (error) { + return this.createFailure(error) + } + } + + async refreshCapabilities(provider: string): Promise { + try { + const catalog = await this.refreshCatalog() + return this.createSuccess(provider, catalog) + } catch (error) { + return this.createFailure(error) + } + } + + private async refreshCatalog(): Promise { + if (this.inFlightRefresh) { + return this.inFlightRefresh + } + + const task = this.probeCatalog() + .then((catalog) => { + this.cachedCatalog = catalog + return catalog + }) + .finally(() => { + this.inFlightRefresh = null + }) + + this.inFlightRefresh = task + return task + } + + private async probeCatalog(): Promise { + const abortController = new AbortController() + const probeQuery = this.queryFactory({ + prompt: (async function* emptyPrompt() {})(), + options: createClaudeSdkOptions({ + abortController, + stderr: (data: string) => { + log.warn({ data: data.trimEnd() }, 'Capability probe stderr') + }, + }), + }) as ProbeQuery + let timeoutId: ReturnType | undefined + + try { + const rawModels = await Promise.race([ + Promise.resolve(probeQuery.supportedModels()), + new Promise((_, reject) => { + timeoutId = setTimeout(() => { + abortController.abort() + reject(AgentChatCapabilityRegistry.createError( + 'CAPABILITY_PROBE_FAILED', + `Capability probe timed out after ${this.probeTimeoutMs}ms`, + true, + )) + }, this.probeTimeoutMs) + }), + ]) + return { + fetchedAt: this.now(), + models: normalizeAgentChatCapabilityCatalog(rawModels), + } + } catch (error) { + if (error instanceof Error && 'code' in error && 'retryable' in error) { + throw error + } + throw AgentChatCapabilityRegistry.createError( + 'CAPABILITY_PROBE_FAILED', + error instanceof Error ? error.message : String(error), + true, + ) + } finally { + if (timeoutId !== undefined) { + clearTimeout(timeoutId) + } + try { + await Promise.resolve(probeQuery.close()) + } catch (closeError) { + log.warn({ err: closeError }, 'Capability probe close failed') + } + } + } + + private createSuccess(provider: string, catalog: CachedCatalog): AgentChatCapabilitiesResponse { + return AgentChatCapabilitiesResponseSchema.parse({ + ok: true, + capabilities: AgentChatCapabilitiesSchema.parse({ + provider, + fetchedAt: catalog.fetchedAt, + models: catalog.models, + }), + }) + } + + private createFailure(error: unknown): AgentChatCapabilitiesResponse { + const normalized = AgentChatCapabilityErrorSchema.parse({ + code: typeof (error as { code?: unknown })?.code === 'string' + ? (error as { code: string }).code + : 'CAPABILITY_PROBE_FAILED', + message: error instanceof Error ? error.message : String(error), + retryable: typeof (error as { retryable?: unknown })?.retryable === 'boolean' + ? (error as { retryable: boolean }).retryable + : true, + }) + + return AgentChatCapabilitiesResponseSchema.parse({ + ok: false, + error: normalized, + }) + } +} diff --git a/server/index.ts b/server/index.ts index 46e16845..a19e90f6 100644 --- a/server/index.ts +++ b/server/index.ts @@ -51,6 +51,8 @@ import { checkForUpdate, createCachedUpdateChecker } from './updater/version-che import { SessionAssociationCoordinator } from './session-association-coordinator.js' import { collectAppliedSessionAssociations } from './session-association-updates.js' import { loadOrCreateServerInstanceId } from './instance-id.js' +import { createAgentChatCapabilitiesRouter } from './agent-chat-capabilities-router.js' +import { AgentChatCapabilityRegistry } from './agent-chat-capability-registry.js' import { createSettingsRouter } from './settings-router.js' import { createPerfRouter } from './perf-router.js' import { createAiRouter } from './ai-router.js' @@ -193,6 +195,7 @@ async function main() { const sessionRepairService = getSessionRepairService({ skipDiscovery: true }) const serverInstanceId = await loadOrCreateServerInstanceId() + const agentChatCapabilityRegistry = new AgentChatCapabilityRegistry() let sdkBridge: SdkBridge @@ -437,6 +440,9 @@ async function main() { applyDebugLogging, validCliProviders: allCliNames, })) + app.use('/api/agent-chat/capabilities', createAgentChatCapabilitiesRouter({ + registry: agentChatCapabilityRegistry, + })) // --- Network management endpoints --- app.use('/api', createNetworkRouter({ diff --git a/server/sdk-bridge.ts b/server/sdk-bridge.ts index 8a72d4b7..9a8b5ca3 100644 --- a/server/sdk-bridge.ts +++ b/server/sdk-bridge.ts @@ -9,11 +9,12 @@ import { type SDKPartialAssistantMessage, type SDKStatusMessage, type Query as SdkQuery, + type Options as SdkOptions, + type CanUseTool, } from '@anthropic-ai/claude-agent-sdk' import type { PermissionResult, PermissionUpdate } from '@anthropic-ai/claude-agent-sdk' import { buildMcpServerCommandArgs } from './mcp/config-writer.js' import { sanitizeAgentChatPluginPaths } from '../shared/agent-chat-plugins.js' -import { formatModelDisplayName } from '../shared/format-model-name.js' import { logger } from './logger.js' import { synthesizeLiveMessageId } from './agent-timeline/ledger.js' import type { AgentHistorySource } from './agent-timeline/history-source.js' @@ -45,10 +46,66 @@ interface SessionProcess { inputStream: InputStreamHandle } +type ClaudeSdkOptionsInput = { + cwd?: string + resumeSessionId?: string + model?: string + permissionMode?: string + effort?: string + plugins?: string[] + abortController?: AbortController + includePartialMessages?: boolean + stderr?: (data: string) => void + canUseTool?: CanUseTool + env?: NodeJS.ProcessEnv +} + +export function createClaudeSdkCleanEnv(env: NodeJS.ProcessEnv = process.env): NodeJS.ProcessEnv { + const { CLAUDECODE: _, ANTHROPIC_API_KEY: __, ...cleanEnv } = env + return cleanEnv +} + +export function createClaudeSdkMcpServers(env: NodeJS.ProcessEnv = process.env): NonNullable { + return { + freshell: { + command: 'node', + args: buildMcpServerCommandArgs(), + env: { + FRESHELL_URL: env.FRESHELL_URL || `http://localhost:${Number(env.PORT || 3001)}`, + FRESHELL_TOKEN: env.AUTH_TOKEN || '', + }, + }, + } +} + +export function createClaudeSdkOptions(input: ClaudeSdkOptionsInput): SdkOptions { + const options: SdkOptions = { + cwd: input.cwd || undefined, + resume: input.resumeSessionId, + model: input.model, + permissionMode: input.permissionMode as any, + effort: input.effort as any, + pathToClaudeCodeExecutable: process.env.CLAUDE_CMD || undefined, + includePartialMessages: input.includePartialMessages, + abortController: input.abortController, + env: createClaudeSdkCleanEnv(input.env), + mcpServers: createClaudeSdkMcpServers(input.env), + stderr: input.stderr, + canUseTool: input.canUseTool, + settingSources: ['user', 'project', 'local'], + } + + if (input.plugins !== undefined) { + options.plugins = sanitizeAgentChatPluginPaths(input.plugins) + .map((pluginPath) => ({ type: 'local' as const, path: pluginPath })) + } + + return options +} + export class SdkBridge extends EventEmitter { private sessions = new Map() private processes = new Map() - private cachedModels: Array<{ value: string; displayName: string; description: string }> | null = null constructor(private readonly agentHistorySource?: AgentHistorySource) { super() @@ -100,7 +157,7 @@ export class SdkBridge extends EventEmitter { resumeSessionId?: string model?: string permissionMode?: string - effort?: 'low' | 'medium' | 'high' | 'max' + effort?: string plugins?: string[] }): Promise { const sessionId = nanoid() @@ -126,34 +183,17 @@ export class SdkBridge extends EventEmitter { const abortController = new AbortController() const { iterable: inputIterable, handle: inputStream } = this.createInputStream() - // Strip env vars that interfere with child Claude Code subprocess behaviour: - // - CLAUDECODE: causes child to refuse startup ("nested session" error) - // - ANTHROPIC_API_KEY: inherited keys override subscription/OAuth auth, - // causing "Invalid API key" errors when the key is stale or invalid - const { CLAUDECODE: _, ANTHROPIC_API_KEY: __, ...cleanEnv } = process.env - const sdkQuery = query({ prompt: inputIterable as AsyncIterable, - options: { - cwd: options.cwd || undefined, - resume: options.resumeSessionId, + options: createClaudeSdkOptions({ + cwd: options.cwd, + resumeSessionId: options.resumeSessionId, model: options.model, - permissionMode: options.permissionMode as any, + permissionMode: options.permissionMode, effort: options.effort, - pathToClaudeCodeExecutable: process.env.CLAUDE_CMD || undefined, - includePartialMessages: true, + plugins: options.plugins, abortController, - env: cleanEnv, - mcpServers: { - freshell: { - command: 'node', - args: buildMcpServerCommandArgs(), - env: { - FRESHELL_URL: process.env.FRESHELL_URL || `http://localhost:${Number(process.env.PORT || 3001)}`, - FRESHELL_TOKEN: process.env.AUTH_TOKEN || '', - }, - }, - }, + includePartialMessages: true, stderr: (data: string) => { log.warn({ sessionId, data: data.trimEnd() }, 'SDK subprocess stderr') }, @@ -168,19 +208,7 @@ export class SdkBridge extends EventEmitter { } return this.handlePermissionRequest(sessionId, toolName, input as Record, ctx) }, - settingSources: ['user', 'project', 'local'], - // Explicit plugins remain supported for non-Freshell Claude SDK bundles. - // Freshell orchestration itself is provided by the MCP server above. - ...((() => { - if (options.plugins !== undefined) { - return { - plugins: sanitizeAgentChatPluginPaths(options.plugins) - .map(p => ({ type: 'local' as const, path: p })), - } - } - return {} - })()), - }, + }), }) this.processes.set(sessionId, { @@ -349,9 +377,6 @@ export class SdkBridge extends EventEmitter { cwd: state.cwd, tools: state.tools, }) - - // Fetch available models and broadcast to client - this.fetchAndBroadcastModels(sessionId) } else if (msg.subtype === 'status') { const statusMsg = msg as SDKStatusMessage if (statusMsg.status === 'compacting') { @@ -750,41 +775,6 @@ export class SdkBridge extends EventEmitter { return true } - private fetchAndBroadcastModels(sessionId: string): void { - // Use cache if available - if (this.cachedModels) { - this.broadcastToSession(sessionId, { - type: 'sdk.models', - sessionId, - models: this.cachedModels, - }) - return - } - - const sp = this.processes.get(sessionId) - if (!sp) return - - sp.query.supportedModels().then((models) => { - const mapped = models.map((m: any) => { - const value = m.value ?? m.id ?? String(m) - const rawName = m.displayName ?? m.display_name ?? value - return { - value, - displayName: formatModelDisplayName(rawName), - description: m.description ?? '', - } - }) - this.cachedModels = mapped - this.broadcastToSession(sessionId, { - type: 'sdk.models', - sessionId, - models: mapped, - }) - }).catch((err) => { - log.warn({ sessionId, err }, 'Failed to fetch supported models') - }) - } - close(): void { for (const [sessionId] of this.processes) { this.killSession(sessionId) diff --git a/server/settings-router.ts b/server/settings-router.ts index c584aa76..fc12db4f 100644 --- a/server/settings-router.ts +++ b/server/settings-router.ts @@ -69,6 +69,24 @@ export const normalizeSettingsPatch = (patch: Record) => { } } + if (patch.agentChat?.providers && typeof patch.agentChat.providers === 'object') { + for (const providerPatch of Object.values(patch.agentChat.providers)) { + if (!providerPatch || typeof providerPatch !== 'object' || Array.isArray(providerPatch)) { + continue + } + const providerPatchRecord = providerPatch as Record + if (Object.prototype.hasOwnProperty.call(providerPatchRecord, 'modelSelection') && providerPatchRecord.modelSelection === null) { + providerPatchRecord.modelSelection = undefined + } + if (Object.prototype.hasOwnProperty.call(providerPatchRecord, 'effort')) { + const raw = providerPatchRecord.effort + if (raw === null || (typeof raw === 'string' && raw.trim() === '')) { + providerPatchRecord.effort = undefined + } + } + } + } + return patch } diff --git a/shared/agent-chat-capabilities.ts b/shared/agent-chat-capabilities.ts new file mode 100644 index 00000000..3f64ced4 --- /dev/null +++ b/shared/agent-chat-capabilities.ts @@ -0,0 +1,60 @@ +import { z } from 'zod' + +export const AGENT_CHAT_CAPABILITY_CACHE_TTL_MS = 5 * 60 * 1000 + +export const AgentChatOpaqueStringSchema = z.string().trim().min(1) + +export const AgentChatTrackedModelSelectionSchema = z.object({ + kind: z.literal('tracked'), + modelId: AgentChatOpaqueStringSchema, +}).strict() + +export const AgentChatExactModelSelectionSchema = z.object({ + kind: z.literal('exact'), + modelId: AgentChatOpaqueStringSchema, +}).strict() + +export const AgentChatModelSelectionSchema = z.discriminatedUnion('kind', [ + AgentChatTrackedModelSelectionSchema, + AgentChatExactModelSelectionSchema, +]) + +export const AgentChatModelCapabilitySchema = z.object({ + id: AgentChatOpaqueStringSchema, + displayName: AgentChatOpaqueStringSchema, + description: z.string().optional(), + supportsEffort: z.boolean(), + supportedEffortLevels: z.array(AgentChatOpaqueStringSchema), + supportsAdaptiveThinking: z.boolean(), +}).strict() + +export const AgentChatCapabilitiesSchema = z.object({ + provider: AgentChatOpaqueStringSchema, + fetchedAt: z.number().int().nonnegative(), + models: z.array(AgentChatModelCapabilitySchema), +}).strict() + +export const AgentChatCapabilityErrorSchema = z.object({ + code: AgentChatOpaqueStringSchema, + message: AgentChatOpaqueStringSchema, + retryable: z.boolean().optional(), +}).strict() + +export const AgentChatCapabilitiesResponseSchema = z.discriminatedUnion('ok', [ + z.object({ + ok: z.literal(true), + capabilities: AgentChatCapabilitiesSchema, + }).strict(), + z.object({ + ok: z.literal(false), + error: AgentChatCapabilityErrorSchema, + }).strict(), +]) + +export type AgentChatModelSelection = z.infer +export type AgentChatTrackedModelSelection = z.infer +export type AgentChatExactModelSelection = z.infer +export type AgentChatModelCapability = z.infer +export type AgentChatCapabilities = z.infer +export type AgentChatCapabilityError = z.infer +export type AgentChatCapabilitiesResponse = z.infer diff --git a/shared/settings.ts b/shared/settings.ts index 450aaa8c..71ce9532 100644 --- a/shared/settings.ts +++ b/shared/settings.ts @@ -1,5 +1,10 @@ import { z } from 'zod' +import { + AgentChatModelSelectionSchema, + AgentChatOpaqueStringSchema, + type AgentChatModelSelection, +} from './agent-chat-capabilities.js' import { sanitizeAgentChatPluginPaths } from './agent-chat-plugins.js' import { DEFAULT_ENABLED_CLI_PROVIDERS } from './coding-cli-defaults.js' import { normalizeTrimmedStringList } from './string-list.js' @@ -27,7 +32,6 @@ export const CODEX_SANDBOX_VALUES = ['read-only', 'workspace-write', 'danger-ful export const CLAUDE_PERMISSION_MODE_VALUES = ['default', 'plan', 'acceptEdits', 'bypassPermissions'] as const const EXTERNAL_EDITOR_VALUES = ['auto', 'cursor', 'code', 'custom'] as const const NETWORK_HOST_VALUES = ['127.0.0.1', '0.0.0.0'] as const -const AGENT_CHAT_EFFORT_VALUES = ['low', 'medium', 'high', 'max'] as const const UI_SCALE_MIN = 0.75 const UI_SCALE_MAX = 1.5 const TERMINAL_FONT_SIZE_MIN = 12 @@ -81,7 +85,7 @@ export type CodexSandboxMode = (typeof CODEX_SANDBOX_VALUES)[number] export type ClaudePermissionMode = (typeof CLAUDE_PERMISSION_MODE_VALUES)[number] export type ExternalEditor = (typeof EXTERNAL_EDITOR_VALUES)[number] export type NetworkHost = (typeof NETWORK_HOST_VALUES)[number] -export type AgentChatEffort = (typeof AGENT_CHAT_EFFORT_VALUES)[number] +export type AgentChatEffort = string export type DeepPartial = T extends readonly (infer U)[] ? U[] @@ -105,9 +109,9 @@ export type CodingCliSettings = { } export type AgentChatProviderDefaults = { - defaultModel?: string + modelSelection?: AgentChatModelSelection defaultPermissionMode?: string - defaultEffort?: AgentChatEffort + effort?: AgentChatEffort } export type ServerSettings = { @@ -231,7 +235,6 @@ const AttentionDismissSchema = z.enum(ATTENTION_DISMISS_VALUES) const SessionOpenModeSchema = z.enum(SESSION_OPEN_MODE_VALUES) const ExternalEditorSchema = z.enum(EXTERNAL_EDITOR_VALUES) const NetworkHostSchema = z.enum(NETWORK_HOST_VALUES) -const AgentChatEffortSchema = z.enum(AGENT_CHAT_EFFORT_VALUES) function hasOwn(value: T | undefined | null, key: PropertyKey): boolean { return !!value && Object.prototype.hasOwnProperty.call(value, key) @@ -542,12 +545,22 @@ function createCodingCliProviderConfigPatchSchema() { .strict() } +function createAgentChatProviderDefaultsSchema() { + return z + .object({ + modelSelection: AgentChatModelSelectionSchema.optional(), + defaultPermissionMode: z.string().optional(), + effort: AgentChatOpaqueStringSchema.optional(), + }) + .strict() +} + function createAgentChatProviderDefaultsPatchSchema() { return z .object({ - defaultModel: z.string().optional(), + modelSelection: AgentChatModelSelectionSchema.nullable().optional(), defaultPermissionMode: z.string().optional(), - defaultEffort: AgentChatEffortSchema.optional(), + effort: z.union([AgentChatOpaqueStringSchema, z.literal('')]).nullable().optional(), }) .strict() } @@ -584,7 +597,7 @@ export function buildServerSettingsSchema(validCliProviders?: readonly string[]) agentChat: z.object({ initialSetupDone: z.boolean().optional(), defaultPlugins: z.array(z.string()), - providers: z.record(z.string(), createAgentChatProviderDefaultsPatchSchema()), + providers: z.record(z.string(), createAgentChatProviderDefaultsSchema()), }).strict(), extensions: z.object({ disabled: z.array(z.string()), @@ -897,13 +910,29 @@ function sanitizeServerSettingsPatch(patch: ServerSettingsPatch): ServerSettings if (isRecord(candidate.agentChat.providers)) { const providers: NonNullable['providers'] = {} for (const [providerName, providerPatch] of Object.entries(candidate.agentChat.providers)) { + const normalizedProviderPatchInput = normalizeLegacyAgentChatProviderDefaultsInput(providerPatch) const parsed = agentChatProviderDefaultsPatchSchema.safeParse( - isRecord(providerPatch) - ? pickKeys(providerPatch, ['defaultModel', 'defaultPermissionMode', 'defaultEffort']) - : providerPatch, + normalizedProviderPatchInput, ) - if (parsed.success && Object.keys(parsed.data).length > 0) { - providers[providerName] = parsed.data + if ( + parsed.success + && isRecord(normalizedProviderPatchInput) + && Object.keys(normalizedProviderPatchInput).length > 0 + ) { + const normalizedProviderPatch: AgentChatProviderDefaults = {} + if (hasOwn(normalizedProviderPatchInput, 'modelSelection')) { + normalizedProviderPatch.modelSelection = parsed.data.modelSelection ?? undefined + } + if (hasOwn(normalizedProviderPatchInput, 'defaultPermissionMode')) { + normalizedProviderPatch.defaultPermissionMode = parsed.data.defaultPermissionMode + } + if (hasOwn(normalizedProviderPatchInput, 'effort')) { + const parsedEffort = parsed.data.effort + normalizedProviderPatch.effort = typeof parsedEffort === 'string' && parsedEffort.trim().length === 0 + ? undefined + : parsedEffort ?? undefined + } + providers[providerName] = normalizedProviderPatch } } if (Object.keys(providers).length > 0) { @@ -946,10 +975,46 @@ function sanitizeServerSettingsPatch(patch: ServerSettingsPatch): ServerSettings return sanitized } +function normalizeLegacyAgentChatProviderDefaultsInput( + providerPatch: unknown, +): Record | unknown { + if (!isRecord(providerPatch)) { + return providerPatch + } + + const normalized = pickOwnKeysPreservingUndefined( + providerPatch, + ['modelSelection', 'defaultPermissionMode', 'effort'], + ) + + if ( + !hasOwn(normalized, 'modelSelection') + && typeof providerPatch.defaultModel === 'string' + && providerPatch.defaultModel.trim().length > 0 + ) { + normalized.modelSelection = { + kind: 'exact', + modelId: providerPatch.defaultModel, + } + } + + if ( + !hasOwn(normalized, 'effort') + && typeof providerPatch.defaultEffort === 'string' + && providerPatch.defaultEffort.trim().length > 0 + ) { + normalized.effort = providerPatch.defaultEffort + } + + return normalized +} + export function mergeServerSettings(base: ServerSettings, patch: ServerSettingsPatch): ServerSettings { const normalizedPatch = sanitizeServerSettingsPatch(patch) const codingCliPatch = normalizedPatch.codingCli const agentChatPatch = normalizedPatch.agentChat + const normalizedAgentChatPatch = agentChatPatch as Partial | undefined + const normalizedAgentChatProvidersPatch = agentChatPatch?.providers as Partial> | undefined return { ...base, @@ -981,11 +1046,11 @@ export function mergeServerSettings(base: ServerSettings, patch: ServerSettingsP }, editor: mergeDefined(base.editor, normalizedPatch.editor), agentChat: { - ...mergeDefined(base.agentChat, agentChatPatch), - defaultPlugins: hasOwn(agentChatPatch, 'defaultPlugins') - ? sanitizeAgentChatPluginPaths(agentChatPatch?.defaultPlugins) + ...mergeDefined(base.agentChat, normalizedAgentChatPatch), + defaultPlugins: hasOwn(normalizedAgentChatPatch, 'defaultPlugins') + ? sanitizeAgentChatPluginPaths(normalizedAgentChatPatch?.defaultPlugins) : base.agentChat.defaultPlugins, - providers: mergeRecordOfObjects(base.agentChat.providers, agentChatPatch?.providers), + providers: mergeRecordOfObjects(base.agentChat.providers, normalizedAgentChatProvidersPatch), }, extensions: { disabled: hasOwn(normalizedPatch.extensions, 'disabled') diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index e4f39f30..4dbf1efb 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -323,7 +323,7 @@ export const SdkCreateSchema = z.object({ resumeSessionId: z.string().optional(), model: z.string().optional(), permissionMode: z.string().optional(), - effort: z.enum(['low', 'medium', 'high', 'max']).optional(), + effort: z.string().trim().min(1).optional(), plugins: z.array(z.string()).optional(), }) @@ -687,7 +687,6 @@ export type SdkServerMessage = | { type: 'sdk.error'; sessionId: string; message: string; code?: string } | { type: 'sdk.exit'; sessionId: string; exitCode?: number } | { type: 'sdk.killed'; sessionId: string; success: boolean } - | { type: 'sdk.models'; sessionId: string; models: Array<{ value: string; displayName: string; description: string }> } | { type: 'sdk.question.request'; sessionId: string; requestId: string; questions: Array<{ question: string; header: string; options: Array<{ label: string; description: string }>; multiSelect: boolean }> } // -- Extensions -- diff --git a/src/components/TabsView.tsx b/src/components/TabsView.tsx index a7750316..8dbb535a 100644 --- a/src/components/TabsView.tsx +++ b/src/components/TabsView.tsx @@ -26,7 +26,12 @@ import { copyText } from '@/lib/clipboard' import { cn } from '@/lib/utils' import { ContextMenu } from '@/components/context-menu/ContextMenu' import type { MenuItem } from '@/components/context-menu/context-menu-types' -import type { PaneContentInput, SessionLocator } from '@/store/paneTypes' +import { + normalizeAgentChatEffortOverride, + normalizeAgentChatModelSelection, + type PaneContentInput, + type SessionLocator, +} from '@/store/paneTypes' import type { CodingCliProviderName, TabMode } from '@/store/types' import type { AgentChatProviderName } from '@/lib/agent-chat-types' @@ -135,9 +140,9 @@ function sanitizePaneSnapshot( resumeSessionId: sameServer ? resumeSessionId : undefined, sessionRef, initialCwd: payload.initialCwd as string | undefined, - model: payload.model as string | undefined, + modelSelection: normalizeAgentChatModelSelection(payload.modelSelection, payload.model), permissionMode: payload.permissionMode as string | undefined, - effort: payload.effort as 'low' | 'medium' | 'high' | 'max' | undefined, + effort: normalizeAgentChatEffortOverride(payload.effort), plugins: payload.plugins as string[] | undefined, } } diff --git a/src/components/agent-chat/AgentChatSettings.tsx b/src/components/agent-chat/AgentChatSettings.tsx index f5508027..8155ea03 100644 --- a/src/components/agent-chat/AgentChatSettings.tsx +++ b/src/components/agent-chat/AgentChatSettings.tsx @@ -1,14 +1,17 @@ -import { useCallback, useEffect, useId, useRef, useState } from 'react' +import { useCallback, useEffect, useId, useMemo, useRef, useState } from 'react' import { Settings } from 'lucide-react' -import { cn } from '@/lib/utils' + import { Switch } from '@/components/ui/switch' import { useMobile } from '@/hooks/useMobile' import { useKeyboardInset } from '@/hooks/useKeyboardInset' -import type { AgentChatPaneContent } from '@/store/paneTypes' +import type { AgentChatSettingsModelOption } from '@/lib/agent-chat-capabilities' import type { AgentChatProviderConfig } from '@/lib/agent-chat-types' -import { formatModelDisplayName } from '../../../shared/format-model-name' +import { cn } from '@/lib/utils' +import type { AgentChatPaneContent } from '@/store/paneTypes' +import type { AgentChatProviderCapabilitiesState } from '@/store/agentChatTypes' -type SettingsFields = Pick & { +type SettingsFields = Pick & { + model?: string showThinking?: boolean showTools?: boolean showTimecodes?: boolean @@ -23,32 +26,22 @@ interface AgentChatSettingsProps { showTimecodes: boolean sessionStarted: boolean defaultOpen?: boolean - modelOptions?: Array<{ value: string; displayName: string }> + modelOptions?: AgentChatSettingsModelOption[] + effortOptions?: string[] + capabilitiesStatus?: AgentChatProviderCapabilitiesState['status'] + capabilityError?: AgentChatProviderCapabilitiesState['error'] settingsVisibility?: AgentChatProviderConfig['settingsVisibility'] onChange: (changes: Partial) => void onDismiss?: () => void + onRetryCapabilities?: () => void + onOpenChange?: (open: boolean) => void } -const MODEL_OPTIONS = [ - { value: 'claude-opus-4-6', label: 'Opus 4.6' }, - { value: 'claude-sonnet-4-6', label: 'Sonnet 4.6' }, - { value: 'claude-haiku-4-5-20251001', label: 'Haiku 4.5' }, - { value: 'claude-sonnet-4-5-20250929', label: 'Sonnet 4.5' }, - { value: 'claude-opus-4-5', label: 'Opus 4.5' }, -] - const PERMISSION_OPTIONS = [ { value: 'bypassPermissions', label: 'Skip permissions' }, { value: 'default', label: 'Default (ask)' }, ] -const EFFORT_OPTIONS = [ - { value: 'low', label: 'Low' }, - { value: 'medium', label: 'Medium' }, - { value: 'high', label: 'High' }, - { value: 'max', label: 'Max' }, -] - export default function AgentChatSettings({ model, permissionMode, @@ -59,9 +52,14 @@ export default function AgentChatSettings({ sessionStarted, defaultOpen = false, modelOptions, + effortOptions, + capabilitiesStatus = 'idle', + capabilityError, settingsVisibility, onChange, onDismiss, + onRetryCapabilities, + onOpenChange, }: AgentChatSettingsProps) { const instanceId = useId() const isMobile = useMobile() @@ -70,12 +68,14 @@ export default function AgentChatSettings({ const popoverRef = useRef(null) const buttonRef = useRef(null) - // Sync open state when defaultOpen changes (e.g. after settings load, - // or when initialSetupDone arrives after the 2s timeout fallback) useEffect(() => { setOpen(defaultOpen) }, [defaultOpen]) + useEffect(() => { + onOpenChange?.(open) + }, [onOpenChange, open]) + const handleClose = useCallback(() => { setOpen(false) onDismiss?.() @@ -84,76 +84,49 @@ export default function AgentChatSettings({ const handleToggle = useCallback(() => { if (open) { handleClose() - } else { - setOpen(true) + return } - }, [open, handleClose]) + setOpen(true) + }, [handleClose, open]) - // Close on click outside useEffect(() => { if (!open) return - const handleMouseDown = (e: MouseEvent) => { - const target = e.target as Node + + const handleMouseDown = (event: MouseEvent) => { + const target = event.target as Node if ( - popoverRef.current && !popoverRef.current.contains(target) && - buttonRef.current && !buttonRef.current.contains(target) + popoverRef.current && !popoverRef.current.contains(target) + && buttonRef.current && !buttonRef.current.contains(target) ) { handleClose() } } + document.addEventListener('mousedown', handleMouseDown) return () => document.removeEventListener('mousedown', handleMouseDown) - }, [open, handleClose]) + }, [handleClose, open]) - // Close on Escape key — uses document listener so it works regardless of focus location useEffect(() => { if (!open) return - const handleEscape = (e: KeyboardEvent) => { - if (e.key === 'Escape') { - e.stopPropagation() - handleClose() - } - } - document.addEventListener('keydown', handleEscape) - return () => document.removeEventListener('keydown', handleEscape) - }, [open, handleClose]) - // Start with the hardcoded list, then append any dynamic models not already present. - // When the SDK provides a model whose normalized label matches a hardcoded entry - // (e.g. a newer dated ID like claude-sonnet-4-5-20251101 → "Sonnet 4.5"), replace - // the hardcoded value so users get the current model ID without duplicates. - const resolvedModelOptions = (() => { - if (!modelOptions) return MODEL_OPTIONS - - // Map normalized label → SDK value for models with full claude-* IDs. - // When multiple dated IDs map to the same label, keep the latest one. - const sdkByLabel = new Map() - for (const m of modelOptions) { - if (m.value.startsWith('claude-')) { - const label = formatModelDisplayName(m.displayName) - const existing = sdkByLabel.get(label) - if (!existing || m.value > existing) { - sdkByLabel.set(label, m.value) - } - } + const handleEscape = (event: KeyboardEvent) => { + if (event.key !== 'Escape') return + event.stopPropagation() + handleClose() } - // Replace hardcoded values with SDK values when labels match - const usedLabels = new Set() - const base = MODEL_OPTIONS.map((m) => { - usedLabels.add(m.label) - const sdkValue = sdkByLabel.get(m.label) - return sdkValue ? { value: sdkValue, label: m.label } : m - }) - - // Append genuinely new SDK models not matching any hardcoded label - const extras = modelOptions - .filter((m) => m.value.startsWith('claude-')) - .map((m) => ({ value: m.value, label: formatModelDisplayName(m.displayName) })) - .filter((m) => !usedLabels.has(m.label)) + document.addEventListener('keydown', handleEscape) + return () => document.removeEventListener('keydown', handleEscape) + }, [handleClose, open]) - return [...base, ...extras] - })() + const resolvedModelOptions = modelOptions ?? [] + const selectedModelOption = useMemo( + () => resolvedModelOptions.find((option) => option.value === model), + [model, resolvedModelOptions], + ) + const resolvedEffortOptions = effortOptions ?? [] + const showCapabilityControls = capabilitiesStatus !== 'failed' + const showEffortControl = resolvedEffortOptions.length > 0 return (
@@ -195,95 +168,134 @@ export default function AgentChatSettings({ aria-label="Agent chat settings" >
- {/* Model */} - {settingsVisibility?.model !== false && ( -
- - -
- )} + {capabilitiesStatus === 'loading' && ( +
+ Loading available models... +
+ )} - {/* Permission mode */} - {settingsVisibility?.permissionMode !== false && ( -
- - -
- )} + {capabilitiesStatus === 'failed' && capabilityError && ( +
+

{capabilityError.message}

+ {(capabilityError.retryable ?? true) && onRetryCapabilities && ( + + )} +
+ )} - {/* Effort — locked after session starts */} - {settingsVisibility?.effort !== false && ( -
- - -
- )} + {showCapabilityControls && settingsVisibility?.model !== false && ( +
+ + + {selectedModelOption?.description && ( +

+ {selectedModelOption.description} +

+ )} +
+ )} -
+ {settingsVisibility?.permissionMode !== false && ( +
+ + +
+ )} - {/* Display toggles using existing Switch component */} - {settingsVisibility?.thinking !== false && ( - onChange({ showThinking: v })} - /> - )} - {settingsVisibility?.tools !== false && ( - onChange({ showTools: v })} - /> - )} - {settingsVisibility?.timecodes !== false && ( - onChange({ showTimecodes: v })} - /> - )} - {isMobile && ( - - )} -
+ {showCapabilityControls && settingsVisibility?.effort !== false && showEffortControl && ( +
+ + +
+ )} + + {showCapabilityControls && settingsVisibility?.effort !== false && !showEffortControl && ( +

+ This model uses its own default effort behavior. +

+ )} + +
+ + {settingsVisibility?.thinking !== false && ( + onChange({ showThinking: checked })} + /> + )} + {settingsVisibility?.tools !== false && ( + onChange({ showTools: checked })} + /> + )} + {settingsVisibility?.timecodes !== false && ( + onChange({ showTimecodes: checked })} + /> + )} + + {isMobile && ( + + )} +
)} @@ -291,7 +303,15 @@ export default function AgentChatSettings({ ) } -function ToggleRow({ label, checked, onChange }: { label: string; checked: boolean; onChange: (v: boolean) => void }) { +function ToggleRow({ + label, + checked, + onChange, +}: { + label: string + checked: boolean + onChange: (value: boolean) => void +}) { return (
{label} diff --git a/src/components/agent-chat/AgentChatView.tsx b/src/components/agent-chat/AgentChatView.tsx index 68c0be6e..5112fea1 100644 --- a/src/components/agent-chat/AgentChatView.tsx +++ b/src/components/agent-chat/AgentChatView.tsx @@ -12,7 +12,12 @@ import { removePermission, removeQuestion, } from '@/store/agentChatSlice' -import { loadAgentTimelineWindow, loadAgentTurnBody } from '@/store/agentChatThunks' +import { + fetchAgentChatCapabilities, + loadAgentTimelineWindow, + loadAgentTurnBody, + refreshAgentChatCapabilities, +} from '@/store/agentChatThunks' import { getWsClient } from '@/lib/ws-client' import { cn } from '@/lib/utils' import { ChevronDown } from 'lucide-react' @@ -25,6 +30,16 @@ import ThinkingIndicator from './ThinkingIndicator' import { useStreamDebounce } from './useStreamDebounce' import CollapsedTurn from './CollapsedTurn' import type { ChatMessage } from '@/store/agentChatTypes' +import { + getAgentChatSettingsModelOptions, + getAgentChatSettingsModelValue, + getAgentChatSupportedEffortLevels, + isAgentChatCapabilitiesFresh, + isAgentChatEffortSupported, + parseAgentChatSettingsModelValue, + requiresAgentChatCapabilityValidation, + resolveAgentChatModelSelection, +} from '@/lib/agent-chat-capabilities' import { setSessionMetadata } from '@/lib/api' import { getAgentChatProviderConfig } from '@/lib/agent-chat-utils' import { isValidClaudeSessionId } from '@/lib/claude-session-id' @@ -54,6 +69,23 @@ function isStatusRegression(current: string, next: string): boolean { return !EARLY_STATES.has(current) && EARLY_STATES.has(next) } +function modelSelectionsMatch( + left: AgentChatPaneContent['modelSelection'], + right: AgentChatPaneContent['modelSelection'], +): boolean { + if (!left && !right) return true + if (!left || !right) return false + return left.kind === right.kind && left.modelId === right.modelId +} + +function paneMatchesCurrentProviderDefaults( + pane: Pick, + providerDefaults?: Pick, +): boolean { + return modelSelectionsMatch(pane.modelSelection, providerDefaults?.modelSelection) + && pane.effort === providerDefaults?.effort +} + interface AgentChatViewProps { tabId: string paneId: string @@ -63,14 +95,14 @@ interface AgentChatViewProps { export default function AgentChatView({ tabId, paneId, paneContent, hidden }: AgentChatViewProps) { const dispatch = useAppDispatch() - const ws = getWsClient() + const ws = useMemo(() => getWsClient(), []) const isMobile = useMobile() const keyboardInsetPx = useKeyboardInset() const providerConfig = getAgentChatProviderConfig(paneContent.provider) - const defaultModel = providerConfig?.defaultModel ?? 'claude-opus-4-6' + const providerDefaultModelId = providerConfig?.providerDefaultModelId ?? 'opus' const defaultPermissionMode = providerConfig?.defaultPermissionMode ?? 'bypassPermissions' - const defaultEffort = providerConfig?.defaultEffort ?? 'high' const localSettings = useAppSelector((state) => state.settings.settings) + const providerSettings = localSettings.agentChat?.providers?.[paneContent.provider] const defaultShowThinking = localSettings.agentChat.showThinking const defaultShowTools = localSettings.agentChat.showTools const defaultShowTimecodes = localSettings.agentChat.showTimecodes @@ -105,7 +137,40 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag (s as { tabs?: { tabs?: Tab[] } }).tabs?.tabs?.find((entry) => entry.id === tabId) )) const tabTitleSetByUser = currentTab?.titleSetByUser ?? false - const availableModels = useAppSelector((s) => s.agentChat.availableModels) + const providerCapabilitiesState = useAppSelector( + (s) => s.agentChat.capabilitiesByProvider?.[paneContent.provider], + ) + const providerCapabilities = providerCapabilitiesState?.capabilities + const providerCapabilitiesRef = useRef(providerCapabilities) + providerCapabilitiesRef.current = providerCapabilities + const resolvedModelSelection = useMemo( + () => resolveAgentChatModelSelection({ + providerDefaultModelId, + capabilities: providerCapabilities, + modelSelection: paneContent.modelSelection, + }), + [paneContent.modelSelection, providerCapabilities, providerDefaultModelId], + ) + const settingsModelOptions = useMemo( + () => getAgentChatSettingsModelOptions({ + providerDefaultModelId, + capabilities: providerCapabilities, + modelSelection: paneContent.modelSelection, + }), + [paneContent.modelSelection, providerCapabilities, providerDefaultModelId], + ) + const settingsModelValue = getAgentChatSettingsModelValue( + paneContent.modelSelection, + providerCapabilities, + ) + const effortOptions = useMemo( + () => getAgentChatSupportedEffortLevels({ + providerDefaultModelId, + capabilities: providerCapabilities, + modelSelection: paneContent.modelSelection, + }), + [paneContent.modelSelection, providerCapabilities, providerDefaultModelId], + ) const settingsLoaded = useAppSelector((s) => s.settings.loaded) const initialSetupDone = useAppSelector((s) => s.settings.settings.agentChat?.initialSetupDone ?? false) const activePaneId = useAppSelector((s) => s.panes.activePane[tabId]) @@ -142,6 +207,24 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag const suppressNetworkEffects = typeof window !== 'undefined' && window.__FRESHELL_TEST_HARNESS__?.isAgentChatNetworkEffectsSuppressed?.(paneId) === true + const clearPersistedProviderEffortIfPaneMatchesDefaults = useCallback(( + pane: Pick, + ) => { + if (!paneMatchesCurrentProviderDefaults(pane, providerSettings)) { + return + } + + void dispatch(saveServerSettingsPatch({ + agentChat: { + providers: { + [paneContent.provider]: { + effort: undefined, + }, + }, + }, + })) + }, [dispatch, paneContent.provider, providerSettings]) + // Track whether we're waiting for a session restore (persisted sessionId, history not yet loaded). // Fresh creates set historyLoaded=true immediately; reloads wait for the initial // HTTP timeline window (even if it is empty). @@ -387,29 +470,155 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag if (paneContent.sessionId || createSentRef.current) return if (paneContent.status !== 'creating') return + const requestId = paneContent.createRequestId createSentRef.current = true - dispatch(registerPendingCreate({ - requestId: paneContent.createRequestId, - expectsHistoryHydration: Boolean(paneContent.resumeSessionId), - })) - ws.send({ - type: 'sdk.create', - requestId: paneContent.createRequestId, - model: paneContent.model ?? defaultModel, - permissionMode: paneContent.permissionMode ?? defaultPermissionMode, - effort: paneContent.effort ?? defaultEffort, - ...(paneContent.initialCwd ? { cwd: paneContent.initialCwd } : {}), - ...(paneContent.resumeSessionId ? { resumeSessionId: paneContent.resumeSessionId } : {}), - ...(paneContent.plugins ? { plugins: paneContent.plugins } : {}), - }) + let cancelled = false - // Update status to 'starting' - dispatch(updatePaneContent({ - tabId, - paneId, - content: { ...paneContent, status: 'starting' }, - })) - }, [paneContent.createRequestId, paneContent.sessionId, paneContent.status, tabId, paneId, dispatch, suppressNetworkEffects, ws]) + const failCreate = (error: AgentChatPaneContent['createError']) => { + if (cancelled) return + createSentRef.current = false + attachSentRef.current = false + dispatch(updatePaneContent({ + tabId, + paneId, + content: { + ...paneContentRef.current, + status: 'create-failed', + createError: error, + }, + })) + } + + void (async () => { + let capabilities = providerCapabilitiesRef.current + + if (requiresAgentChatCapabilityValidation({ + modelSelection: paneContent.modelSelection, + effort: paneContent.effort, + }) && !isAgentChatCapabilitiesFresh(capabilities)) { + const response = await dispatch(fetchAgentChatCapabilities(paneContent.provider)) + if (cancelled) return + if (!response.ok) { + failCreate(response.error) + return + } + capabilities = response.capabilities + } + + if (cancelled) return + + const currentPane = paneContentRef.current + if ( + currentPane.createRequestId !== requestId + || currentPane.sessionId + || currentPane.status !== 'creating' + ) { + return + } + + // Create-time resume accepts either the canonical durable Claude id or a + // live/named resume token. We persist only canonical ids for reload/attach + // flows, but named resumes still need to launch a restoring session that can + // later upgrade in place once the canonical timeline id is known. + const createResumeSessionId = ( + currentPane.sessionRef?.provider === 'claude' + && isValidClaudeSessionId(currentPane.sessionRef.sessionId) + ) + ? currentPane.sessionRef.sessionId + : (typeof currentPane.resumeSessionId === 'string' && currentPane.resumeSessionId.trim().length > 0 + ? currentPane.resumeSessionId + : undefined) + const resolvedSelection = resolveAgentChatModelSelection({ + providerDefaultModelId, + capabilities, + modelSelection: currentPane.modelSelection, + }) + + if (!resolvedSelection.resolvedModelId) { + const unavailableModelId = + resolvedSelection.unavailableExactSelection?.modelId + ?? currentPane.modelSelection?.modelId + ?? 'the selected model' + failCreate({ + code: 'MODEL_UNAVAILABLE', + message: `Selected model ${unavailableModelId} is no longer available.`, + retryable: false, + }) + return + } + + let resolvedEffort = currentPane.effort + let shouldClearPersistedProviderEffort = false + if (resolvedEffort) { + if (!resolvedSelection.capability) { + failCreate({ + code: 'CAPABILITY_VALIDATION_REQUIRED', + message: 'Could not validate the selected effort for this model.', + retryable: true, + }) + return + } + + if (!isAgentChatEffortSupported(resolvedSelection.capability, resolvedEffort)) { + resolvedEffort = undefined + shouldClearPersistedProviderEffort = paneMatchesCurrentProviderDefaults( + currentPane, + providerSettings, + ) + } + } + + dispatch(registerPendingCreate({ + requestId, + expectsHistoryHydration: Boolean(createResumeSessionId), + })) + ws.send({ + type: 'sdk.create', + requestId, + model: resolvedSelection.resolvedModelId, + permissionMode: currentPane.permissionMode ?? defaultPermissionMode, + ...(resolvedEffort ? { effort: resolvedEffort } : {}), + ...(currentPane.initialCwd ? { cwd: currentPane.initialCwd } : {}), + ...(createResumeSessionId ? { resumeSessionId: createResumeSessionId } : {}), + ...(currentPane.plugins ? { plugins: currentPane.plugins } : {}), + }) + + dispatch(updatePaneContent({ + tabId, + paneId, + content: { + ...currentPane, + status: 'starting', + ...(resolvedEffort ? {} : { effort: undefined }), + createError: undefined, + }, + })) + + if (shouldClearPersistedProviderEffort) { + clearPersistedProviderEffortIfPaneMatchesDefaults(currentPane) + } + + })() + + return () => { + cancelled = true + } + }, [ + defaultPermissionMode, + dispatch, + paneContent.createRequestId, + paneContent.effort, + paneContent.modelSelection, + paneContent.provider, + paneContent.sessionId, + paneContent.status, + paneId, + providerDefaultModelId, + providerSettings, + suppressNetworkEffects, + tabId, + ws, + ]) // Attach to existing session on mount (e.g. after page refresh with persisted pane). // Skip when session is already fully hydrated (e.g. split-induced remount) — the WS @@ -574,10 +783,26 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag const handleSettingsChange = useCallback((changes: Record) => { const paneChanges: Partial = {} const localChanges: Record = {} + const hasModelChange = Object.prototype.hasOwnProperty.call(changes, 'model') + const hasPermissionModeChange = Object.prototype.hasOwnProperty.call(changes, 'permissionMode') + const hasEffortChange = Object.prototype.hasOwnProperty.call(changes, 'effort') + const nextModelValue = typeof changes.model === 'string' && changes.model.trim().length > 0 + ? changes.model + : undefined + const nextModelSelection = nextModelValue + ? parseAgentChatSettingsModelValue(nextModelValue) + : undefined + const nextEffort = typeof changes.effort === 'string' && changes.effort.trim().length > 0 + ? changes.effort + : undefined for (const [key, value] of Object.entries(changes)) { if (key === 'showThinking' || key === 'showTools' || key === 'showTimecodes') { localChanges[key] = value + } else if (key === 'model') { + paneChanges.modelSelection = nextModelSelection + } else if (key === 'effort') { + paneChanges.effort = nextEffort } else { (paneChanges as Record)[key] = value } @@ -598,26 +823,111 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag const pc = paneContentRef.current // Mid-session model change - if (changes.model && pc.sessionId && pc.status !== 'creating') { - ws.send({ type: 'sdk.set-model', sessionId: pc.sessionId, model: changes.model as string }) + if (hasModelChange && nextModelValue && pc.sessionId && pc.status !== 'creating') { + const resolvedSelection = resolveAgentChatModelSelection({ + providerDefaultModelId, + capabilities: providerCapabilitiesRef.current, + modelSelection: nextModelSelection, + }) + if (resolvedSelection.resolvedModelId) { + ws.send({ type: 'sdk.set-model', sessionId: pc.sessionId, model: resolvedSelection.resolvedModelId }) + } } // Mid-session permission mode change - if (changes.permissionMode && pc.sessionId && pc.status !== 'creating') { + if (hasPermissionModeChange && changes.permissionMode && pc.sessionId && pc.status !== 'creating') { ws.send({ type: 'sdk.set-permission-mode', sessionId: pc.sessionId, permissionMode: changes.permissionMode as string }) } // Persist as defaults - const defaultsPatch: Record = {} - if (changes.model) defaultsPatch.defaultModel = changes.model as string - if (changes.permissionMode) defaultsPatch.defaultPermissionMode = changes.permissionMode as string - if (changes.effort) defaultsPatch.defaultEffort = changes.effort as string - if (Object.keys(defaultsPatch).length > 0) { + if (hasModelChange) { void dispatch(saveServerSettingsPatch({ - agentChat: { providers: { [paneContent.provider]: defaultsPatch } }, + agentChat: { + providers: { + [paneContent.provider]: { + modelSelection: nextModelSelection, + }, + }, + }, })) } - }, [tabId, paneId, dispatch, ws]) + + if (hasPermissionModeChange && changes.permissionMode) { + void dispatch(saveServerSettingsPatch({ + agentChat: { + providers: { + [paneContent.provider]: { + defaultPermissionMode: changes.permissionMode as string, + }, + }, + }, + })) + } + + if (hasEffortChange) { + void dispatch(saveServerSettingsPatch({ + agentChat: { + providers: { + [paneContent.provider]: { + effort: nextEffort, + }, + }, + }, + })) + } + + const effectiveEffort = hasEffortChange ? nextEffort : pc.effort + + if (hasModelChange && nextModelValue && effectiveEffort) { + void (async () => { + let capabilities = providerCapabilitiesRef.current + if (!capabilities) { + const response = await dispatch(fetchAgentChatCapabilities(pc.provider)) + if (!response.ok) return + capabilities = response.capabilities + } + + const resolvedSelection = resolveAgentChatModelSelection({ + providerDefaultModelId, + capabilities, + modelSelection: nextModelSelection, + }) + if (!resolvedSelection.capability || isAgentChatEffortSupported(resolvedSelection.capability, effectiveEffort)) { + return + } + + dispatch(mergePaneContent({ + tabId, + paneId, + updates: { effort: undefined }, + })) + clearPersistedProviderEffortIfPaneMatchesDefaults(pc) + })() + } + }, [clearPersistedProviderEffortIfPaneMatchesDefaults, dispatch, paneContent.provider, paneId, providerDefaultModelId, tabId, ws]) + + useEffect(() => { + if (paneContent.status === 'creating') return + if (!paneContent.effort) return + if (!resolvedModelSelection.capability) return + if (isAgentChatEffortSupported(resolvedModelSelection.capability, paneContent.effort)) return + + dispatch(mergePaneContent({ + tabId, + paneId, + updates: { effort: undefined }, + })) + clearPersistedProviderEffortIfPaneMatchesDefaults(paneContent) + }, [ + clearPersistedProviderEffortIfPaneMatchesDefaults, + dispatch, + paneContent.effort, + paneContent.provider, + paneContent.status, + paneId, + resolvedModelSelection.capability, + tabId, + ]) const handleSettingsDismiss = useCallback(() => { dispatch(updatePaneContent({ @@ -762,16 +1072,29 @@ export default function AgentChatView({ tabId, paneId, paneContent, hidden }: Ag {paneContent.initialCwd} )} 0 ? availableModels : undefined} + modelOptions={settingsModelOptions} + effortOptions={effortOptions} + capabilitiesStatus={providerCapabilitiesState?.status ?? 'idle'} + capabilityError={providerCapabilitiesState?.error} settingsVisibility={providerConfig?.settingsVisibility} + onRetryCapabilities={() => { + void dispatch(refreshAgentChatCapabilities(paneContent.provider)) + }} + onOpenChange={(open) => { + if (!open) return + const status = providerCapabilitiesState?.status ?? 'idle' + if (status === 'loading' || status === 'failed') return + if (isAgentChatCapabilitiesFresh(providerCapabilitiesState?.capabilities)) return + void dispatch(fetchAgentChatCapabilities(paneContent.provider)) + }} onChange={handleSettingsChange} onDismiss={handleSettingsDismiss} /> diff --git a/src/components/panes/PaneContainer.tsx b/src/components/panes/PaneContainer.tsx index f2297bc5..15581869 100644 --- a/src/components/panes/PaneContainer.tsx +++ b/src/components/panes/PaneContainer.tsx @@ -33,6 +33,7 @@ import { ContextIds } from '@/components/context-menu/context-menu-constants' import type { CodingCliProviderName } from '@/lib/coding-cli-types' import type { ChatSessionState, PendingAgentCreate } from '@/store/agentChatTypes' import type { AgentChatPaneContent } from '@/store/paneTypes' +import { normalizeAgentChatEffortOverride, normalizeAgentChatModelSelection } from '@/store/paneTypes' import { clearPaneAttention, clearTabAttention } from '@/store/turnCompletionSlice' import { clearPendingCreate, removeSession } from '@/store/agentChatSlice' import { cancelCreate } from '@/lib/sdk-message-handler' @@ -543,9 +544,9 @@ function PickerWrapper({ provider: type, createRequestId: nanoid(), status: 'creating', - model: providerSettings?.defaultModel ?? providerConfig.defaultModel, + modelSelection: normalizeAgentChatModelSelection(providerSettings?.modelSelection), permissionMode: providerSettings?.defaultPermissionMode ?? providerConfig.defaultPermissionMode, - effort: providerSettings?.defaultEffort ?? providerConfig.defaultEffort, + effort: normalizeAgentChatEffortOverride(providerSettings?.effort), plugins: agentChatSettings?.defaultPlugins, ...(cwd ? { initialCwd: cwd } : {}), } diff --git a/src/lib/agent-chat-capabilities.ts b/src/lib/agent-chat-capabilities.ts new file mode 100644 index 00000000..39e33a0d --- /dev/null +++ b/src/lib/agent-chat-capabilities.ts @@ -0,0 +1,269 @@ +import type { + AgentChatCapabilities, + AgentChatExactModelSelection, + AgentChatModelCapability, + AgentChatModelSelection, +} from '@shared/agent-chat-capabilities' +import { AGENT_CHAT_CAPABILITY_CACHE_TTL_MS as AGENT_CHAT_CAPABILITY_CACHE_TTL_MS_VALUE } from '@shared/agent-chat-capabilities' + +export const AGENT_CHAT_CAPABILITY_CACHE_TTL_MS = AGENT_CHAT_CAPABILITY_CACHE_TTL_MS_VALUE + +const AGENT_CHAT_MODEL_SELECTION_OPTION_VALUE_PREFIX = '__agent_chat_selection__:' + +type EncodedAgentChatSettingsModelValue = + | { kind: 'provider-default' } + | AgentChatModelSelection + +function encodeAgentChatSettingsModelValue( + value: EncodedAgentChatSettingsModelValue, +): string { + return `${AGENT_CHAT_MODEL_SELECTION_OPTION_VALUE_PREFIX}${encodeURIComponent(JSON.stringify(value))}` +} + +function decodeAgentChatSettingsModelValue( + value: string, +): EncodedAgentChatSettingsModelValue | undefined { + if (!value.startsWith(AGENT_CHAT_MODEL_SELECTION_OPTION_VALUE_PREFIX)) { + return undefined + } + + try { + const parsed = JSON.parse( + decodeURIComponent(value.slice(AGENT_CHAT_MODEL_SELECTION_OPTION_VALUE_PREFIX.length)), + ) as unknown + if (!parsed || typeof parsed !== 'object' || !('kind' in parsed)) { + return undefined + } + + const candidate = parsed as { kind?: unknown; modelId?: unknown } + if (candidate.kind === 'provider-default') { + return { kind: 'provider-default' } + } + if ( + (candidate.kind === 'tracked' || candidate.kind === 'exact') + && typeof candidate.modelId === 'string' + && candidate.modelId.trim().length > 0 + ) { + return { + kind: candidate.kind, + modelId: candidate.modelId, + } + } + } catch { + return undefined + } + + return undefined +} + +export const AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE = encodeAgentChatSettingsModelValue({ + kind: 'provider-default', +}) + +export type AgentChatSettingsModelOption = { + value: string + label: string + description?: string + unavailable?: boolean +} + +export type ResolvedAgentChatModelSelection = + | { + source: 'provider-default' + resolvedModelId: string + capability?: AgentChatModelCapability + unavailableExactSelection?: undefined + } + | { + source: 'tracked' + resolvedModelId: string + capability?: AgentChatModelCapability + unavailableExactSelection?: undefined + } + | { + source: 'exact' + resolvedModelId: string + capability: AgentChatModelCapability + unavailableExactSelection?: undefined + } + | { + source: 'exact' + resolvedModelId?: undefined + capability?: undefined + unavailableExactSelection: AgentChatExactModelSelection + } + +type ResolveAgentChatModelSelectionArgs = { + providerDefaultModelId: string + capabilities?: AgentChatCapabilities + modelSelection?: AgentChatModelSelection +} + +export function getAgentChatModelCapability( + capabilities: AgentChatCapabilities | undefined, + modelId: string, +): AgentChatModelCapability | undefined { + return capabilities?.models.find((model) => model.id === modelId) +} + +export function resolveAgentChatModelSelection( + args: ResolveAgentChatModelSelectionArgs, +): ResolvedAgentChatModelSelection { + if (!args.modelSelection) { + return { + source: 'provider-default', + resolvedModelId: args.providerDefaultModelId, + capability: getAgentChatModelCapability(args.capabilities, args.providerDefaultModelId), + } + } + + const capability = getAgentChatModelCapability(args.capabilities, args.modelSelection.modelId) + if (args.modelSelection.kind === 'tracked') { + return { + source: 'tracked', + resolvedModelId: args.modelSelection.modelId, + capability, + } + } + + if (capability) { + return { + source: 'exact', + resolvedModelId: args.modelSelection.modelId, + capability, + } + } + + return { + source: 'exact', + resolvedModelId: undefined, + unavailableExactSelection: args.modelSelection, + } +} + +export function getAgentChatSupportedEffortLevels( + args: ResolveAgentChatModelSelectionArgs, +): string[] { + return resolveAgentChatModelSelection(args).capability?.supportedEffortLevels ?? [] +} + +export function isAgentChatCapabilitiesFresh( + capabilities: AgentChatCapabilities | undefined, + now: number = Date.now(), + ttlMs: number = AGENT_CHAT_CAPABILITY_CACHE_TTL_MS, +): boolean { + return Boolean(capabilities && now - capabilities.fetchedAt <= ttlMs) +} + +export function getAgentChatSettingsModelValue( + modelSelection: AgentChatModelSelection | undefined, + capabilities?: AgentChatCapabilities, +): string { + if (!modelSelection) { + return AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE + } + + if ( + modelSelection.kind === 'exact' + && !getAgentChatModelCapability(capabilities, modelSelection.modelId) + ) { + return encodeAgentChatSettingsModelValue(modelSelection) + } + + return encodeAgentChatSettingsModelValue({ + kind: 'tracked', + modelId: modelSelection.modelId, + }) +} + +export function parseAgentChatSettingsModelValue( + value: string, +): AgentChatModelSelection | undefined { + const decoded = decodeAgentChatSettingsModelValue(value) + if (decoded) { + if (decoded.kind === 'provider-default') { + return undefined + } + return decoded + } + + if (value.trim().length === 0) { + return undefined + } + + return { + kind: 'tracked', + modelId: value, + } +} + +export function requiresAgentChatCapabilityValidation(args: { + modelSelection?: AgentChatModelSelection + effort?: string +}): boolean { + return Boolean(args.effort) || args.modelSelection?.kind === 'exact' +} + +export function isAgentChatEffortSupported( + capability: AgentChatModelCapability | undefined, + effort: string | undefined, +): boolean { + return Boolean( + capability + && effort + && capability.supportedEffortLevels.includes(effort), + ) +} + +export function getAgentChatModelOptions( + capabilities: AgentChatCapabilities | undefined, +): Array<{ value: string; displayName: string; description?: string }> | undefined { + const options = capabilities?.models.map((model) => ({ + value: model.id, + displayName: model.displayName, + description: model.description, + })) ?? [] + + return options.length > 0 ? options : undefined +} + +export function getAgentChatSettingsModelOptions(args: ResolveAgentChatModelSelectionArgs): AgentChatSettingsModelOption[] { + const options: AgentChatSettingsModelOption[] = [ + { + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }, + ...(args.capabilities?.models.map((model) => ({ + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: model.id }), + label: model.displayName, + description: model.description, + })) ?? []), + ] + + const resolvedSelection = resolveAgentChatModelSelection(args) + if (resolvedSelection.source === 'tracked' && !resolvedSelection.capability) { + options.push({ + value: getAgentChatSettingsModelValue({ + kind: 'tracked', + modelId: resolvedSelection.resolvedModelId, + }), + label: `${resolvedSelection.resolvedModelId} (Saved selection)`, + description: 'Saved tracked model is not in the latest capability catalog.', + }) + } + + if (resolvedSelection.unavailableExactSelection) { + options.push({ + value: getAgentChatSettingsModelValue( + resolvedSelection.unavailableExactSelection, + args.capabilities, + ), + label: `${resolvedSelection.unavailableExactSelection.modelId} (Unavailable)`, + description: 'Saved legacy model is no longer available.', + unavailable: true, + }) + } + + return options +} diff --git a/src/lib/agent-chat-types.ts b/src/lib/agent-chat-types.ts index 3c2d92fe..9da4adf8 100644 --- a/src/lib/agent-chat-types.ts +++ b/src/lib/agent-chat-types.ts @@ -1,7 +1,14 @@ import type { CodingCliProviderName } from '@/lib/coding-cli-types' +import type { AgentChatModelSelection } from '@shared/agent-chat-capabilities' export type AgentChatProviderName = 'freshclaude' | 'kilroy' +export type AgentChatProviderSettings = { + modelSelection?: AgentChatModelSelection + defaultPermissionMode?: string + effort?: string +} + export interface AgentChatProviderConfig { /** Unique identifier for this agent chat provider */ name: AgentChatProviderName @@ -11,12 +18,10 @@ export interface AgentChatProviderConfig { codingCliProvider: CodingCliProviderName /** React component for the pane icon */ icon: React.ComponentType<{ className?: string }> - /** Default model ID */ - defaultModel: string + /** Stable provider-default track alias used when no selection is stored */ + providerDefaultModelId: string /** Default permission mode */ defaultPermissionMode: string - /** Default effort level */ - defaultEffort: 'low' | 'medium' | 'high' | 'max' /** Which settings are visible in the settings popover */ settingsVisibility: { model: boolean diff --git a/src/lib/agent-chat-utils.ts b/src/lib/agent-chat-utils.ts index 85c63140..957c1973 100644 --- a/src/lib/agent-chat-utils.ts +++ b/src/lib/agent-chat-utils.ts @@ -14,9 +14,8 @@ export const AGENT_CHAT_PROVIDER_CONFIGS: AgentChatProviderConfig[] = [ label: 'Freshclaude', codingCliProvider: 'claude', icon: FreshclaudeIcon, - defaultModel: 'claude-opus-4-6', + providerDefaultModelId: 'opus', defaultPermissionMode: 'bypassPermissions', - defaultEffort: 'high', settingsVisibility: { model: true, permissionMode: true, @@ -32,9 +31,8 @@ export const AGENT_CHAT_PROVIDER_CONFIGS: AgentChatProviderConfig[] = [ label: 'Kilroy', codingCliProvider: 'claude', icon: KilroyIcon, - defaultModel: 'claude-opus-4-6', + providerDefaultModelId: 'opus', defaultPermissionMode: 'bypassPermissions', - defaultEffort: 'high', settingsVisibility: { model: true, permissionMode: true, diff --git a/src/lib/api.ts b/src/lib/api.ts index 6c97f208..c8daa563 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -3,6 +3,10 @@ import { getClientPerfConfig, isClientPerfLoggingEnabled, logClientPerf } from ' import { getAuthToken } from '@/lib/auth' import { sanitizeSessionLocators } from '@/lib/session-utils' import type { SessionLocator } from '@/store/paneTypes' +import { + AgentChatCapabilitiesResponseSchema, + type AgentChatCapabilitiesResponse, +} from '@shared/agent-chat-capabilities' import { AgentTimelinePageQuerySchema, AgentTimelineTurnBodyQuerySchema, @@ -31,6 +35,33 @@ export type ApiRequestOptions = { signal?: AbortSignal } +function getApiErrorMessage(data: unknown, fallback: string): string { + if (typeof data === 'object' && data !== null) { + const candidate = data as { message?: unknown; error?: unknown } + if (typeof candidate.message === 'string' && candidate.message.trim().length > 0) { + return candidate.message + } + if (typeof candidate.error === 'string' && candidate.error.trim().length > 0) { + return candidate.error + } + } + return fallback +} + +function getTypedCapabilityFailure(error: unknown): AgentChatCapabilitiesResponse | undefined { + if (!error || typeof error !== 'object' || !('details' in error)) { + return undefined + } + + const parsed = AgentChatCapabilitiesResponseSchema.safeParse( + (error as { details?: unknown }).details, + ) + if (!parsed.success || parsed.data.ok) { + return undefined + } + return parsed.data +} + export function isApiUnauthorizedError(error: unknown): error is ApiError { return ( typeof error === 'object' && @@ -119,7 +150,7 @@ async function request(path: string, options: RequestInit = {}): Promis if (!res.ok) { const err: ApiError = { status: res.status, - message: (data && (data.message || data.error)) || res.statusText, + message: getApiErrorMessage(data, res.statusText), details: data, } throw err @@ -160,6 +191,40 @@ export async function getBootstrap(options: ApiRequestOptions = {}): Promise { + try { + return AgentChatCapabilitiesResponseSchema.parse( + await api.get(`/api/agent-chat/capabilities/${encodeURIComponent(provider)}`, options), + ) + } catch (error) { + const typedFailure = getTypedCapabilityFailure(error) + if (typedFailure) { + return typedFailure + } + throw error + } +} + +export async function refreshAgentChatCapabilities( + provider: string, + options: ApiRequestOptions = {}, +): Promise { + try { + return AgentChatCapabilitiesResponseSchema.parse( + await api.post(`/api/agent-chat/capabilities/${encodeURIComponent(provider)}/refresh`, {}, options), + ) + } catch (error) { + const typedFailure = getTypedCapabilityFailure(error) + if (typedFailure) { + return typedFailure + } + throw error + } +} + export async function getSessionDirectoryPage( query: SessionDirectoryQuery, options: ApiRequestOptions = {}, diff --git a/src/lib/sdk-message-handler.ts b/src/lib/sdk-message-handler.ts index 3b379768..a1aafed6 100644 --- a/src/lib/sdk-message-handler.ts +++ b/src/lib/sdk-message-handler.ts @@ -19,7 +19,6 @@ import { sessionError, markSessionLost, removeSession, - setAvailableModels, } from '@/store/agentChatSlice' /** @@ -198,13 +197,6 @@ export function handleSdkMessage(dispatch: AppDispatch, msg: Record, - })) - return true - default: return false } diff --git a/src/lib/session-type-utils.ts b/src/lib/session-type-utils.ts index 9b5dc931..5950bbbc 100644 --- a/src/lib/session-type-utils.ts +++ b/src/lib/session-type-utils.ts @@ -2,7 +2,7 @@ import type { ComponentType } from 'react' import { PROVIDER_ICONS, DefaultProviderIcon } from '@/components/icons/provider-icons' import { isNonShellMode, getProviderLabel } from '@/lib/coding-cli-utils' import { getAgentChatProviderConfig } from '@/lib/agent-chat-utils' -import type { AgentChatProviderName } from '@/lib/agent-chat-types' +import type { AgentChatProviderName, AgentChatProviderSettings } from '@/lib/agent-chat-types' import type { CodingCliProviderName } from '@/store/types' import type { AgentChatPaneInput, TerminalPaneInput } from '@/store/paneTypes' import type { ClientExtensionEntry } from '@shared/extension-types' @@ -46,11 +46,7 @@ export function buildResumeContent(opts: { sessionType: string sessionId: string cwd?: string - agentChatProviderSettings?: { - defaultModel?: string - defaultPermissionMode?: string - defaultEffort?: 'low' | 'medium' | 'high' | 'max' - } + agentChatProviderSettings?: AgentChatProviderSettings }): TerminalPaneInput | AgentChatPaneInput { const agentConfig = getAgentChatProviderConfig(opts.sessionType) if (agentConfig) { @@ -60,9 +56,9 @@ export function buildResumeContent(opts: { provider: agentConfig.name as AgentChatProviderName, resumeSessionId: opts.sessionId, initialCwd: opts.cwd, - model: ps?.defaultModel ?? agentConfig.defaultModel, + modelSelection: ps?.modelSelection, permissionMode: ps?.defaultPermissionMode ?? agentConfig.defaultPermissionMode, - effort: ps?.defaultEffort ?? agentConfig.defaultEffort, + effort: ps?.effort, } } // Terminal pane (claude CLI, codex CLI, or fallback to 'claude') diff --git a/src/lib/tab-registry-snapshot.ts b/src/lib/tab-registry-snapshot.ts index b56c99ea..f3f24a5f 100644 --- a/src/lib/tab-registry-snapshot.ts +++ b/src/lib/tab-registry-snapshot.ts @@ -57,7 +57,7 @@ function stripPanePayload(content: PaneContent, serverInstanceId: string): Recor resumeSessionId: content.resumeSessionId, sessionRef, initialCwd: content.initialCwd, - model: content.model, + modelSelection: content.modelSelection, permissionMode: content.permissionMode, effort: content.effort, plugins: content.plugins, diff --git a/src/store/agentChatSlice.ts b/src/store/agentChatSlice.ts index 54ceff14..a12b4cf1 100644 --- a/src/store/agentChatSlice.ts +++ b/src/store/agentChatSlice.ts @@ -1,6 +1,7 @@ import { createSlice, type PayloadAction } from '@reduxjs/toolkit' import { isValidClaudeSessionId } from '@/lib/claude-session-id' import type { + AgentChatProviderCapabilitiesState, AgentChatState, AgentTimelineItem, AgentTimelineTurn, @@ -21,7 +22,7 @@ const initialState: AgentChatState = { sessions: {}, pendingCreates: {}, pendingCreateFailures: {}, - availableModels: [], + capabilitiesByProvider: {}, } /** Create a default empty session if one doesn't already exist. */ @@ -100,6 +101,18 @@ function requestRestoreHydrationRestart(session: ChatSessionState): void { session.restoreHydrationRequestId = (session.restoreHydrationRequestId ?? 0) + 1 } +function ensureCapabilitiesState( + state: AgentChatState, + provider: string, +): AgentChatProviderCapabilitiesState { + if (!state.capabilitiesByProvider[provider]) { + state.capabilitiesByProvider[provider] = { + status: 'idle', + } + } + return state.capabilitiesByProvider[provider] +} + const agentChatSlice = createSlice({ name: 'agentChat', initialState, @@ -490,10 +503,30 @@ const agentChatSlice = createSlice({ delete state.sessions[action.payload.sessionId] }, - setAvailableModels(state, action: PayloadAction<{ - models: Array<{ value: string; displayName: string; description: string }> + capabilityFetchStarted(state, action: PayloadAction<{ provider: string }>) { + const providerState = ensureCapabilitiesState(state, action.payload.provider) + providerState.status = 'loading' + providerState.error = undefined + }, + + capabilityFetchSucceeded(state, action: PayloadAction<{ + provider: string + capabilities: AgentChatProviderCapabilitiesState['capabilities'] + }>) { + state.capabilitiesByProvider[action.payload.provider] = { + status: 'succeeded', + capabilities: action.payload.capabilities, + error: undefined, + } + }, + + capabilityFetchFailed(state, action: PayloadAction<{ + provider: string + error: NonNullable }>) { - state.availableModels = action.payload.models + const providerState = ensureCapabilitiesState(state, action.payload.provider) + providerState.status = 'failed' + providerState.error = action.payload.error }, }, }) @@ -527,7 +560,9 @@ export const { clearPendingCreateFailure, clearPendingCreate, removeSession, - setAvailableModels, + capabilityFetchStarted, + capabilityFetchSucceeded, + capabilityFetchFailed, } = agentChatSlice.actions export default agentChatSlice.reducer diff --git a/src/store/agentChatThunks.ts b/src/store/agentChatThunks.ts index 23d9e683..531c3770 100644 --- a/src/store/agentChatThunks.ts +++ b/src/store/agentChatThunks.ts @@ -1,11 +1,21 @@ import { createAsyncThunk } from '@reduxjs/toolkit' import { + getAgentChatCapabilities as getAgentChatCapabilitiesApi, getAgentTimelinePage, getAgentTurnBody, + refreshAgentChatCapabilities as refreshAgentChatCapabilitiesApi, } from '@/lib/api' import { isValidClaudeSessionId } from '@/lib/claude-session-id' +import { + AgentChatCapabilitiesResponseSchema, + type AgentChatCapabilitiesResponse, + type AgentChatCapabilityError, +} from '@shared/agent-chat-capabilities' import type { AppDispatch, RootState } from './store' import { + capabilityFetchFailed, + capabilityFetchStarted, + capabilityFetchSucceeded, restoreRetryRequested, timelineLoadFailed, timelineLoadStarted, @@ -88,6 +98,73 @@ function requestStaleRestoreRetry( return true } +function normalizeCapabilityFetchError(error: unknown): AgentChatCapabilityError { + const parsed = AgentChatCapabilitiesResponseSchema.safeParse( + typeof error === 'object' && error !== null && 'details' in error + ? (error as { details?: unknown }).details + : undefined, + ) + if (parsed.success && !parsed.data.ok) { + return parsed.data.error + } + + return { + code: 'CAPABILITY_FETCH_FAILED', + message: getTimelineErrorMessage(error, 'Capability request failed'), + retryable: true, + } +} + +async function loadAgentChatCapabilitiesForProvider( + dispatch: AppDispatch, + provider: string, + refresh: boolean, +): Promise { + dispatch(capabilityFetchStarted({ provider })) + + try { + const response = refresh + ? await refreshAgentChatCapabilitiesApi(provider, {}) + : await getAgentChatCapabilitiesApi(provider, {}) + + if (response.ok) { + dispatch(capabilityFetchSucceeded({ + provider, + capabilities: response.capabilities, + })) + } else { + dispatch(capabilityFetchFailed({ + provider, + error: response.error, + })) + } + + return response + } catch (error) { + const normalizedError = normalizeCapabilityFetchError(error) + dispatch(capabilityFetchFailed({ + provider, + error: normalizedError, + })) + return { + ok: false, + error: normalizedError, + } + } +} + +export function fetchAgentChatCapabilities(provider: string) { + return async (dispatch: AppDispatch): Promise => { + return loadAgentChatCapabilitiesForProvider(dispatch, provider, false) + } +} + +export function refreshAgentChatCapabilities(provider: string) { + return async (dispatch: AppDispatch): Promise => { + return loadAgentChatCapabilitiesForProvider(dispatch, provider, true) + } +} + export const loadAgentTurnBody = createAsyncThunk< void, LoadAgentTurnBodyArgs, diff --git a/src/store/agentChatTypes.ts b/src/store/agentChatTypes.ts index 0ec4d1fa..23fdaad5 100644 --- a/src/store/agentChatTypes.ts +++ b/src/store/agentChatTypes.ts @@ -1,3 +1,8 @@ +import type { + AgentChatCapabilities, + AgentChatCapabilityError, +} from '@shared/agent-chat-capabilities' + export interface ChatContentBlock { type: 'text' | 'thinking' | 'tool_use' | 'tool_result' // text block @@ -121,12 +126,18 @@ export interface PendingAgentCreate { expectsHistoryHydration: boolean } +export interface AgentChatProviderCapabilitiesState { + status: 'idle' | 'loading' | 'succeeded' | 'failed' + capabilities?: AgentChatCapabilities + error?: AgentChatCapabilityError +} + export interface AgentChatState { sessions: Record /** Maps createRequestId -> sessionId for correlating sdk.created responses */ pendingCreates: Record /** Request-scoped pre-session failures keyed by createRequestId. */ pendingCreateFailures: Record - /** Available models from SDK supportedModels() */ - availableModels: Array<{ value: string; displayName: string; description: string }> + /** Runtime capability catalogs keyed by provider. */ + capabilitiesByProvider: Record } diff --git a/src/store/paneTreeValidation.ts b/src/store/paneTreeValidation.ts index 1316f9c9..ecb40c7d 100644 --- a/src/store/paneTreeValidation.ts +++ b/src/store/paneTreeValidation.ts @@ -1,4 +1,4 @@ -import type { PaneNode } from './paneTypes' +import { isAgentChatModelSelection, normalizeAgentChatEffortOverride, type PaneNode } from './paneTypes' function isRecord(value: unknown): value is Record { return !!value && typeof value === 'object' @@ -41,13 +41,9 @@ function isPaneContentShape(content: unknown): boolean { && isOptionalString(content.sessionId) && isOptionalString(content.resumeSessionId) && isOptionalString(content.initialCwd) - && isOptionalString(content.model) + && (content.modelSelection === undefined || isAgentChatModelSelection(content.modelSelection)) && isOptionalString(content.permissionMode) - && (content.effort === undefined - || content.effort === 'low' - || content.effort === 'medium' - || content.effort === 'high' - || content.effort === 'max') + && (content.effort === undefined || normalizeAgentChatEffortOverride(content.effort) !== undefined) && (content.plugins === undefined || (Array.isArray(content.plugins) && content.plugins.every((plugin) => typeof plugin === 'string'))) && (content.settingsDismissed === undefined || typeof content.settingsDismissed === 'boolean') diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index bbbdbf8b..77803150 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -1,9 +1,44 @@ import type { TerminalStatus, TabMode, ShellType } from './types' import type { AgentChatProviderName } from '@/lib/agent-chat-types' +import { + AgentChatModelSelectionSchema, + type AgentChatModelSelection, +} from '@shared/agent-chat-capabilities' import type { SessionLocator as SharedSessionLocator } from '@shared/ws-protocol' export type SessionLocator = SharedSessionLocator +export function isAgentChatModelSelection(value: unknown): value is AgentChatModelSelection { + return AgentChatModelSelectionSchema.safeParse(value).success +} + +export function normalizeAgentChatModelSelection( + value: unknown, + legacyModel?: unknown, +): AgentChatModelSelection | undefined { + const parsed = AgentChatModelSelectionSchema.safeParse(value) + if (parsed.success) { + return parsed.data + } + + if (typeof legacyModel === 'string' && legacyModel.trim().length > 0) { + return { + kind: 'exact', + modelId: legacyModel, + } + } + + return undefined +} + +export function normalizeAgentChatEffortOverride(value: unknown): string | undefined { + if (typeof value !== 'string') { + return undefined + } + const trimmed = value.trim() + return trimmed.length > 0 ? trimmed : undefined +} + /** * Terminal pane content with full lifecycle management. * Each terminal pane owns its backend terminal process. @@ -96,12 +131,12 @@ export type AgentChatPaneContent = { initialCwd?: string /** Request-scoped create failure promoted into pane-local visible state. */ createError?: AgentChatCreateError - /** Model to use (default from provider config) */ - model?: string + /** Stored selection strategy; omit to use the provider-default track. */ + modelSelection?: AgentChatModelSelection /** Permission mode (default from provider config) */ permissionMode?: string - /** Effort level (default from provider config, creation-time only) */ - effort?: 'low' | 'medium' | 'high' | 'max' + /** Explicit effort override; omit to use the model default behavior. */ + effort?: string /** Plugin paths to load into this session (absolute paths to plugin directories) */ plugins?: string[] /** Whether the user has dismissed the first-launch settings popover */ diff --git a/src/store/panesSlice.ts b/src/store/panesSlice.ts index d47c6d55..31f53d44 100644 --- a/src/store/panesSlice.ts +++ b/src/store/panesSlice.ts @@ -1,6 +1,14 @@ import { createSlice, PayloadAction } from '@reduxjs/toolkit' import { nanoid } from 'nanoid' -import type { PanesState, PaneContent, PaneContentInput, PaneNode, PaneRefreshRequest } from './paneTypes' +import { + normalizeAgentChatEffortOverride, + normalizeAgentChatModelSelection, + type PanesState, + type PaneContent, + type PaneContentInput, + type PaneNode, + type PaneRefreshRequest, +} from './paneTypes' import { derivePaneTitle } from '@/lib/derivePaneTitle' import { matchesDerivedPaneTitle } from '@/lib/pane-title' import { isValidClaudeSessionId } from '@/lib/claude-session-id' @@ -120,9 +128,12 @@ function normalizePaneContent( ...(sessionRef ? { sessionRef } : {}), initialCwd: input.initialCwd, createError: input.createError, - model: input.model, + modelSelection: normalizeAgentChatModelSelection( + (input as { modelSelection?: unknown }).modelSelection, + (input as { model?: unknown }).model, + ), permissionMode: input.permissionMode, - effort: input.effort, + effort: normalizeAgentChatEffortOverride(input.effort), plugins: input.plugins, settingsDismissed: input.settingsDismissed, } diff --git a/src/store/persistMiddleware.ts b/src/store/persistMiddleware.ts index f8a7a138..e0e1275a 100644 --- a/src/store/persistMiddleware.ts +++ b/src/store/persistMiddleware.ts @@ -9,6 +9,7 @@ import { PANES_SCHEMA_VERSION, LAYOUT_SCHEMA_VERSION, parsePersistedLayoutRaw } import { LAYOUT_STORAGE_KEY, PANES_STORAGE_KEY } from './storage-keys' import { createLogger } from '@/lib/client-logger' import { flushPersistedLayoutNow } from './persistControl' +import { normalizeAgentChatEffortOverride, normalizeAgentChatModelSelection } from './paneTypes' const log = createLogger('PanesPersist') @@ -128,6 +129,14 @@ function migratePaneContent(content: any): any { if (!content || typeof content !== 'object') { return content } + if (content.kind === 'agent-chat') { + const { model: _legacyModel, ...rest } = content + return { + ...rest, + modelSelection: normalizeAgentChatModelSelection(content.modelSelection, content.model), + effort: normalizeAgentChatEffortOverride(content.effort), + } + } if (content.kind === 'browser') { return { ...content, @@ -334,6 +343,15 @@ function migratePanesData(parsed: any): any | null { layouts = migratedLayouts } + // Version 6 -> 7: migrate agent-chat model/effort persistence to selection strategies. + if (currentVersion < 7) { + const migratedLayouts: Record = {} + for (const [tabId, node] of Object.entries(layouts)) { + migratedLayouts[tabId] = migrateNode(node) + } + layouts = migratedLayouts + } + const sanitizedLayouts: Record = {} const droppedTabIds = new Set() for (const [tabId, node] of Object.entries(layouts)) { diff --git a/src/store/persistedState.ts b/src/store/persistedState.ts index c57cf058..3386f1dc 100644 --- a/src/store/persistedState.ts +++ b/src/store/persistedState.ts @@ -4,7 +4,7 @@ import { LAYOUT_STORAGE_KEY, TABS_STORAGE_KEY, PANES_STORAGE_KEY } from './stora export { LAYOUT_STORAGE_KEY, TABS_STORAGE_KEY, PANES_STORAGE_KEY } export const TABS_SCHEMA_VERSION = 1 -export const PANES_SCHEMA_VERSION = 6 +export const PANES_SCHEMA_VERSION = 7 const zTabMode = z.enum(['shell', 'claude', 'codex', 'opencode', 'gemini', 'kimi']) const zCodingCliProvider = z.enum(['claude', 'codex', 'opencode', 'gemini', 'kimi']) diff --git a/src/store/settingsThunks.ts b/src/store/settingsThunks.ts index 61e26cca..d13423d9 100644 --- a/src/store/settingsThunks.ts +++ b/src/store/settingsThunks.ts @@ -44,6 +44,27 @@ function normalizeCodingCliProviderPatchForApi( return normalizedProviderPatch } +function normalizeAgentChatProviderPatchForApi( + providerPatch: Record, +): Record { + const normalizedProviderPatch = { ...providerPatch } + + if ( + Object.prototype.hasOwnProperty.call(providerPatch, 'modelSelection') + && providerPatch.modelSelection === undefined + ) { + normalizedProviderPatch.modelSelection = null + } + if ( + Object.prototype.hasOwnProperty.call(providerPatch, 'effort') + && providerPatch.effort === undefined + ) { + normalizedProviderPatch.effort = null + } + + return normalizedProviderPatch +} + export function normalizeServerSettingsPatchForApi(patch: ServerSettingsPatch): ServerSettingsPatch | Record { const normalizedPatch = isRecord(patch) ? { ...stripLocalSettings(patch) } @@ -64,6 +85,17 @@ export function normalizeServerSettingsPatchForApi(patch: ServerSettingsPatch): } } + if (isRecord(normalizedPatch.agentChat) && isRecord(normalizedPatch.agentChat.providers)) { + normalizedPatch.agentChat = { + ...normalizedPatch.agentChat, + providers: Object.fromEntries( + Object.entries(normalizedPatch.agentChat.providers).map(([providerName, providerPatch]) => ( + [providerName, isRecord(providerPatch) ? normalizeAgentChatProviderPatchForApi(providerPatch) : providerPatch] + )), + ), + } + } + return normalizedPatch } diff --git a/src/store/types.ts b/src/store/types.ts index e3016bca..26df40cb 100644 --- a/src/store/types.ts +++ b/src/store/types.ts @@ -1,7 +1,6 @@ export type TerminalStatus = 'creating' | 'running' | 'exited' | 'error' import type { - AgentChatEffort, AttentionDismiss, ClaudePermissionMode, CodingCliSettings, @@ -120,7 +119,6 @@ export interface TerminalOverride { } export type { - AgentChatEffort, AttentionDismiss, ClaudePermissionMode, CodingCliSettings, diff --git a/test/e2e-browser/helpers/test-server.test.ts b/test/e2e-browser/helpers/test-server.test.ts index 2fe2971d..ca322106 100644 --- a/test/e2e-browser/helpers/test-server.test.ts +++ b/test/e2e-browser/helpers/test-server.test.ts @@ -71,6 +71,35 @@ describe('TestServer', () => { expect(res.status).toBe(200) }) + it('preserves setupHome config seeds while adding the network bootstrap defaults', async () => { + server = new TestServer({ + preserveHomeOnStop: true, + setupHome: async (homeDir) => { + const freshellDir = path.join(homeDir, '.freshell') + await fs.mkdir(freshellDir, { recursive: true }) + await fs.writeFile(path.join(freshellDir, 'config.json'), JSON.stringify({ + version: 1, + settings: { + defaultCwd: '/tmp/freshell-seeded', + }, + legacyLocalSettingsSeed: { + theme: 'light', + }, + }, null, 2)) + }, + }) + + const info = await server.start() + const config = JSON.parse(await fs.readFile(path.join(info.homeDir, '.freshell', 'config.json'), 'utf8')) + + expect(config.settings.defaultCwd).toBe('/tmp/freshell-seeded') + expect(config.legacyLocalSettingsSeed).toMatchObject({ theme: 'light' }) + expect(config.settings.network).toMatchObject({ + configured: true, + host: '127.0.0.1', + }) + }) + it('rejects bootstrap auth against the project runtime root', () => { expect(() => new TestServer({ authStrategy: 'bootstrap', diff --git a/test/e2e-browser/helpers/test-server.ts b/test/e2e-browser/helpers/test-server.ts index 90396cac..37bce3d5 100644 --- a/test/e2e-browser/helpers/test-server.ts +++ b/test/e2e-browser/helpers/test-server.ts @@ -273,16 +273,40 @@ export class TestServer { await fsp.mkdir(freshellDir, { recursive: true }) // Pre-seed config.json so the SetupWizard does not block the UI. - // On non-WSL systems (including CI), the client shows a SetupWizard modal - // when config.json is missing, blocking all interaction. This minimal config - // marks the network as already configured, bypassing the wizard. + // Preserve any setupHome-provided config and only ensure the network + // bootstrap fields needed for the browser harness are present. const configPath = path.join(freshellDir, 'config.json') + let existingConfig: Record | null = null + try { + existingConfig = JSON.parse(await fsp.readFile(configPath, 'utf8')) as Record + } catch { + existingConfig = null + } + + const existingSettings = + existingConfig && typeof existingConfig.settings === 'object' && !Array.isArray(existingConfig.settings) + ? existingConfig.settings as Record + : {} + const existingNetwork = + typeof existingSettings.network === 'object' && !Array.isArray(existingSettings.network) + ? existingSettings.network as Record + : {} + await fsp.writeFile(configPath, JSON.stringify({ - version: 1, + ...(existingConfig ?? {}), + version: + existingConfig && typeof existingConfig.version === 'number' + ? existingConfig.version + : 1, settings: { + ...existingSettings, network: { + ...existingNetwork, configured: true, - host: '127.0.0.1', + host: + typeof existingNetwork.host === 'string' && existingNetwork.host.length > 0 + ? existingNetwork.host + : '127.0.0.1', }, }, }, null, 2)) diff --git a/test/e2e-browser/specs/agent-chat.spec.ts b/test/e2e-browser/specs/agent-chat.spec.ts index fa1cc158..1fae4084 100644 --- a/test/e2e-browser/specs/agent-chat.spec.ts +++ b/test/e2e-browser/specs/agent-chat.spec.ts @@ -9,12 +9,66 @@ test.describe('Agent Chat', () => { // Helper: open the pane picker by splitting a terminal pane. // Uses role="menuitem" for "Split horizontally" in the terminal context menu. async function openPanePicker(page: any) { + const existingPicker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + if (await existingPicker.isVisible().catch(() => false)) { + return existingPicker + } + const termContainer = page.locator('.xterm').first() - await termContainer.click({ button: 'right' }) - await page.getByRole('menuitem', { name: /split horizontally/i }).click() - // Wait for picker to appear (role="toolbar" aria-label="Pane type picker") - await expect(page.getByRole('toolbar', { name: /pane type picker/i })) - .toBeVisible({ timeout: 10_000 }) + if (await termContainer.isVisible().catch(() => false)) { + await termContainer.click({ button: 'right' }) + await page.getByRole('menuitem', { name: /split horizontally/i }).click() + } else { + await page.getByRole('button', { name: /add pane/i }).click() + } + const picker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + await expect(picker).toBeVisible({ timeout: 10_000 }) + return picker + } + + async function openFreshclaudeSettings(page: any) { + const pane = page.getByRole('group', { name: /pane: freshclaude/i }).last() + await expect(pane).toBeVisible({ timeout: 10_000 }) + + const dialog = pane.getByRole('dialog', { name: 'Agent chat settings' }) + if (!await dialog.isVisible().catch(() => false)) { + await pane.getByRole('button', { name: /^settings$/i }).click() + } + + await expect(dialog).toBeVisible({ timeout: 10_000 }) + return dialog + } + + async function confirmFreshclaudeDirectory(page: any, cwd: string) { + const directoryInput = page.getByRole('combobox', { name: /starting directory for freshclaude/i }).last() + const pickerAppeared = await directoryInput + .waitFor({ state: 'visible', timeout: 2_000 }) + .then(() => true) + .catch(() => false) + if (!pickerAppeared) { + return + } + + const waitForDismissal = async (timeout: number) => { + try { + await directoryInput.waitFor({ state: 'hidden', timeout }) + return true + } catch { + return false + } + } + + const suggestionList = page.getByRole('listbox').last() + if (await suggestionList.isVisible().catch(() => false)) { + await suggestionList.getByRole('option').first().click({ force: true }) + if (await waitForDismissal(2_000)) { + return + } + } + + await directoryInput.fill(cwd) + await directoryInput.press('Enter') + await directoryInput.waitFor({ state: 'hidden', timeout: 10_000 }) } async function getActiveLeaf(harness: any) { @@ -25,6 +79,24 @@ test.describe('Agent Chat', () => { return { tabId: tabId!, paneId: layout.id as string } } + async function enableFreshclaude(page: any) { + await page.evaluate(() => { + const harness = window.__FRESHELL_TEST_HARNESS__ + harness?.dispatch({ + type: 'connection/setAvailableClis', + payload: { claude: true }, + }) + harness?.dispatch({ + type: 'settings/updateSettingsLocal', + payload: { + codingCli: { + enabledProviders: ['claude'], + }, + }, + }) + }) + } + test('pane picker shows base pane types', async ({ freshellPage, page, terminal }) => { await terminal.waitForTerminal() await openPanePicker(page) @@ -47,24 +119,183 @@ test.describe('Agent Chat', () => { test('agent chat provider appears when the Claude CLI is available and enabled', async ({ freshellPage, page, terminal }) => { await terminal.waitForTerminal() - await page.evaluate(() => { - const harness = window.__FRESHELL_TEST_HARNESS__ - harness?.dispatch({ - type: 'connection/setAvailableClis', - payload: { claude: true }, + await enableFreshclaude(page) + + const picker = await openPanePicker(page) + await expect(picker.getByRole('button', { name: /^Freshclaude$/i })).toBeVisible() + }) + + test('freshclaude settings render provider-default tracking and create with opus', async ({ freshellPage: _freshellPage, page, harness, serverInfo, terminal }) => { + await terminal.waitForTerminal() + await enableFreshclaude(page) + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }), }) - harness?.dispatch({ - type: 'settings/updateSettingsLocal', + }) + + await harness.clearSentWsMessages() + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshclaude$/i }).click({ force: true }) + await confirmFreshclaudeDirectory(page, serverInfo.homeDir) + + const dialog = await openFreshclaudeSettings(page) + + const modelLabels = await dialog.getByRole('combobox', { name: /^Model$/i }).locator('option').evaluateAll( + (options) => options.map((option) => option.textContent), + ) + expect(modelLabels).toEqual([ + 'Provider default (track latest Opus)', + 'Opus', + 'Opus 1M', + 'Haiku', + ]) + await expect(dialog.getByText('Tracks latest Opus automatically.')).toBeVisible() + + const effortLabels = await dialog.getByRole('combobox', { name: /^Effort$/i }).locator('option').evaluateAll( + (options) => options.map((option) => option.textContent), + ) + expect(effortLabels).toEqual(['Model default', 'turbo', 'warp']) + await expect(dialog).toHaveScreenshot('freshclaude-settings-surface.png') + + await expect.poll(async () => { + const sent = await harness.getSentWsMessages() + return sent.find((msg: any) => msg?.type === 'sdk.create') ?? null + }).toMatchObject({ + type: 'sdk.create', + model: 'opus', + }) + }) + + test('opening freshclaude settings refreshes stale cached capabilities before rendering live options', async ({ freshellPage: _freshellPage, page, harness, serverInfo, terminal }) => { + await terminal.waitForTerminal() + await enableFreshclaude(page) + + let capabilityRequests = 0 + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + capabilityRequests += 1 + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }), + }) + }) + + await harness.clearSentWsMessages() + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshclaude$/i }).click({ force: true }) + await confirmFreshclaudeDirectory(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await harness.getSentWsMessages() + return sent.find((msg: any) => msg?.type === 'sdk.create') ?? null + }).toMatchObject({ + type: 'sdk.create', + model: 'opus', + }) + + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.dispatch({ + type: 'agentChat/capabilityFetchSucceeded', payload: { - codingCli: { - enabledProviders: ['claude'], + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: 0, + models: [ + { + id: 'legacy-default', + displayName: 'Legacy Default', + description: 'Old cached row', + supportsEffort: true, + supportedEffortLevels: ['old-effort'], + supportsAdaptiveThinking: false, + }, + ], }, }, }) }) - await openPanePicker(page) - await expect(page.getByRole('button', { name: /^Freshclaude$/i })).toBeVisible() + const dialog = await openFreshclaudeSettings(page) + await expect.poll(() => capabilityRequests > 0).toBe(true) + + await expect.poll(async () => ( + dialog.getByRole('combobox', { name: /^Model$/i }).locator('option').evaluateAll( + (options) => options.map((option) => option.textContent), + ) + )).toEqual([ + 'Provider default (track latest Opus)', + 'Opus', + 'Haiku', + ]) + await expect(dialog.getByText('Old cached row')).toHaveCount(0) + + await expect.poll(async () => ( + dialog.getByRole('combobox', { name: /^Effort$/i }).locator('option').evaluateAll( + (options) => options.map((option) => option.textContent), + ) + )).toEqual(['Model default', 'turbo', 'warp']) + await expect(dialog).toHaveScreenshot('freshclaude-stale-capability-refresh.png') }) test('agent chat permission banners appear and allow sends a response', async ({ freshellPage, page, harness, terminal }) => { diff --git a/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-settings-surface-chromium-linux.png b/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-settings-surface-chromium-linux.png new file mode 100644 index 00000000..f9b2d8c8 Binary files /dev/null and b/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-settings-surface-chromium-linux.png differ diff --git a/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-stale-capability-refresh-chromium-linux.png b/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-stale-capability-refresh-chromium-linux.png new file mode 100644 index 00000000..4c208107 Binary files /dev/null and b/test/e2e-browser/specs/agent-chat.spec.ts-snapshots/freshclaude-stale-capability-refresh-chromium-linux.png differ diff --git a/test/e2e-browser/specs/settings-persistence-split.spec.ts b/test/e2e-browser/specs/settings-persistence-split.spec.ts index d4ecb658..37f9e5f8 100644 --- a/test/e2e-browser/specs/settings-persistence-split.spec.ts +++ b/test/e2e-browser/specs/settings-persistence-split.spec.ts @@ -18,6 +18,13 @@ const test = base.extend({ configured: true, host: '127.0.0.1', }, + codingCli: { + providers: { + claude: { + cwd: homeDir, + }, + }, + }, }, legacyLocalSettingsSeed: { theme: 'light', @@ -41,13 +48,134 @@ async function waitForReady(page: any): Promise { async function openSettings(page: any): Promise { await page.getByRole('button', { name: /settings/i }).click() - await expect(page.getByText('Terminal').first()).toBeVisible({ timeout: 10_000 }) + await expect(page.getByRole('tab', { name: /^Appearance$/i })).toBeVisible({ timeout: 10_000 }) +} + +async function enableFreshclaude(page: any): Promise { + await page.evaluate(() => { + const harness = window.__FRESHELL_TEST_HARNESS__ + harness?.dispatch({ + type: 'connection/setAvailableClis', + payload: { claude: true }, + }) + harness?.dispatch({ + type: 'settings/updateSettingsLocal', + payload: { + codingCli: { + enabledProviders: ['claude'], + }, + }, + }) + }) +} + +async function openPanePicker(page: any) { + const existingPicker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + if (await existingPicker.isVisible().catch(() => false)) { + return existingPicker + } + + const termContainer = page.locator('.xterm').first() + if (await termContainer.isVisible().catch(() => false)) { + await termContainer.click({ button: 'right' }) + await page.getByRole('menuitem', { name: /split horizontally/i }).click() + } else { + await page.getByRole('button', { name: /add pane/i }).click() + } + const picker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + await expect(picker).toBeVisible({ timeout: 10_000 }) + return picker +} + +async function openFreshclaudeSettings(page: any) { + const pane = page.getByRole('group', { name: /pane: freshclaude/i }).last() + await expect(pane).toBeVisible({ timeout: 10_000 }) + + const dialog = pane.getByRole('dialog', { name: 'Agent chat settings' }) + if (!await dialog.isVisible().catch(() => false)) { + await pane.getByRole('button', { name: /^settings$/i }).click() + } + + await expect(dialog).toBeVisible({ timeout: 10_000 }) + return dialog +} + +async function confirmFreshclaudeDirectory(page: any, cwd: string) { + const directoryInput = page.getByRole('combobox', { name: /starting directory for freshclaude/i }).last() + const pickerAppeared = await directoryInput + .waitFor({ state: 'visible', timeout: 2_000 }) + .then(() => true) + .catch(() => false) + if (!pickerAppeared) { + return + } + + const waitForDismissal = async (timeout: number) => { + try { + await directoryInput.waitFor({ state: 'hidden', timeout }) + return true + } catch { + return false + } + } + + const suggestionList = page.getByRole('listbox').last() + if (await suggestionList.isVisible().catch(() => false)) { + await suggestionList.getByRole('option').first().click({ force: true }) + if (await waitForDismissal(2_000)) { + return + } + } + + await directoryInput.fill(cwd) + await directoryInput.press('Enter') + await directoryInput.waitFor({ state: 'hidden', timeout: 10_000 }) +} + +async function createFreshclaudePane(page: any, cwd: string): Promise { + const initialCount = await page.getByRole('group', { name: /pane: freshclaude/i }).count() + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshclaude$/i }).click({ force: true }) + await confirmFreshclaudeDirectory(page, cwd) + await expect.poll(async () => ( + await page.getByRole('group', { name: /pane: freshclaude/i }).count() + )).toBe(initialCount + 1) +} + +async function clearSentWsMessages(page: any): Promise { + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.clearSentWsMessages?.() + }) +} + +async function getSentWsMessages(page: any): Promise { + return page.evaluate(() => window.__FRESHELL_TEST_HARNESS__?.getSentWsMessages?.() ?? []) } async function getResolvedSettings(page: any) { return page.evaluate(() => window.__FRESHELL_TEST_HARNESS__?.getState()?.settings?.settings ?? null) } +async function patchServerSettings(page: any, serverInfo: any, patch: Record) { + const response = await page.evaluate(async ({ baseUrl, token, patchPayload }) => { + const result = await fetch(`${baseUrl}/api/settings`, { + method: 'PATCH', + headers: { + 'Content-Type': 'application/json', + 'x-auth-token': token, + }, + body: JSON.stringify(patchPayload), + }) + return { ok: result.ok, status: result.status } + }, { + baseUrl: serverInfo.baseUrl, + token: serverInfo.token, + patchPayload: patch, + }) + + expect(response.ok).toBe(true) +} + async function getBrowserPreferences(page: any) { return page.evaluate((storageKey) => { const raw = window.localStorage.getItem(storageKey) @@ -137,4 +265,467 @@ test.describe('Settings Persistence Split', () => { await contextB.close() await contextA.close() }) + + test('freshclaude create-time validation clears unsupported provider-default effort from persisted settings', async ({ browser, serverInfo }) => { + const context = await browser.newContext() + const page = await context.newPage() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 2_222, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }), + }) + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await waitForReady(page) + await enableFreshclaude(page) + + await patchServerSettings(page, serverInfo, { + agentChat: { + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus', + effort: null, + }) + + await expect.poll(async () => { + const settings = await getResolvedSettings(page) + return settings?.agentChat?.providers?.freshclaude?.effort ?? null + }).toBeNull() + + const configPath = path.join(serverInfo.homeDir, '.freshell', 'config.json') + await expect.poll(async () => { + const config = JSON.parse(await fs.readFile(configPath, 'utf8')) + return config.settings.agentChat?.providers?.freshclaude?.effort ?? null + }).toBeNull() + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus', + effort: null, + }) + + await context.close() + }) + + test('freshclaude tracked model selections persist across reload and provider-default clears the override', async ({ browser, serverInfo }) => { + const context = await browser.newContext() + const page = await context.newPage() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await waitForReady(page) + await enableFreshclaude(page) + + await createFreshclaudePane(page, serverInfo.homeDir) + + const dialog = await openFreshclaudeSettings(page) + await dialog.getByRole('combobox', { name: /^Model$/i }).selectOption({ label: 'Opus 1M' }) + + await expect.poll(async () => { + const settings = await getResolvedSettings(page) + return settings?.agentChat?.providers?.freshclaude?.modelSelection?.modelId ?? null + }).toBe('opus[1m]') + await expect.poll(async () => { + const settings = await getResolvedSettings(page) + return settings?.agentChat?.providers?.freshclaude?.effort ?? null + }).toBeNull() + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus[1m]', + effort: null, + }) + + const refreshedDialog = await openFreshclaudeSettings(page) + await refreshedDialog.getByRole('combobox', { name: /^Model$/i }).selectOption({ + label: 'Provider default (track latest Opus)', + }) + + await expect.poll(async () => { + const settings = await getResolvedSettings(page) + return settings?.agentChat?.providers?.freshclaude?.modelSelection + }).toBeUndefined() + await expect.poll(async () => { + const settings = await getResolvedSettings(page) + return settings?.agentChat?.providers?.freshclaude?.effort ?? null + }).toBeNull() + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus', + effort: null, + }) + + const configPath = path.join(serverInfo.homeDir, '.freshell', 'config.json') + const config = JSON.parse(await fs.readFile(configPath, 'utf8')) + expect(config.settings.agentChat?.providers?.freshclaude?.effort).toBeUndefined() + expect(config.settings.agentChat?.providers?.freshclaude?.defaultEffort).toBeUndefined() + + await context.close() + }) + + test('freshclaude saved tracked selections stay visible and still launch when the live catalog drops them', async ({ browser, serverInfo }) => { + const context = await browser.newContext() + const page = await context.newPage() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 2_468, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await waitForReady(page) + await enableFreshclaude(page) + + await patchServerSettings(page, serverInfo, { + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + effort: null, + }, + }, + }, + }) + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'haiku', + effort: null, + }) + + const dialog = await openFreshclaudeSettings(page) + await expect( + dialog.getByRole('combobox', { name: /^Model$/i }).locator('option:checked'), + ).toHaveText('haiku (Saved selection)') + await expect(dialog.getByText('Saved tracked model is not in the latest capability catalog.')).toBeVisible() + + await context.close() + }) + + test('freshclaude legacy exact selections stay visible after reload instead of silently migrating', async ({ browser, serverInfo }) => { + const context = await browser.newContext() + const page = await context.newPage() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 3_456, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await waitForReady(page) + await enableFreshclaude(page) + + await patchServerSettings(page, serverInfo, { + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + effort: null, + }, + }, + }, + }) + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + await expect(page.getByText('Session start failed')).toBeVisible() + await expect(page.getByText('Selected model claude-opus-4-6 is no longer available.')).toBeVisible() + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + return sent.filter((message) => message?.type === 'sdk.create').length + }).toBe(0) + + const dialog = await openFreshclaudeSettings(page) + await expect( + dialog.getByRole('combobox', { name: /^Model$/i }).locator('option:checked'), + ).toHaveText('claude-opus-4-6 (Unavailable)') + await expect(dialog.getByText('Saved legacy model is no longer available.')).toBeVisible() + + await context.close() + }) + + test('freshclaude unfamiliar effort strings render and round-trip correctly across reload', async ({ browser, serverInfo }) => { + const context = await browser.newContext() + const page = await context.newPage() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 4_567, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['warp-core', 'plaid'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await waitForReady(page) + await enableFreshclaude(page) + + await patchServerSettings(page, serverInfo, { + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'warp-core', + }, + }, + }, + }) + + await page.reload() + await waitForReady(page) + await enableFreshclaude(page) + + await clearSentWsMessages(page) + await createFreshclaudePane(page, serverInfo.homeDir) + + const dialog = await openFreshclaudeSettings(page) + await expect( + dialog.getByRole('combobox', { name: /^Model$/i }).locator('option:checked'), + ).toHaveText('Opus 1M') + + const effortSelect = dialog.getByRole('combobox', { name: /^Effort$/i }) + const effortLabels = await effortSelect.locator('option').evaluateAll( + (options) => options.map((option) => option.textContent), + ) + expect(effortLabels).toEqual(['Model default', 'warp-core', 'plaid']) + await expect(effortSelect.locator('option:checked')).toHaveText('warp-core') + + await expect.poll(async () => { + const sent = await getSentWsMessages(page) + const create = sent.find((message) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus[1m]', + effort: 'warp-core', + }) + await expect(effortSelect).toBeDisabled() + + const configPath = path.join(serverInfo.homeDir, '.freshell', 'config.json') + const config = JSON.parse(await fs.readFile(configPath, 'utf8')) + expect(config.settings.agentChat?.providers?.freshclaude?.effort).toBe('warp-core') + + await context.close() + }) }) diff --git a/test/e2e-browser/specs/settings.spec.ts b/test/e2e-browser/specs/settings.spec.ts index ce4890b4..649c34a6 100644 --- a/test/e2e-browser/specs/settings.spec.ts +++ b/test/e2e-browser/specs/settings.spec.ts @@ -7,18 +7,125 @@ test.describe('Settings', () => { async function openSettings(page: any) { const settingsButton = page.getByRole('button', { name: /settings/i }) await settingsButton.click() - // Settings view renders SettingsSection headers like "Terminal", "Appearance", etc. - await expect(page.getByText('Terminal').first()).toBeVisible({ timeout: 5_000 }) + await expect(page.getByRole('tab', { name: /^Appearance$/i })).toBeVisible({ timeout: 5_000 }) + } + + async function openSettingsSection(page: any, name: string) { + await page.getByRole('tab', { name: new RegExp(`^${name}$`, 'i') }).click() + await expect(page.getByRole('tabpanel', { name: new RegExp(`${name} settings`, 'i') })).toBeVisible({ + timeout: 5_000, + }) + } + + async function enableFreshclaude(page: any) { + await page.evaluate(() => { + const harness = window.__FRESHELL_TEST_HARNESS__ + harness?.dispatch({ + type: 'connection/setAvailableClis', + payload: { claude: true }, + }) + harness?.dispatch({ + type: 'settings/updateSettingsLocal', + payload: { + codingCli: { + enabledProviders: ['claude'], + }, + }, + }) + }) + } + + async function openPanePicker(page: any) { + const existingPicker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + if (await existingPicker.isVisible().catch(() => false)) { + return existingPicker + } + + const termContainer = page.locator('.xterm').first() + if (await termContainer.isVisible().catch(() => false)) { + await termContainer.click({ button: 'right' }) + await page.getByRole('menuitem', { name: /split horizontally/i }).click() + } else { + await page.getByRole('button', { name: /add pane/i }).click() + } + const picker = page.getByRole('toolbar', { name: /pane type picker/i }).last() + await expect(picker).toBeVisible({ timeout: 10_000 }) + return picker + } + + async function openFreshclaudeSettings(page: any) { + const pane = page.getByRole('group', { name: /pane: freshclaude/i }).last() + await expect(pane).toBeVisible({ timeout: 10_000 }) + + const dialog = pane.getByRole('dialog', { name: 'Agent chat settings' }) + if (!await dialog.isVisible().catch(() => false)) { + await pane.getByRole('button', { name: /^settings$/i }).click() + } + + await expect(dialog).toBeVisible({ timeout: 10_000 }) + return dialog + } + + async function confirmFreshclaudeDirectory(page: any, cwd: string) { + const directoryInput = page.getByRole('combobox', { name: /starting directory for freshclaude/i }).last() + const pickerAppeared = await directoryInput + .waitFor({ state: 'visible', timeout: 2_000 }) + .then(() => true) + .catch(() => false) + if (!pickerAppeared) { + return + } + + const waitForDismissal = async (timeout: number) => { + try { + await directoryInput.waitFor({ state: 'hidden', timeout }) + return true + } catch { + return false + } + } + + const suggestionList = page.getByRole('listbox').last() + if (await suggestionList.isVisible().catch(() => false)) { + await suggestionList.getByRole('option').first().click({ force: true }) + if (await waitForDismissal(2_000)) { + return + } + } + + await directoryInput.fill(cwd) + await directoryInput.press('Enter') + await directoryInput.waitFor({ state: 'hidden', timeout: 10_000 }) + } + + async function patchServerSettings(page: any, serverInfo: any, patch: Record) { + const response = await page.evaluate(async ({ baseUrl, token, patchPayload }) => { + const result = await fetch(`${baseUrl}/api/settings`, { + method: 'PATCH', + headers: { + 'Content-Type': 'application/json', + 'x-auth-token': token, + }, + body: JSON.stringify(patchPayload), + }) + return { ok: result.ok, status: result.status } + }, { + baseUrl: serverInfo.baseUrl, + token: serverInfo.token, + patchPayload: patch, + }) + + expect(response.ok).toBe(true) } test('settings view is accessible from sidebar', async ({ freshellPage, page }) => { await openSettings(page) - // Verify multiple settings sections are visible - // SettingsSection titles: "Appearance", "Terminal", "Debugging" (from SettingsView.tsx) - await expect(page.getByText('Appearance').first()).toBeVisible() - await expect(page.getByText('Terminal').first()).toBeVisible() - await expect(page.getByText('Debugging').first()).toBeVisible() + await expect(page.getByRole('tab', { name: /^Appearance$/i })).toBeVisible() + await expect(page.getByRole('tab', { name: /^Workspace$/i })).toBeVisible() + await expect(page.getByRole('tab', { name: /^AI$/i })).toBeVisible() + await expect(page.getByRole('tab', { name: /^Safety$/i })).toBeVisible() + await expect(page.getByRole('tab', { name: /^Advanced$/i })).toBeVisible() }) test('terminal font size slider changes setting', async ({ freshellPage, page, harness }) => { @@ -121,6 +228,7 @@ test.describe('Settings', () => { test('scrollback lines slider changes setting', async ({ freshellPage, page, harness }) => { await openSettings(page) + await openSettingsSection(page, 'Advanced') // "Scrollback lines" row with RangeSlider const scrollbackRow = page.getByText('Scrollback lines') @@ -139,6 +247,7 @@ test.describe('Settings', () => { test('debug logging toggle', async ({ freshellPage, page, harness }) => { await openSettings(page) + await openSettingsSection(page, 'Advanced') // Scroll down to "Debugging" section, find "Debug logging" row const debugLoggingRow = page.getByText('Debug logging') @@ -168,4 +277,173 @@ test.describe('Settings', () => { page.getByRole('button', { name: /system|light|dark/i }).first() ).toBeVisible() }) + + test('freshclaude settings show an explicit capability error while provider-default create remains safe', async ({ freshellPage: _freshellPage, page, harness, serverInfo, terminal }) => { + await terminal.waitForTerminal() + await enableFreshclaude(page) + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 503, + contentType: 'application/json', + body: JSON.stringify({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'Probe failed upstream', + retryable: true, + }, + }), + }) + }) + + let refreshed = false + await page.route('**/api/agent-chat/capabilities/freshclaude/refresh', async (route) => { + refreshed = true + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await harness.clearSentWsMessages() + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshclaude$/i }).click({ force: true }) + await confirmFreshclaudeDirectory(page, serverInfo.homeDir) + + await expect.poll(async () => { + const sent = await harness.getSentWsMessages() + const create = sent.find((message: any) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus', + effort: null, + }) + + const dialog = await openFreshclaudeSettings(page) + await expect(dialog.getByRole('alert')).toContainText('Probe failed upstream') + + await dialog.getByRole('button', { name: 'Retry model load' }).click() + + await expect.poll(() => refreshed).toBe(true) + await expect(dialog.getByText('Tracks latest Opus automatically.')).toBeVisible() + }) + + test('freshclaude capability failures block validation-dependent create until retry succeeds', async ({ freshellPage: _freshellPage, page, harness, serverInfo, terminal }) => { + await terminal.waitForTerminal() + + await page.route('**/api/agent-chat/capabilities/freshclaude', async (route) => { + await route.fulfill({ + status: 503, + contentType: 'application/json', + body: JSON.stringify({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'Probe failed upstream', + retryable: true, + }, + }), + }) + }) + + let refreshed = false + await page.route('**/api/agent-chat/capabilities/freshclaude/refresh', async (route) => { + refreshed = true + await route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }), + }) + }) + + await patchServerSettings(page, serverInfo, { + agentChat: { + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + + await page.goto(`${serverInfo.baseUrl}/?token=${serverInfo.token}&e2e=1`) + await harness.waitForHarness() + await harness.waitForConnection() + await terminal.waitForTerminal() + await enableFreshclaude(page) + + await harness.clearSentWsMessages() + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshclaude$/i }).click({ force: true }) + await confirmFreshclaudeDirectory(page, serverInfo.homeDir) + + const createFailedAlert = page.getByRole('alert').filter({ hasText: 'Session start failed' }) + await expect(createFailedAlert).toBeVisible() + await expect(createFailedAlert).toContainText('Probe failed upstream') + await expect.poll(async () => { + const sent = await harness.getSentWsMessages() + return sent.filter((message: any) => message?.type === 'sdk.create').length + }).toBe(0) + + const dialog = await openFreshclaudeSettings(page) + await expect(dialog.getByRole('alert')).toContainText('Probe failed upstream') + await dialog.getByRole('button', { name: 'Retry model load' }).click() + + await expect.poll(() => refreshed).toBe(true) + await expect(dialog.getByText('Tracks latest Opus automatically.')).toBeVisible() + await page.getByRole('button', { name: 'Retry', exact: true }).click() + + await expect.poll(async () => { + const sent = await harness.getSentWsMessages() + const create = sent.find((message: any) => message?.type === 'sdk.create') + return create + ? { + model: create.model ?? null, + effort: Object.prototype.hasOwnProperty.call(create, 'effort') ? create.effort : null, + } + : null + }).toEqual({ + model: 'opus', + effort: 'turbo', + }) + }) }) diff --git a/test/e2e/agent-chat-capability-settings-flow.test.tsx b/test/e2e/agent-chat-capability-settings-flow.test.tsx new file mode 100644 index 00000000..bea5617e --- /dev/null +++ b/test/e2e/agent-chat-capability-settings-flow.test.tsx @@ -0,0 +1,628 @@ +import { describe, it, expect, vi, afterEach, beforeAll, beforeEach } from 'vitest' +import { configureStore } from '@reduxjs/toolkit' +import { Provider, useSelector } from 'react-redux' +import { cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react' + +import AgentChatView from '@/components/agent-chat/AgentChatView' +import agentChatReducer from '@/store/agentChatSlice' +import panesReducer, { initLayout } from '@/store/panesSlice' +import settingsReducer, { defaultSettings } from '@/store/settingsSlice' +import type { AgentChatPaneContent } from '@/store/paneTypes' +import { + AGENT_CHAT_CAPABILITY_CACHE_TTL_MS, + AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + parseAgentChatSettingsModelValue, +} from '@/lib/agent-chat-capabilities' + +beforeAll(() => { + Element.prototype.scrollIntoView = vi.fn() +}) + +const wsSend = vi.fn() +const refreshAgentChatCapabilities = vi.fn() +const getAgentChatCapabilities = vi.fn() +const setSessionMetadata = vi.fn(() => Promise.resolve(undefined)) + +vi.mock('@/lib/ws-client', () => ({ + getWsClient: () => ({ + send: wsSend, + onReconnect: vi.fn(() => vi.fn()), + }), +})) + +vi.mock('@/lib/api', async () => { + const actual = await vi.importActual('@/lib/api') + return { + ...actual, + getAgentChatCapabilities: (...args: unknown[]) => getAgentChatCapabilities(...args), + refreshAgentChatCapabilities: (...args: unknown[]) => refreshAgentChatCapabilities(...args), + setSessionMetadata: (...args: unknown[]) => setSessionMetadata(...args), + } +}) + +vi.mock('@/store/settingsThunks', () => ({ + saveServerSettingsPatch: (patch: unknown) => ({ + type: 'settings/saveServerSettingsPatch', + payload: patch, + }), +})) + +function makeStore(preloadedAgentChat: Record = {}) { + return configureStore({ + reducer: { + agentChat: agentChatReducer, + panes: panesReducer, + settings: settingsReducer, + }, + preloadedState: { + settings: { + settings: { + ...defaultSettings, + agentChat: { + ...defaultSettings.agentChat, + initialSetupDone: true, + }, + }, + loaded: true, + lastSavedAt: 0, + }, + agentChat: { + sessions: {}, + pendingCreates: {}, + pendingCreateFailures: {}, + capabilitiesByProvider: {}, + ...preloadedAgentChat, + }, + }, + }) +} + +const BASE_PANE: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-1', + sessionId: 'sess-1', + status: 'idle', +} + +function renderStoreBackedPane( + paneContent: AgentChatPaneContent, + preloadedAgentChat: Record = {}, +) { + const store = makeStore(preloadedAgentChat) + store.dispatch(initLayout({ tabId: 't1', paneId: 'p1', content: paneContent })) + + function Wrapper() { + const root = useSelector((state: ReturnType) => state.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + return store +} + +function getRenderedPaneContent(store: ReturnType): AgentChatPaneContent { + const root = store.getState().panes.layouts.t1 + if (root?.type !== 'leaf' || root.content.kind !== 'agent-chat') { + throw new Error('Expected an agent chat pane at t1/p1') + } + return root.content +} + +function freshFetchedAt(): number { + return Date.now() +} + +function makeFreshOpusCapabilities(fetchedAt: number = freshFetchedAt()) { + return { + ok: true as const, + capabilities: { + provider: 'freshclaude', + fetchedAt, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + } +} + +describe('agent chat capability settings flow', () => { + beforeEach(() => { + getAgentChatCapabilities.mockResolvedValue(makeFreshOpusCapabilities()) + refreshAgentChatCapabilities.mockResolvedValue(makeFreshOpusCapabilities()) + }) + + afterEach(() => { + cleanup() + wsSend.mockClear() + refreshAgentChatCapabilities.mockReset() + getAgentChatCapabilities.mockReset() + setSessionMetadata.mockClear() + }) + + it('shows provider-default tracking plus live capability rows only, with dynamic effort options', () => { + const store = makeStore({ + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }, + }, + }) + + render( + + + , + ) + + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + + const modelSelect = screen.getByLabelText('Model') + const modelLabels = Array.from(modelSelect.querySelectorAll('option')).map((option) => option.textContent) + expect(modelLabels).toEqual([ + 'Provider default (track latest Opus)', + 'Opus', + 'Haiku', + ]) + expect(screen.getByText('Tracks latest Opus automatically.')).toBeInTheDocument() + + const effortSelect = screen.getByLabelText('Effort') + const effortLabels = Array.from(effortSelect.querySelectorAll('option')).map((option) => option.textContent) + expect(effortLabels).toEqual(['Model default', 'turbo', 'warp']) + expect(screen.queryByRole('option', { name: 'High' })).not.toBeInTheDocument() + }) + + it('keeps an unavailable exact model visible and selected until the user changes it', () => { + const store = makeStore({ + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }, + }, + }) + + render( + + + , + ) + + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + + const modelSelect = screen.getByLabelText('Model') as HTMLSelectElement + expect(parseAgentChatSettingsModelValue(modelSelect.value)).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(screen.getByRole('option', { name: 'claude-opus-4-6 (Unavailable)' })).toBeInTheDocument() + expect(screen.getByText('Saved legacy model is no longer available.')).toBeInTheDocument() + }) + + it('blocks unavailable exact create until the user switches to provider-default and retries', async () => { + const store = renderStoreBackedPane({ + ...BASE_PANE, + sessionId: undefined, + createRequestId: 'req-unavailable-exact', + status: 'creating', + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }, { + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }, + }, + }) + + expect(await screen.findByText('Session start failed')).toBeInTheDocument() + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + const modelSelect = screen.getByLabelText('Model') as HTMLSelectElement + expect(parseAgentChatSettingsModelValue(modelSelect.value)).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(screen.getByRole('option', { name: 'claude-opus-4-6 (Unavailable)' })).toBeInTheDocument() + expect(screen.getByText('Saved legacy model is no longer available.')).toBeInTheDocument() + expect(screen.getByText('Selected model claude-opus-4-6 is no longer available.')).toBeInTheDocument() + expect(wsSend.mock.calls.filter((call) => call[0]?.type === 'sdk.create')).toHaveLength(0) + expect(getRenderedPaneContent(store)).toEqual(expect.objectContaining({ + status: 'create-failed', + createError: expect.objectContaining({ + code: 'MODEL_UNAVAILABLE', + }), + })) + + fireEvent.change(modelSelect, { + target: { value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE }, + }) + expect(getRenderedPaneContent(store).modelSelection).toBeUndefined() + + fireEvent.click(screen.getByRole('button', { name: 'Retry' })) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + model: 'opus', + })) + }) + }) + + it('shows a retryable capability error and keeps a persisted tracked selection visible after retry', async () => { + refreshAgentChatCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }) + + const store = makeStore({ + capabilitiesByProvider: { + freshclaude: { + status: 'failed', + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + error: { + code: 'CAPABILITY_FETCH_FAILED', + message: 'Capability request failed', + retryable: true, + }, + }, + }, + }) + + render( + + + , + ) + + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + + expect(screen.getByRole('alert')).toHaveTextContent('Capability request failed') + expect(screen.queryByLabelText('Model')).not.toBeInTheDocument() + expect(screen.queryByLabelText('Effort')).not.toBeInTheDocument() + expect(screen.queryByText('Latest Opus track')).not.toBeInTheDocument() + fireEvent.click(screen.getByRole('button', { name: 'Retry model load' })) + + expect(refreshAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(await screen.findByText('Provider default (track latest Opus)')).toBeInTheDocument() + const modelSelect = screen.getByLabelText('Model') as HTMLSelectElement + expect(parseAgentChatSettingsModelValue(modelSelect.value)).toEqual({ + kind: 'tracked', + modelId: 'haiku', + }) + expect(screen.getByRole('option', { name: 'haiku (Saved selection)' })).toBeInTheDocument() + expect(screen.getByText('Saved tracked model is not in the latest capability catalog.')).toBeInTheDocument() + }) + + it('revalidates stale cached capabilities when settings open instead of treating them as session-lifetime truth', async () => { + getAgentChatCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + + const staleFetchedAt = Date.now() - AGENT_CHAT_CAPABILITY_CACHE_TTL_MS - 1 + const store = makeStore({ + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: staleFetchedAt, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Old Opus track', + supportsEffort: true, + supportedEffortLevels: ['old-effort'], + supportsAdaptiveThinking: true, + }, + ], + }, + }, + }, + }) + + render( + + + , + ) + + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + + await waitFor(() => { + expect(getAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + }) + expect(await screen.findByRole('option', { name: 'Haiku' })).toBeInTheDocument() + expect(screen.queryByText('Old Opus track')).not.toBeInTheDocument() + }) + + it('revalidates stale cached capabilities before validation-dependent create instead of trusting expired data', async () => { + const capabilityError = { + code: 'CAPABILITY_FETCH_FAILED', + message: 'Capability request failed', + retryable: true, + } as const + getAgentChatCapabilities.mockResolvedValue({ + ok: false, + error: capabilityError, + }) + + const staleFetchedAt = Date.now() - AGENT_CHAT_CAPABILITY_CACHE_TTL_MS - 1 + const store = renderStoreBackedPane({ + ...BASE_PANE, + sessionId: undefined, + createRequestId: 'req-stale-validation', + status: 'creating', + effort: 'turbo', + }, { + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: staleFetchedAt, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Old Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }, + }, + }) + + expect(await screen.findByText('Session start failed')).toBeInTheDocument() + expect(getAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(wsSend.mock.calls.filter((call) => call[0]?.type === 'sdk.create')).toHaveLength(0) + expect(getRenderedPaneContent(store)).toEqual(expect.objectContaining({ + status: 'create-failed', + createError: expect.objectContaining({ + code: 'CAPABILITY_FETCH_FAILED', + }), + })) + }) + + it('lets safe tracked creates proceed during capability failure but blocks validation-dependent create until retry succeeds', async () => { + const capabilityError = { + code: 'CAPABILITY_FETCH_FAILED', + message: 'Capability request failed', + retryable: true, + } as const + + refreshAgentChatCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: freshFetchedAt(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + + const safeStore = renderStoreBackedPane({ + ...BASE_PANE, + sessionId: undefined, + createRequestId: 'req-safe-tracked', + status: 'creating', + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + }, { + capabilitiesByProvider: { + freshclaude: { + status: 'failed', + error: capabilityError, + }, + }, + }) + + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + expect(screen.getByRole('alert')).toHaveTextContent('Capability request failed') + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + model: 'haiku', + })) + }) + expect(getAgentChatCapabilities).not.toHaveBeenCalled() + expect(getRenderedPaneContent(safeStore)).toEqual(expect.objectContaining({ + status: 'starting', + })) + + cleanup() + wsSend.mockClear() + getAgentChatCapabilities.mockReset() + + getAgentChatCapabilities.mockResolvedValue({ + ok: false, + error: capabilityError, + }) + + const blockedStore = renderStoreBackedPane({ + ...BASE_PANE, + sessionId: undefined, + createRequestId: 'req-blocked-validation', + status: 'creating', + effort: 'turbo', + }, { + capabilitiesByProvider: { + freshclaude: { + status: 'failed', + error: capabilityError, + }, + }, + }) + + expect(await screen.findByText('Session start failed')).toBeInTheDocument() + fireEvent.click(screen.getByRole('button', { name: 'Settings' })) + expect(screen.getAllByText('Capability request failed').length).toBeGreaterThan(0) + expect(getAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(wsSend.mock.calls.filter((call) => call[0]?.type === 'sdk.create')).toHaveLength(0) + expect(getRenderedPaneContent(blockedStore)).toEqual(expect.objectContaining({ + status: 'create-failed', + createError: expect.objectContaining({ + code: 'CAPABILITY_FETCH_FAILED', + }), + })) + + fireEvent.click(screen.getByRole('button', { name: 'Retry model load' })) + expect(refreshAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(await screen.findByText('Opus')).toBeInTheDocument() + + fireEvent.click(screen.getByRole('button', { name: 'Retry' })) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + model: 'opus', + effort: 'turbo', + })) + }) + }) +}) diff --git a/test/e2e/pane-activity-indicator-flow.test.tsx b/test/e2e/pane-activity-indicator-flow.test.tsx index 84df0a21..8a5c15a7 100644 --- a/test/e2e/pane-activity-indicator-flow.test.tsx +++ b/test/e2e/pane-activity-indicator-flow.test.tsx @@ -1,5 +1,5 @@ -import { describe, it, expect, vi, afterEach } from 'vitest' -import { render, screen, cleanup, within, act } from '@testing-library/react' +import { describe, it, expect, vi, afterEach, beforeAll } from 'vitest' +import { render, screen, cleanup, within, act, fireEvent, waitFor } from '@testing-library/react' import { Provider } from 'react-redux' import { configureStore } from '@reduxjs/toolkit' import TabBar from '@/components/TabBar' @@ -29,9 +29,19 @@ import type { } from '@/store/paneTypes' import type { Tab } from '@/store/types' +const wsSend = vi.hoisted(() => vi.fn()) +const getAgentChatCapabilities = vi.hoisted(() => vi.fn()) +const refreshAgentChatCapabilities = vi.hoisted(() => vi.fn()) +const setSessionMetadata = vi.hoisted(() => vi.fn(() => Promise.resolve(undefined))) +const saveServerSettingsPatchSpy = vi.hoisted(() => vi.fn((patch: unknown) => ({ + type: 'settings/saveServerSettingsPatch', + payload: patch, +}))) + vi.mock('@/lib/ws-client', () => ({ getWsClient: () => ({ - send: vi.fn(), + send: wsSend, + onReconnect: vi.fn(() => () => {}), }), })) @@ -42,6 +52,13 @@ vi.mock('@/lib/api', () => ({ patch: vi.fn().mockResolvedValue({}), delete: vi.fn().mockResolvedValue({}), }, + getAgentChatCapabilities: (...args: unknown[]) => getAgentChatCapabilities(...args), + refreshAgentChatCapabilities: (...args: unknown[]) => refreshAgentChatCapabilities(...args), + setSessionMetadata: (...args: unknown[]) => setSessionMetadata(...args), +})) + +vi.mock('@/store/settingsThunks', () => ({ + saveServerSettingsPatch: (patch: unknown) => saveServerSettingsPatchSpy(patch), })) vi.mock('@/store/sessionsThunks', () => ({ @@ -82,8 +99,14 @@ type RenderHarnessOptions = { paneTitle?: string paneRuntimeActivity?: PaneRuntimeActivityState agentChat?: AgentChatState + settingsOverrides?: Record } +beforeAll(() => { + Element.prototype.scrollIntoView = vi.fn() + HTMLElement.prototype.scrollIntoView = vi.fn() +}) + function renderHarness(options: RenderHarnessOptions) { const tab: Tab = { id: 'tab-activity', @@ -131,7 +154,10 @@ function renderHarness(options: RenderHarnessOptions) { refreshRequestsByPane: {}, }, settings: { - settings: defaultSettings, + settings: { + ...defaultSettings, + ...(options.settingsOverrides ?? {}), + } as typeof defaultSettings, loaded: true, lastSavedAt: null, }, @@ -184,6 +210,11 @@ function getVisibleSinglePaneTab() { describe('pane activity indicator flow (e2e)', () => { afterEach(() => { cleanup() + wsSend.mockClear() + getAgentChatCapabilities.mockReset() + refreshAgentChatCapabilities.mockReset() + setSessionMetadata.mockClear() + saveServerSettingsPatchSpy.mockClear() }) it('shows browser loading activity as blue and clears when the pane returns to idle', () => { @@ -428,4 +459,130 @@ describe('pane activity indicator flow (e2e)', () => { expect(within(paneHeader).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') expect(within(getVisibleSinglePaneTab()).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') }) + + it('keeps FreshClaude activity blue while mid-session model changes send sdk.set-model without rewriting provider defaults', async () => { + const ActualAgentChatView = ( + await vi.importActual('@/components/agent-chat/AgentChatView') + ).default + + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-agent', + sessionId: 'sess-1', + status: 'running', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'turbo', + } + + const { store } = renderHarness({ + pane, + settingsOverrides: { + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'turbo', + }, + }, + }, + }, + agentChat: { + sessions: { + 'sess-1': { + sessionId: 'sess-1', + status: 'running', + messages: [], + timelineItems: [], + timelineBodies: {}, + streamingText: '', + streamingActive: false, + pendingPermissions: {}, + pendingQuestions: {}, + totalCostUsd: 0, + totalInputTokens: 0, + totalOutputTokens: 0, + }, + }, + pendingCreates: {}, + pendingCreateFailures: {}, + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }, + }, + } as AgentChatState, + }) + + render( + +
+ +
+
, + ) + + const paneHeader = screen.getByRole('banner', { name: 'Pane: Activity Pane' }) + expect(within(paneHeader).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') + expect(within(getVisibleSinglePaneTab()).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') + + const agentChatHarness = screen.getByTestId('actual-agent-chat-harness') + const modelSelect = await within(agentChatHarness).findByLabelText('Model') as HTMLSelectElement + const haikuValue = Array.from(modelSelect.options).find((option) => option.text === 'Haiku')?.value + fireEvent.change(modelSelect, { target: { value: haikuValue } }) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith({ + type: 'sdk.set-model', + sessionId: 'sess-1', + model: 'haiku', + }) + }) + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledTimes(2) + expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(1, { + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + }, + }, + }, + }) + }) + expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(2, { + agentChat: { + providers: { + freshclaude: { + effort: undefined, + }, + }, + }, + }) + expect(within(paneHeader).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') + expect(within(getVisibleSinglePaneTab()).getByTestId('pane-icon').getAttribute('class')).toContain('text-blue-500') + }) }) diff --git a/test/e2e/pane-header-runtime-meta-flow.test.tsx b/test/e2e/pane-header-runtime-meta-flow.test.tsx index 71509c8e..f77c0c60 100644 --- a/test/e2e/pane-header-runtime-meta-flow.test.tsx +++ b/test/e2e/pane-header-runtime-meta-flow.test.tsx @@ -1,8 +1,10 @@ -import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' -import { render, screen, waitFor, cleanup, act } from '@testing-library/react' +import { describe, it, expect, vi, beforeAll, beforeEach, afterEach } from 'vitest' +import { render, screen, waitFor, cleanup, act, fireEvent, within } from '@testing-library/react' import { Provider } from 'react-redux' import { configureStore } from '@reduxjs/toolkit' import App from '@/App' +import TabBar from '@/components/TabBar' +import PaneContainer from '@/components/panes/PaneContainer' import settingsReducer, { defaultSettings } from '@/store/settingsSlice' import tabsReducer from '@/store/tabsSlice' import connectionReducer from '@/store/connectionSlice' @@ -50,6 +52,13 @@ const wsMocks = vi.hoisted(() => { const apiGet = vi.hoisted(() => vi.fn()) const fetchSidebarSessionsSnapshot = vi.hoisted(() => vi.fn()) +const getAgentChatCapabilities = vi.hoisted(() => vi.fn()) +const refreshAgentChatCapabilities = vi.hoisted(() => vi.fn()) +const setSessionMetadata = vi.hoisted(() => vi.fn(() => Promise.resolve(undefined))) +const saveServerSettingsPatchSpy = vi.hoisted(() => vi.fn((patch: unknown) => ({ + type: 'settings/saveServerSettingsPatch', + payload: patch, +}))) vi.mock('@/lib/ws-client', () => ({ getWsClient: () => ({ @@ -76,10 +85,17 @@ vi.mock('@/lib/api', () => ({ patch: vi.fn().mockResolvedValue({}), post: vi.fn().mockResolvedValue({}), }, + getAgentChatCapabilities: (...args: unknown[]) => getAgentChatCapabilities(...args), + refreshAgentChatCapabilities: (...args: unknown[]) => refreshAgentChatCapabilities(...args), + setSessionMetadata: (...args: unknown[]) => setSessionMetadata(...args), fetchSidebarSessionsSnapshot: (options?: unknown) => fetchSidebarSessionsSnapshot(options), isApiUnauthorizedError: (err: any) => !!err && typeof err === 'object' && err.status === 401, })) +vi.mock('@/store/settingsThunks', () => ({ + saveServerSettingsPatch: (patch: unknown) => saveServerSettingsPatchSpy(patch), +})) + vi.mock('@/hooks/useTheme', () => ({ useThemeEffect: () => {}, })) @@ -299,6 +315,11 @@ function createStore(options?: { }) } +beforeAll(() => { + Element.prototype.scrollIntoView = vi.fn() + HTMLElement.prototype.scrollIntoView = vi.fn() +}) + describe('pane header runtime metadata flow (e2e)', () => { beforeEach(() => { cleanup() @@ -309,6 +330,10 @@ describe('pane header runtime metadata flow (e2e)', () => { fetchSidebarSessionsSnapshot.mockReset() fetchSidebarSessionsSnapshot.mockResolvedValue([]) + getAgentChatCapabilities.mockReset() + refreshAgentChatCapabilities.mockReset() + setSessionMetadata.mockClear() + saveServerSettingsPatchSpy.mockClear() apiGet.mockImplementation((url: string) => { if (url === '/api/bootstrap') { @@ -769,6 +794,139 @@ describe('pane header runtime metadata flow (e2e)', () => { }) }) + it('keeps FreshClaude runtime metadata visible while provider-default creates with opus and tracked changes persist', async () => { + const ActualAgentChatView = ( + await vi.importActual('@/components/agent-chat/AgentChatView') + ).default + + const freshClaudePane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-fresh-create', + resumeSessionId: 'claude-session-1', + status: 'creating', + } + + const store = createStore({ + activeTabId: 'tab-fresh', + freshClaudeTab: { + id: 'tab-fresh', + createRequestId: 'req-fresh-create', + title: 'FreshClaude Tab', + status: 'running', + mode: 'claude', + resumeSessionId: 'claude-session-1', + createdAt: Date.now(), + }, + freshClaudePane, + agentChatState: { + sessions: {}, + pendingCreates: {}, + pendingCreateFailures: {}, + capabilitiesByProvider: { + freshclaude: { + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context window', + supportsEffort: true, + supportedEffortLevels: ['warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + }, + }, + } satisfies Partial, + }) + + store.dispatch(setProjects([ + { + projectPath: '/home/user/code/freshell', + sessions: [ + { + provider: 'claude', + sessionType: 'freshclaude', + sessionId: 'claude-session-1', + projectPath: '/home/user/code/freshell', + cwd: '/home/user/code/freshell/.worktrees/issue-163', + gitBranch: 'main', + isDirty: true, + lastActivityAt: 1, + tokenUsage: { + inputTokens: 10, + outputTokens: 5, + cachedTokens: 0, + totalTokens: 15, + contextTokens: 15, + compactThresholdTokens: 60, + compactPercent: 25, + }, + }, + ], + }, + ] as any)) + + render( + + <> + + +
+ +
+ +
+ ) + + await waitFor(() => { + expect(screen.getByText(/freshell \(main\*\)\s+25%/)).toBeInTheDocument() + }) + + await waitFor(() => { + expect(wsMocks.send).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-fresh-create', + model: 'opus', + })) + }) + + const agentChatHarness = screen.getByTestId('actual-agent-chat-harness') + const modelSelect = await within(agentChatHarness).findByLabelText('Model') as HTMLSelectElement + const opusOneMillionValue = Array.from(modelSelect.options).find( + (option) => option.text === 'Opus 1M', + )?.value + fireEvent.change(modelSelect, { + target: { value: opusOneMillionValue }, + }) + + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledWith({ + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + }, + }, + }, + }) + }) + expect(screen.getByText(/freshell \(main\*\)\s+25%/)).toBeInTheDocument() + }) + it('restores FreshClaude pane header metadata from timelineSessionId before cliSessionId exists', async () => { fetchSidebarSessionsSnapshot.mockResolvedValueOnce({ projects: [ diff --git a/test/integration/server/agent-chat-capabilities-router.test.ts b/test/integration/server/agent-chat-capabilities-router.test.ts new file mode 100644 index 00000000..1c1e0a49 --- /dev/null +++ b/test/integration/server/agent-chat-capabilities-router.test.ts @@ -0,0 +1,155 @@ +// @vitest-environment node +import express, { type Express } from 'express' +import request from 'supertest' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { createAgentChatCapabilitiesRouter } from '../../../server/agent-chat-capabilities-router.js' + +describe('agent chat capabilities router', () => { + let app: Express + let registry: { + getCapabilities: ReturnType + refreshCapabilities: ReturnType + } + + beforeEach(() => { + registry = { + getCapabilities: vi.fn(), + refreshCapabilities: vi.fn(), + } + + app = express() + app.use(express.json()) + app.use('/api/agent-chat/capabilities', createAgentChatCapabilitiesRouter({ registry })) + }) + + it('returns normalized runtime capabilities on success', async () => { + registry.getCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Primary track', + supportsEffort: true, + supportedEffortLevels: ['medium', 'high'], + supportsAdaptiveThinking: true, + }, + ], + }, + }) + + const res = await request(app).get('/api/agent-chat/capabilities/freshclaude') + + expect(res.status).toBe(200) + expect(res.body).toEqual({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Primary track', + supportsEffort: true, + supportedEffortLevels: ['medium', 'high'], + supportsAdaptiveThinking: true, + }, + ], + }, + }) + expect(registry.getCapabilities).toHaveBeenCalledWith('freshclaude') + }) + + it('returns refreshed capability data through the same contract', async () => { + registry.refreshCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'kilroy', + fetchedAt: 9_999, + models: [ + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + + const res = await request(app).post('/api/agent-chat/capabilities/kilroy/refresh') + + expect(res.status).toBe(200) + expect(res.body).toEqual({ + ok: true, + capabilities: { + provider: 'kilroy', + fetchedAt: 9_999, + models: [ + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + expect(registry.refreshCapabilities).toHaveBeenCalledWith('kilroy') + }) + + it('returns a typed error payload when the probe fails', async () => { + registry.getCapabilities.mockResolvedValue({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + }) + + const res = await request(app).get('/api/agent-chat/capabilities/freshclaude') + + expect(res.status).toBe(503) + expect(res.body).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + }) + }) + + it('returns typed payload-invalid errors without collapsing them', async () => { + registry.getCapabilities.mockResolvedValue({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload has invalid supported effort levels for opus', + retryable: false, + }, + }) + + const res = await request(app).get('/api/agent-chat/capabilities/freshclaude') + + expect(res.status).toBe(503) + expect(res.body).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload has invalid supported effort levels for opus', + retryable: false, + }, + }) + }) +}) diff --git a/test/integration/server/settings-api.test.ts b/test/integration/server/settings-api.test.ts index 3408fd1f..2be7771e 100644 --- a/test/integration/server/settings-api.test.ts +++ b/test/integration/server/settings-api.test.ts @@ -128,6 +128,46 @@ describe('Settings API Integration', () => { expect(res.body.agentChat.defaultPlugins).toEqual(['fs', 'search']) }) + it('PATCH /api/settings stores tracked and exact agent-chat model selections with dynamic effort strings', async () => { + const tracked = await request(app) + .patch('/api/settings') + .set('x-auth-token', TEST_AUTH_TOKEN) + .send({ + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'ultra', + }, + }, + }, + }) + + expect(tracked.status).toBe(200) + expect(tracked.body.agentChat.providers.freshclaude).toEqual({ + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'ultra', + }) + + const exact = await request(app) + .patch('/api/settings') + .set('x-auth-token', TEST_AUTH_TOKEN) + .send({ + agentChat: { + providers: { + kilroy: { + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }, + }, + }, + }) + + expect(exact.status).toBe(200) + expect(exact.body.agentChat.providers.kilroy).toEqual({ + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }) + }) + it('PATCH /api/settings preserves runtime CLI providers outside the built-in defaults', async () => { const res = await request(app) .patch('/api/settings') @@ -214,6 +254,44 @@ describe('Settings API Integration', () => { expect(res.body.codingCli.providers.codex.sandbox).toBeUndefined() }) + it('PATCH /api/settings accepts null and empty sentinels to clear agent-chat model selections and effort overrides', async () => { + await request(app) + .patch('/api/settings') + .set('x-auth-token', TEST_AUTH_TOKEN) + .send({ + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'ultra', + }, + }, + }, + }) + + const res = await request(app) + .patch('/api/settings') + .set('x-auth-token', TEST_AUTH_TOKEN) + .send({ + agentChat: { + providers: { + freshclaude: { + modelSelection: null, + effort: '', + }, + }, + }, + }) + + expect(res.status).toBe(200) + expect(res.body.agentChat.providers.freshclaude.modelSelection).toBeUndefined() + expect(res.body.agentChat.providers.freshclaude.effort).toBeUndefined() + + const stored = await configStore.getSettings() + expect(stored.agentChat.providers.freshclaude.modelSelection).toBeUndefined() + expect(stored.agentChat.providers.freshclaude.effort).toBeUndefined() + }) + it('PATCH /api/settings rejects local-only settings fields', async () => { const payloads = [ { theme: 'dark' }, diff --git a/test/unit/client/agentChatSlice.test.ts b/test/unit/client/agentChatSlice.test.ts index 780b14bf..a4572243 100644 --- a/test/unit/client/agentChatSlice.test.ts +++ b/test/unit/client/agentChatSlice.test.ts @@ -3,6 +3,9 @@ import agentChatReducer, { sessionCreated, registerPendingCreate, createFailed, + capabilityFetchFailed, + capabilityFetchStarted, + capabilityFetchSucceeded, clearPendingCreateFailure, sessionInit, sessionMetadataReceived, @@ -22,7 +25,6 @@ import agentChatReducer, { sessionError, clearPendingCreate, removeSession, - setAvailableModels, } from '../../../src/store/agentChatSlice' function makeChatMessage(role: 'user' | 'assistant', text: string) { @@ -83,7 +85,8 @@ describe('agentChatSlice', () => { it('has empty initial state', () => { expect(initial.sessions).toEqual({}) - expect(initial.availableModels).toEqual([]) + expect(initial.capabilitiesByProvider).toEqual({}) + expect(initial).not.toHaveProperty('availableModels') }) it('creates a session', () => { @@ -827,14 +830,126 @@ describe('agentChatSlice', () => { expect(state.sessions['s1']).toBeUndefined() }) - it('setAvailableModels populates models', () => { - const models = [ - { value: 'claude-opus-4-6', displayName: 'Opus 4.6', description: 'Most capable' }, - { value: 'claude-sonnet-4-5-20250929', displayName: 'Sonnet 4.5', description: 'Fast' }, - ] - const state = agentChatReducer(initial, setAvailableModels({ models })) - expect(state.availableModels).toEqual(models) - expect(state.availableModels).toHaveLength(2) + it('tracks capability fetch lifecycle by provider', () => { + let state = agentChatReducer(initial, capabilityFetchStarted({ + provider: 'freshclaude', + })) + + expect(state.capabilitiesByProvider.freshclaude).toEqual({ + status: 'loading', + capabilities: undefined, + error: undefined, + }) + + state = agentChatReducer(state, capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + })) + + expect(state.capabilitiesByProvider.freshclaude).toEqual({ + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + error: undefined, + }) + }) + + it('stores typed capability failures by provider', () => { + const state = agentChatReducer(initial, capabilityFetchFailed({ + provider: 'kilroy', + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + })) + + expect(state.capabilitiesByProvider.kilroy).toEqual({ + status: 'failed', + capabilities: undefined, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + }) + }) + + it('preserves previously loaded capabilities when a later fetch fails', () => { + const succeeded = agentChatReducer(initial, capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + })) + + const failed = agentChatReducer(succeeded, capabilityFetchFailed({ + provider: 'freshclaude', + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + })) + + expect(failed.capabilitiesByProvider.freshclaude).toEqual({ + status: 'failed', + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + }) }) it('accumulates cost when costUsd is 0', () => { diff --git a/test/unit/client/components/TabsView.test.tsx b/test/unit/client/components/TabsView.test.tsx index 8aa29a42..cb1e5ef5 100644 --- a/test/unit/client/components/TabsView.test.tsx +++ b/test/unit/client/components/TabsView.test.tsx @@ -133,6 +133,81 @@ describe('TabsView', () => { expect(tabs.some((t) => t.title === 'remote open')).toBe(true) }) + it('rehydrates remote agent-chat panes with selection strategies', () => { + const store = configureStore({ + reducer: { + tabs: tabsReducer, + panes: panesReducer, + tabRegistry: tabRegistryReducer, + connection: connectionReducer, + }, + }) + store.dispatch(setServerInstanceId('srv-local')) + store.dispatch(addTab({ id: 'local-tab', title: 'local tab', mode: 'shell' })) + store.dispatch(initLayout({ + tabId: 'local-tab', + content: { kind: 'terminal', mode: 'shell' }, + })) + store.dispatch(setTabRegistrySnapshot({ + localOpen: [], + remoteOpen: [{ + tabKey: 'remote:agent', + tabId: 'agent-1', + serverInstanceId: 'srv-remote', + deviceId: 'remote', + deviceLabel: 'remote-device', + tabName: 'remote agent', + status: 'open', + revision: 1, + createdAt: 1, + updatedAt: 2, + paneCount: 1, + titleSetByUser: false, + panes: [{ + paneId: 'pane-agent', + kind: 'agent-chat', + payload: { + provider: 'freshclaude', + resumeSessionId: '00000000-0000-4000-8000-000000000444', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + permissionMode: 'plan', + effort: 'turbo', + plugins: ['planner'], + }, + }], + }], + closed: [], + })) + + render( + + + , + ) + + fireEvent.click(screen.getByLabelText('remote-device: remote agent')) + + const copiedTab = store.getState().tabs.tabs.find((tab) => tab.title === 'remote agent') + expect(copiedTab).toBeDefined() + if (!copiedTab) throw new Error('expected copied tab') + + const copiedLayout = store.getState().panes.layouts[copiedTab.id] as any + expect(copiedLayout.content).toMatchObject({ + kind: 'agent-chat', + provider: 'freshclaude', + resumeSessionId: undefined, + sessionRef: { + provider: 'claude', + sessionId: '00000000-0000-4000-8000-000000000444', + serverInstanceId: 'srv-remote', + }, + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + permissionMode: 'plan', + effort: 'turbo', + plugins: ['planner'], + }) + }) + it('shows context menu on right-click with appropriate items', () => { const store = createStore() render( diff --git a/test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx b/test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx index b72001cc..322429a7 100644 --- a/test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx +++ b/test/unit/client/components/agent-chat/AgentChatSettings.mobile.test.tsx @@ -3,21 +3,34 @@ import { render, screen, fireEvent, cleanup } from '@testing-library/react' const useKeyboardInsetMock = vi.hoisted(() => vi.fn(() => 0)) vi.mock('@/hooks/useKeyboardInset', () => ({ useKeyboardInset: useKeyboardInsetMock })) +import AgentChatSettings from '@/components/agent-chat/AgentChatSettings' +import { AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE } from '@/lib/agent-chat-capabilities' vi.mock('lucide-react', () => ({ Settings: (props: any) => , })) -import AgentChatSettings from '@/components/agent-chat/AgentChatSettings' - describe('AgentChatSettings mobile layout', () => { const defaults = { - model: 'claude-opus-4-6', + model: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, permissionMode: 'default', - effort: 'high', + effort: '', showThinking: true, showTools: true, showTimecodes: false, + modelOptions: [ + { + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }, + { + value: 'opus[1m]', + label: 'Opus 1M', + description: 'Long context window.', + }, + ], + effortOptions: ['turbo'], } afterEach(() => { @@ -34,7 +47,7 @@ describe('AgentChatSettings mobile layout', () => { sessionStarted={false} defaultOpen={true} onChange={vi.fn()} - /> + />, ) const dialog = screen.getByRole('dialog', { name: 'Agent chat settings' }) @@ -43,6 +56,27 @@ describe('AgentChatSettings mobile layout', () => { expect(screen.getByRole('button', { name: 'Done' })).toBeInTheDocument() }) + it('shows capability load failures and retry controls on mobile', () => { + ;(globalThis as any).setMobileForTest(true) + const onRetryCapabilities = vi.fn() + + render( + , + ) + + expect(screen.getByRole('alert')).toHaveTextContent('Capability request failed') + fireEvent.click(screen.getByRole('button', { name: 'Retry model load' })) + expect(onRetryCapabilities).toHaveBeenCalledTimes(1) + }) + it('closes when backdrop is pressed on mobile', () => { ;(globalThis as any).setMobileForTest(true) @@ -52,7 +86,7 @@ describe('AgentChatSettings mobile layout', () => { sessionStarted={false} defaultOpen={true} onChange={vi.fn()} - /> + />, ) fireEvent.click(screen.getByRole('button', { name: 'Close settings' })) diff --git a/test/unit/client/components/agent-chat/AgentChatSettings.test.tsx b/test/unit/client/components/agent-chat/AgentChatSettings.test.tsx index 6c5cc665..5bffc6b9 100644 --- a/test/unit/client/components/agent-chat/AgentChatSettings.test.tsx +++ b/test/unit/client/components/agent-chat/AgentChatSettings.test.tsx @@ -1,22 +1,43 @@ import { describe, it, expect, vi, afterEach } from 'vitest' import { render, screen, cleanup, fireEvent } from '@testing-library/react' + import AgentChatSettings from '@/components/agent-chat/AgentChatSettings' +import { AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE } from '@/lib/agent-chat-capabilities' -// Mock lucide-react vi.mock('lucide-react', () => ({ Settings: (props: any) => , })) +const MODEL_OPTIONS = [ + { + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }, + { + value: 'opus[1m]', + label: 'Opus 1M', + description: 'Long context window.', + }, + { + value: 'haiku', + label: 'Haiku', + description: 'Fast path.', + }, +] + describe('AgentChatSettings', () => { afterEach(cleanup) const defaults = { - model: 'claude-opus-4-6', - permissionMode: 'dangerouslySkipPermissions', - effort: 'high', + model: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + permissionMode: 'default', + effort: '', showThinking: true, showTools: true, showTimecodes: false, + modelOptions: MODEL_OPTIONS, + effortOptions: ['turbo', 'warp'], } it('renders the settings gear button', () => { @@ -25,99 +46,215 @@ describe('AgentChatSettings', () => { {...defaults} sessionStarted={false} onChange={vi.fn()} - /> + />, ) + expect(screen.getByRole('button', { name: /settings/i })).toBeInTheDocument() }) - it('opens popover when gear button is clicked', () => { + it('renders provider-default plus live capability rows only', () => { render( + />, ) - fireEvent.click(screen.getByRole('button', { name: /settings/i })) - expect(screen.getByText('Model')).toBeInTheDocument() - expect(screen.getByText('Permissions')).toBeInTheDocument() + + const modelSelect = screen.getByLabelText('Model') + const labels = Array.from(modelSelect.querySelectorAll('option')).map((option) => option.textContent) + + expect(labels).toEqual([ + 'Provider default (track latest Opus)', + 'Opus 1M', + 'Haiku', + ]) + expect(screen.getByText('Tracks latest Opus automatically.')).toBeInTheDocument() + expect(screen.queryByText('Opus 4.6')).not.toBeInTheDocument() + expect(screen.queryByText('Sonnet 4.6')).not.toBeInTheDocument() }) - it('closes popover on Escape key', () => { + it('keeps an unavailable exact model visible and selected until the user changes it', () => { render( + />, ) - expect(screen.getByText('Model')).toBeInTheDocument() - fireEvent.keyDown(document, { key: 'Escape' }) - expect(screen.queryByText('Model')).not.toBeInTheDocument() + + const modelSelect = screen.getByLabelText('Model') as HTMLSelectElement + expect(modelSelect.value).toBe('claude-opus-4-6') + expect(screen.getByRole('option', { name: 'claude-opus-4-6 (Unavailable)' })).toBeInTheDocument() + expect(screen.getByText('Saved legacy model is no longer available.')).toBeInTheDocument() }) - it('closes popover on click outside', () => { + it('renders effort choices from the selected capability payload only', () => { render( -
- - -
+ , ) - expect(screen.getByText('Model')).toBeInTheDocument() - fireEvent.mouseDown(screen.getByTestId('outside')) - expect(screen.queryByText('Model')).not.toBeInTheDocument() + + const effortSelect = screen.getByLabelText('Effort') + const labels = Array.from(effortSelect.querySelectorAll('option')).map((option) => option.textContent) + + expect(labels).toEqual(['Model default', 'turbo', 'warp']) + expect(screen.queryByRole('option', { name: 'Low' })).not.toBeInTheDocument() + expect(screen.queryByRole('option', { name: 'High' })).not.toBeInTheDocument() + }) + + it('hides the effort selector when the selected model does not support effort', () => { + render( + , + ) + + expect(screen.queryByLabelText('Effort')).not.toBeInTheDocument() + expect(screen.getByText('This model uses its own default effort behavior.')).toBeInTheDocument() + }) + + it('shows an explicit loading state while capabilities are loading', () => { + render( + , + ) + + expect(screen.getByRole('status')).toHaveTextContent('Loading available models...') + }) + + it('shows a retryable error state when capability loading fails', () => { + const onRetryCapabilities = vi.fn() + + render( + , + ) + + expect(screen.getByRole('alert')).toHaveTextContent('Capability request failed') + fireEvent.click(screen.getByRole('button', { name: 'Retry model load' })) + expect(onRetryCapabilities).toHaveBeenCalledTimes(1) }) - it('allows model and permission changes mid-session, disables effort', () => { + it('hides stale capability-driven controls when capability loading fails after a prior success', () => { + render( + , + ) + + expect(screen.getByRole('alert')).toHaveTextContent('Capability request failed') + expect(screen.queryByLabelText('Model')).not.toBeInTheDocument() + expect(screen.queryByLabelText('Effort')).not.toBeInTheDocument() + expect(screen.queryByText('Opus 1M')).not.toBeInTheDocument() + expect(screen.queryByText('This model uses its own default effort behavior.')).not.toBeInTheDocument() + expect(screen.getByLabelText('Permissions')).toBeInTheDocument() + }) + + it('allows model and permission changes mid-session while keeping effort read-only', () => { render( + />, ) - const modelSelect = screen.getByLabelText('Model') - expect(modelSelect).not.toBeDisabled() - const permSelect = screen.getByLabelText('Permissions') - expect(permSelect).not.toBeDisabled() - const effortSelect = screen.getByLabelText('Effort') - expect(effortSelect).toBeDisabled() + + expect(screen.getByLabelText('Model')).not.toBeDisabled() + expect(screen.getByLabelText('Permissions')).not.toBeDisabled() + expect(screen.getByLabelText('Effort')).toBeDisabled() }) it('calls onChange when a display toggle is changed', () => { const onChange = vi.fn() + render( + />, ) + fireEvent.click(screen.getByRole('switch', { name: /show timecodes/i })) expect(onChange).toHaveBeenCalledWith({ showTimecodes: true }) }) it('calls onChange when model is changed', () => { const onChange = vi.fn() + render( + />, ) - fireEvent.change(screen.getByLabelText('Model'), { target: { value: 'claude-sonnet-4-5-20250929' } }) - expect(onChange).toHaveBeenCalledWith({ model: 'claude-sonnet-4-5-20250929' }) + + fireEvent.change(screen.getByLabelText('Model'), { target: { value: 'opus[1m]' } }) + expect(onChange).toHaveBeenCalledWith({ model: 'opus[1m]' }) + }) + + it('calls onChange when provider-default is selected', () => { + const onChange = vi.fn() + + render( + , + ) + + fireEvent.change(screen.getByLabelText('Model'), { + target: { value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE }, + }) + expect(onChange).toHaveBeenCalledWith({ model: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE }) }) it('opens automatically when defaultOpen is true', () => { @@ -127,13 +264,29 @@ describe('AgentChatSettings', () => { sessionStarted={false} defaultOpen={true} onChange={vi.fn()} - /> + />, ) + expect(screen.getByText('Model')).toBeInTheDocument() }) + it('closes popover on Escape key', () => { + render( + , + ) + + fireEvent.keyDown(document, { key: 'Escape' }) + expect(screen.queryByText('Model')).not.toBeInTheDocument() + }) + it('calls onDismiss when closed', () => { const onDismiss = vi.fn() + render( { defaultOpen={true} onChange={vi.fn()} onDismiss={onDismiss} - /> + />, ) + fireEvent.click(screen.getByRole('button', { name: /settings/i })) expect(onDismiss).toHaveBeenCalled() }) - - describe('model display names', () => { - it('shows human-readable names for dynamic model options with raw IDs', () => { - render( - - ) - const modelSelect = screen.getByLabelText('Model') - const options = modelSelect.querySelectorAll('option') - const labels = Array.from(options).map((o) => o.textContent) - // All hardcoded entries present with human-readable names - expect(labels).toContain('Opus 4.6') - expect(labels).toContain('Sonnet 4.6') - expect(labels).toContain('Sonnet 4.5') - expect(labels).toContain('Haiku 4.5') - expect(labels).toContain('Opus 4.5') - }) - - it('deduplicates SDK models whose normalized label matches a hardcoded entry', () => { - render( - - ) - const modelSelect = screen.getByLabelText('Model') - const options = modelSelect.querySelectorAll('option') - const labels = Array.from(options).map((o) => o.textContent) - // "Sonnet 4.5" should appear exactly once - expect(labels.filter((l) => l === 'Sonnet 4.5')).toHaveLength(1) - // And its value should be the SDK's newer ID - const sonnet45 = Array.from(options).find((o) => o.textContent === 'Sonnet 4.5') - expect(sonnet45?.getAttribute('value')).toBe('claude-sonnet-4-5-20251101') - }) - - it('picks the latest dated ID when SDK returns multiple candidates for the same label', () => { - render( - - ) - const modelSelect = screen.getByLabelText('Model') - const options = modelSelect.querySelectorAll('option') - const labels = Array.from(options).map((o) => o.textContent) - expect(labels.filter((l) => l === 'Sonnet 4.5')).toHaveLength(1) - const sonnet45 = Array.from(options).find((o) => o.textContent === 'Sonnet 4.5') - expect(sonnet45?.getAttribute('value')).toBe('claude-sonnet-4-5-20251101') - }) - - it('preserves already-formatted display names from SDK', () => { - render( - - ) - const modelSelect = screen.getByLabelText('Model') - const options = modelSelect.querySelectorAll('option') - const labels = Array.from(options).map((o) => o.textContent) - expect(labels).toContain('Opus 4.6') - expect(labels).toContain('Sonnet 4.5') - // No duplicate labels - expect(new Set(labels).size).toBe(labels.length) - }) - }) }) diff --git a/test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx b/test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx index 97cc2b1e..e533d519 100644 --- a/test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx +++ b/test/unit/client/components/agent-chat/AgentChatView.behavior.test.tsx @@ -1,9 +1,10 @@ import { describe, it, expect, vi, afterEach, beforeAll } from 'vitest' -import { render, screen, cleanup, within, act, fireEvent } from '@testing-library/react' +import { render, screen, cleanup, within, act, fireEvent, waitFor } from '@testing-library/react' import { configureStore } from '@reduxjs/toolkit' import { Provider } from 'react-redux' import AgentChatView from '@/components/agent-chat/AgentChatView' import agentChatReducer, { + capabilityFetchSucceeded, sessionCreated, addUserMessage, addAssistantMessage, @@ -21,9 +22,11 @@ beforeAll(() => { Element.prototype.scrollIntoView = vi.fn() }) +const wsSendSpy = vi.hoisted(() => vi.fn()) + vi.mock('@/lib/ws-client', () => ({ getWsClient: () => ({ - send: vi.fn(), + send: wsSendSpy, onReconnect: vi.fn(() => vi.fn()), }), })) @@ -58,6 +61,7 @@ function makeStore(settingsOverrides?: Record) { } afterEach(() => { + wsSendSpy.mockClear() saveServerSettingsPatchSpy.mockClear() }) @@ -465,6 +469,31 @@ describe('AgentChatView settings auto-open (#110)', () => { it('persists provider defaults through saveServerSettingsPatch when settings change', () => { const store = makeStore() store.dispatch(sessionCreated({ requestId: 'req-1', sessionId: 'sess-1' })) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['low', 'medium', 'high'], + supportsAdaptiveThinking: true, + }, + { + id: 'claude-sonnet-4-6', + displayName: 'Claude Sonnet 4.6', + description: 'Balanced path', + supportsEffort: true, + supportedEffortLevels: ['medium', 'high'], + supportsAdaptiveThinking: true, + }, + ], + }, + })) render( @@ -473,19 +502,422 @@ describe('AgentChatView settings auto-open (#110)', () => { ) const dialog = screen.getByRole('dialog', { name: 'Agent chat settings' }) - fireEvent.change(within(dialog).getByLabelText('Model'), { target: { value: 'claude-sonnet-4-6' } }) + const modelSelect = within(dialog).getByLabelText('Model') as HTMLSelectElement + const sonnetValue = Array.from(modelSelect.options).find( + (option) => option.text === 'Claude Sonnet 4.6', + )?.value + fireEvent.change(modelSelect, { target: { value: sonnetValue } }) fireEvent.change(within(dialog).getByLabelText('Permissions'), { target: { value: 'default' } }) fireEvent.change(within(dialog).getByLabelText('Effort'), { target: { value: 'medium' } }) + expect(wsSendSpy).toHaveBeenCalledWith({ + type: 'sdk.set-model', + sessionId: 'sess-1', + model: 'claude-sonnet-4-6', + }) expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(1, { - agentChat: { providers: { freshclaude: { defaultModel: 'claude-sonnet-4-6' } } }, + agentChat: { providers: { freshclaude: { modelSelection: { kind: 'tracked', modelId: 'claude-sonnet-4-6' } } } }, }) expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(2, { agentChat: { providers: { freshclaude: { defaultPermissionMode: 'default' } } }, }) expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(3, { - agentChat: { providers: { freshclaude: { defaultEffort: 'medium' } } }, + agentChat: { providers: { freshclaude: { effort: 'medium' } } }, + }) + }) + + it('clears unsupported effort overrides from pane state and persisted defaults when switching to a model that does not support them', async () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(sessionCreated({ requestId: 'req-1', sessionId: 'sess-1' })) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'claude-haiku-4-5-20251001', + displayName: 'Haiku 4.5', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + })) + + render( + + + , + ) + + const dialog = screen.getByRole('dialog', { name: 'Agent chat settings' }) + const modelSelect = within(dialog).getByLabelText('Model') as HTMLSelectElement + const haikuValue = Array.from(modelSelect.options).find( + (option) => option.text === 'Haiku 4.5', + )?.value + fireEvent.change(modelSelect, { target: { value: haikuValue } }) + + await waitFor(() => { + expect(wsSendSpy).toHaveBeenCalledWith({ + type: 'sdk.set-model', + sessionId: 'sess-1', + model: 'claude-haiku-4-5-20251001', + }) + }) + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledTimes(2) + }) + expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(1, { + agentChat: { providers: { freshclaude: { modelSelection: { kind: 'tracked', modelId: 'claude-haiku-4-5-20251001' } } } }, }) + expect(saveServerSettingsPatchSpy).toHaveBeenNthCalledWith(2, { + agentChat: { providers: { freshclaude: { effort: undefined } } }, + }) + }) + + it('does not clear persisted defaults when a stale pane snapshot switches to a model that does not support its local effort', async () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(sessionCreated({ requestId: 'req-1', sessionId: 'sess-1' })) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + })) + + render( + + + , + ) + + const dialog = screen.getByRole('dialog', { name: 'Agent chat settings' }) + const modelSelect = within(dialog).getByLabelText('Model') as HTMLSelectElement + const haikuValue = Array.from(modelSelect.options).find( + (option) => option.text === 'Haiku', + )?.value + fireEvent.change(modelSelect, { target: { value: haikuValue } }) + + await waitFor(() => { + expect(wsSendSpy).toHaveBeenCalledWith({ + type: 'sdk.set-model', + sessionId: 'sess-1', + model: 'haiku', + }) + }) + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledTimes(1) + }) + expect(saveServerSettingsPatchSpy).toHaveBeenCalledWith({ + agentChat: { providers: { freshclaude: { modelSelection: { kind: 'tracked', modelId: 'haiku' } } } }, + }) + }) + + it('clears persisted provider defaults when create-time cleanup drops an unsupported provider-default effort', async () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: true, + }, + ], + }, + })) + + render( + + + , + ) + + await waitFor(() => { + expect(wsSendSpy).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-create-default-effort', + model: 'opus', + })) + }) + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledTimes(1) + }) + expect(saveServerSettingsPatchSpy).toHaveBeenCalledWith({ + agentChat: { providers: { freshclaude: { effort: undefined } } }, + }) + }) + + it('does not rewrite provider defaults when create-time cleanup drops an unsupported pane-local effort', async () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + })) + + render( + + + , + ) + + await waitFor(() => { + expect(wsSendSpy).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-create-unsupported-effort', + model: 'haiku', + })) + }) + expect(saveServerSettingsPatchSpy).not.toHaveBeenCalled() + }) + + it('clears persisted defaults when passive cleanup drops an unsupported provider-default effort', async () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(sessionCreated({ requestId: 'req-1', sessionId: 'sess-1' })) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: true, + }, + ], + }, + })) + + render( + + + , + ) + + await waitFor(() => { + expect(saveServerSettingsPatchSpy).toHaveBeenCalledTimes(1) + }) + expect(saveServerSettingsPatchSpy).toHaveBeenCalledWith({ + agentChat: { providers: { freshclaude: { effort: undefined } } }, + }) + }) + + it('does not rewrite provider defaults when passive cleanup drops an unsupported pane-local effort', () => { + const store = makeStore({ + agentChat: { + ...defaultSettings.agentChat, + providers: { + freshclaude: { + effort: 'turbo', + }, + }, + }, + }) + store.dispatch(sessionCreated({ requestId: 'req-1', sessionId: 'sess-1' })) + store.dispatch(capabilityFetchSucceeded({ + provider: 'freshclaude', + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + })) + + render( + + + , + ) + + expect(saveServerSettingsPatchSpy).not.toHaveBeenCalled() }) it('persists initial setup completion through saveServerSettingsPatch when settings are dismissed', () => { diff --git a/test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx b/test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx index 74c1ab0f..7d717693 100644 --- a/test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx +++ b/test/unit/client/components/agent-chat/AgentChatView.reload.test.tsx @@ -33,7 +33,12 @@ beforeAll(() => { const wsSend = vi.fn() const getAgentTimelinePage = vi.fn() const getAgentTurnBody = vi.fn() +const getAgentChatCapabilities = vi.fn() const setSessionMetadata = vi.fn(() => Promise.resolve(undefined)) +const saveServerSettingsPatchSpy = vi.hoisted(() => vi.fn((patch: unknown) => ({ + type: 'settings/saveServerSettingsPatch', + payload: patch, +}))) vi.mock('@/lib/ws-client', () => ({ getWsClient: () => ({ @@ -42,12 +47,17 @@ vi.mock('@/lib/ws-client', () => ({ }), })) +vi.mock('@/store/settingsThunks', () => ({ + saveServerSettingsPatch: (patch: unknown) => saveServerSettingsPatchSpy(patch), +})) + vi.mock('@/lib/api', async () => { const actual = await vi.importActual('@/lib/api') return { ...actual, getAgentTimelinePage: (...args: unknown[]) => getAgentTimelinePage(...args), getAgentTurnBody: (...args: unknown[]) => getAgentTurnBody(...args), + getAgentChatCapabilities: (...args: unknown[]) => getAgentChatCapabilities(...args), setSessionMetadata: (...args: unknown[]) => setSessionMetadata(...args), } }) @@ -145,6 +155,7 @@ describe('AgentChatView reload/restore behavior', () => { localStorage.clear() getAgentTimelinePage.mockReset() getAgentTurnBody.mockReset() + getAgentChatCapabilities.mockReset() setSessionMetadata.mockReset() setSessionMetadata.mockResolvedValue(undefined) }) @@ -152,6 +163,7 @@ describe('AgentChatView reload/restore behavior', () => { afterEach(() => { cleanup() wsSend.mockClear() + saveServerSettingsPatchSpy.mockClear() localStorage.clear() delete window.__FRESHELL_TEST_HARNESS__ }) @@ -329,6 +341,237 @@ describe('AgentChatView reload/restore behavior', () => { expect((retriedContent as AgentChatPaneContent).createError).toBeUndefined() }) + it('sends provider-default creates as the stable opus track alias', async () => { + const store = makeStore() + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-provider-default', + status: 'creating', + } + + store.dispatch(initLayout({ tabId: 't1', content: pane, paneId: 'p1' })) + + function Wrapper() { + const root = useSelector((s: ReturnType) => s.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-provider-default', + model: 'opus', + })) + }) + const createCall = wsSend.mock.calls.find((call) => call[0]?.type === 'sdk.create')?.[0] + expect(createCall).not.toHaveProperty('effort') + }) + + it('creates tracked live models directly without a capability fetch when no effort validation is needed', async () => { + const store = makeStore() + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-tracked', + status: 'creating', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + } + + store.dispatch(initLayout({ tabId: 't1', content: pane, paneId: 'p1' })) + + function Wrapper() { + const root = useSelector((s: ReturnType) => s.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-tracked', + model: 'opus[1m]', + })) + }) + expect(getAgentChatCapabilities).not.toHaveBeenCalled() + }) + + it('validates explicit effort overrides before create and clears them when unsupported', async () => { + getAgentChatCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + + const store = makeStore() + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-unsupported-effort', + status: 'creating', + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + effort: 'turbo', + } + + store.dispatch(initLayout({ tabId: 't1', content: pane, paneId: 'p1' })) + + function Wrapper() { + const root = useSelector((s: ReturnType) => s.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + await waitFor(() => { + expect(getAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-unsupported-effort', + model: 'haiku', + })) + }) + const createCall = wsSend.mock.calls.find((call) => call[0]?.type === 'sdk.create')?.[0] + expect(createCall).not.toHaveProperty('effort') + expect(getPaneContent(store, 't1', 'p1')?.effort).toBeUndefined() + }) + + it('passes named resume tokens through sdk.create and keeps the session in restore mode until it upgrades', async () => { + const store = makeStore() + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-named-resume', + status: 'creating', + resumeSessionId: 'named-resume', + } + + store.dispatch(initLayout({ tabId: 't1', content: pane, paneId: 'p1' })) + + function Wrapper() { + const root = useSelector((s: ReturnType) => s.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + await waitFor(() => { + expect(wsSend).toHaveBeenCalledWith(expect.objectContaining({ + type: 'sdk.create', + requestId: 'req-named-resume', + model: 'opus', + resumeSessionId: 'named-resume', + })) + }) + + expect(store.getState().agentChat.pendingCreates['req-named-resume']).toEqual({ + sessionId: undefined, + expectsHistoryHydration: true, + }) + }) + + it('blocks create when an exact unavailable selection cannot be launched safely', async () => { + getAgentChatCapabilities.mockResolvedValue({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: Date.now(), + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + }, + }) + + const store = makeStore() + const pane: AgentChatPaneContent = { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-unavailable-exact', + status: 'creating', + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + } + + store.dispatch(initLayout({ tabId: 't1', content: pane, paneId: 'p1' })) + + function Wrapper() { + const root = useSelector((s: ReturnType) => s.panes.layouts.t1) + const content = root?.type === 'leaf' && root.content.kind === 'agent-chat' + ? root.content + : undefined + if (!content) return null + return + } + + render( + + + , + ) + + expect(await screen.findByText('Session start failed')).toBeInTheDocument() + expect(screen.getByText(/no longer available/i)).toBeInTheDocument() + expect(wsSend.mock.calls.filter((call) => call[0]?.type === 'sdk.create')).toHaveLength(0) + expect(getPaneContent(store, 't1', 'p1')).toEqual(expect.objectContaining({ + status: 'create-failed', + createError: expect.objectContaining({ + code: 'MODEL_UNAVAILABLE', + }), + })) + }) + it('shows loading state instead of welcome screen when sessionId is set but messages have not arrived', () => { const store = makeStore() render( diff --git a/test/unit/client/components/panes/PaneContainer.createContent.test.tsx b/test/unit/client/components/panes/PaneContainer.createContent.test.tsx index 5dd1cdcb..21482902 100644 --- a/test/unit/client/components/panes/PaneContainer.createContent.test.tsx +++ b/test/unit/client/components/panes/PaneContainer.createContent.test.tsx @@ -316,9 +316,9 @@ describe('createContentForType with ext: prefix', () => { defaultPlugins: ['planner', 'sandbox'], providers: { freshclaude: { - defaultModel: 'claude-sonnet-4-6', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, defaultPermissionMode: 'default', - defaultEffort: 'medium', + effort: 'turbo', }, }, }, @@ -353,9 +353,9 @@ describe('createContentForType with ext: prefix', () => { if (paneContent.kind === 'agent-chat') { expect(paneContent.provider).toBe('freshclaude') expect(paneContent.plugins).toEqual(['planner', 'sandbox']) - expect(paneContent.model).toBe('claude-sonnet-4-6') + expect(paneContent.modelSelection).toEqual({ kind: 'tracked', modelId: 'opus[1m]' }) expect(paneContent.permissionMode).toBe('default') - expect(paneContent.effort).toBe('medium') + expect(paneContent.effort).toBe('turbo') } }) }) diff --git a/test/unit/client/lib/agent-chat-capabilities.test.ts b/test/unit/client/lib/agent-chat-capabilities.test.ts new file mode 100644 index 00000000..d4ce47ae --- /dev/null +++ b/test/unit/client/lib/agent-chat-capabilities.test.ts @@ -0,0 +1,287 @@ +import { describe, expect, it } from 'vitest' + +import { + AGENT_CHAT_CAPABILITY_CACHE_TTL_MS, + AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + getAgentChatSettingsModelOptions, + getAgentChatSettingsModelValue, + getAgentChatSupportedEffortLevels, + isAgentChatEffortSupported, + isAgentChatCapabilitiesFresh, + parseAgentChatSettingsModelValue, + requiresAgentChatCapabilityValidation, + resolveAgentChatModelSelection, +} from '@/lib/agent-chat-capabilities' + +const capabilities = { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'opus[1m]', + displayName: 'Opus 1M', + description: 'Long context', + supportsEffort: true, + supportedEffortLevels: ['warp'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: 'Fast path', + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], +} as const + +describe('agent-chat-capabilities helpers', () => { + it('resolves provider-default to the stable opus track alias', () => { + const resolved = resolveAgentChatModelSelection({ + providerDefaultModelId: 'opus', + capabilities, + }) + + expect(resolved).toMatchObject({ + source: 'provider-default', + resolvedModelId: 'opus', + capability: expect.objectContaining({ id: 'opus' }), + }) + }) + + it('resolves tracked aliases without local remapping', () => { + const resolved = resolveAgentChatModelSelection({ + providerDefaultModelId: 'opus', + capabilities, + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + }) + + expect(resolved).toMatchObject({ + source: 'tracked', + resolvedModelId: 'opus[1m]', + capability: expect.objectContaining({ id: 'opus[1m]' }), + }) + }) + + it('surfaces unavailable exact selections instead of silently healing them', () => { + const resolved = resolveAgentChatModelSelection({ + providerDefaultModelId: 'opus', + capabilities, + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }) + + expect(resolved).toMatchObject({ + source: 'exact', + resolvedModelId: undefined, + unavailableExactSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }) + }) + + it('derives effort options only from the resolved capability payload', () => { + expect(getAgentChatSupportedEffortLevels({ + providerDefaultModelId: 'opus', + capabilities, + })).toEqual(['turbo', 'warp']) + + expect(getAgentChatSupportedEffortLevels({ + providerDefaultModelId: 'opus', + capabilities, + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + })).toEqual([]) + }) + + it('treats supportedEffortLevels as the effort support source of truth', () => { + const inconsistentCapabilities = { + provider: 'freshclaude', + fetchedAt: 2_345, + models: [ + { + id: 'opus', + displayName: 'Opus', + supportsEffort: false, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + ], + } as const + + const resolved = resolveAgentChatModelSelection({ + providerDefaultModelId: 'opus', + capabilities: inconsistentCapabilities, + }) + + expect(getAgentChatSupportedEffortLevels({ + providerDefaultModelId: 'opus', + capabilities: inconsistentCapabilities, + })).toEqual(['turbo']) + expect(isAgentChatEffortSupported(resolved.capability, 'turbo')).toBe(true) + }) + + it('builds settings options from provider-default, live capabilities, and unavailable exact selections', () => { + expect(getAgentChatSettingsModelOptions({ + providerDefaultModelId: 'opus', + capabilities, + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + })).toEqual([ + { + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'opus' }), + label: 'Opus', + description: 'Latest Opus track', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'opus[1m]' }), + label: 'Opus 1M', + description: 'Long context', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'haiku' }), + label: 'Haiku', + description: 'Fast path', + }, + { + value: getAgentChatSettingsModelValue( + { kind: 'exact', modelId: 'claude-opus-4-6' }, + capabilities, + ), + label: 'claude-opus-4-6 (Unavailable)', + description: 'Saved legacy model is no longer available.', + unavailable: true, + }, + ]) + }) + + it('keeps a persisted tracked selection represented when the refreshed catalog drops it', () => { + expect(getAgentChatSettingsModelOptions({ + providerDefaultModelId: 'opus', + capabilities: { + ...capabilities, + models: capabilities.models.filter((model) => model.id !== 'haiku'), + }, + modelSelection: { kind: 'tracked', modelId: 'haiku' }, + })).toEqual([ + { + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'opus' }), + label: 'Opus', + description: 'Latest Opus track', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'opus[1m]' }), + label: 'Opus 1M', + description: 'Long context', + }, + { + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'haiku' }), + label: 'haiku (Saved selection)', + description: 'Saved tracked model is not in the latest capability catalog.', + }, + ]) + }) + + it('maps provider-default and tracked settings values back into selection strategies', () => { + expect(getAgentChatSettingsModelValue(undefined)).toBe(AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE) + expect(parseAgentChatSettingsModelValue(AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE)).toBeUndefined() + expect(parseAgentChatSettingsModelValue( + getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'opus[1m]' }), + )).toEqual({ + kind: 'tracked', + modelId: 'opus[1m]', + }) + }) + + it('treats raw magic-string lookalikes as opaque tracked ids', () => { + expect(parseAgentChatSettingsModelValue('__provider_default__')).toEqual({ + kind: 'tracked', + modelId: '__provider_default__', + }) + expect(parseAgentChatSettingsModelValue('__exact__:haiku')).toEqual({ + kind: 'tracked', + modelId: '__exact__:haiku', + }) + }) + + it('round-trips unavailable exact settings values without downgrading them to tracked', () => { + const unavailableOption = getAgentChatSettingsModelOptions({ + providerDefaultModelId: 'opus', + capabilities, + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + }).find((option) => option.unavailable) + + expect(unavailableOption).toBeDefined() + const selection = parseAgentChatSettingsModelValue(unavailableOption!.value) + expect(selection).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(requiresAgentChatCapabilityValidation({ modelSelection: selection ?? undefined })).toBe(true) + }) + + it('treats fetchedAt as a bounded freshness window instead of an unused field', () => { + expect(isAgentChatCapabilitiesFresh(capabilities, capabilities.fetchedAt)).toBe(true) + expect( + isAgentChatCapabilitiesFresh( + capabilities, + capabilities.fetchedAt + AGENT_CHAT_CAPABILITY_CACHE_TTL_MS, + ), + ).toBe(true) + expect( + isAgentChatCapabilitiesFresh( + capabilities, + capabilities.fetchedAt + AGENT_CHAT_CAPABILITY_CACHE_TTL_MS + 1, + ), + ).toBe(false) + }) + + it('builds a large capability catalog without catastrophic option-building regressions', () => { + const largeCatalog = { + provider: 'freshclaude', + fetchedAt: 9_999, + models: Array.from({ length: 2_000 }, (_, index) => ({ + id: `model-${index}`, + displayName: `Model ${index}`, + description: `Synthetic model ${index}`, + supportsEffort: index % 2 === 0, + supportedEffortLevels: index % 2 === 0 ? ['turbo', 'warp', `custom-${index % 5}`] : [], + supportsAdaptiveThinking: index % 3 === 0, + })), + } as const + + const start = performance.now() + const options = getAgentChatSettingsModelOptions({ + providerDefaultModelId: 'opus', + capabilities: largeCatalog, + }) + const durationMs = performance.now() - start + + expect(options).toHaveLength(2_001) + expect(options[0]).toEqual({ + value: AGENT_CHAT_PROVIDER_DEFAULT_OPTION_VALUE, + label: 'Provider default (track latest Opus)', + description: 'Tracks latest Opus automatically.', + }) + expect(options.at(-1)).toEqual({ + value: getAgentChatSettingsModelValue({ kind: 'tracked', modelId: 'model-1999' }), + label: 'Model 1999', + description: 'Synthetic model 1999', + }) + expect(durationMs).toBeLessThan(1_000) + }) +}) diff --git a/test/unit/client/lib/agent-chat-utils.test.ts b/test/unit/client/lib/agent-chat-utils.test.ts index 29a16d5e..66a408bd 100644 --- a/test/unit/client/lib/agent-chat-utils.test.ts +++ b/test/unit/client/lib/agent-chat-utils.test.ts @@ -27,9 +27,9 @@ describe('agent-chat-utils', () => { const config = getAgentChatProviderConfig('freshclaude') expect(config).toBeDefined() expect(config!.label).toBe('Freshclaude') - expect(config!.defaultModel).toBe('claude-opus-4-6') + expect(config!.providerDefaultModelId).toBe('opus') expect(config!.defaultPermissionMode).toBe('bypassPermissions') - expect(config!.defaultEffort).toBe('high') + expect('defaultEffort' in config!).toBe(false) }) it('returns undefined for unknown provider', () => { @@ -54,9 +54,9 @@ describe('agent-chat-utils', () => { expect(config!.name).toBe('kilroy') expect(config!.label).toBe('Kilroy') expect(config!.codingCliProvider).toBe('claude') - expect(config!.defaultModel).toBe('claude-opus-4-6') + expect(config!.providerDefaultModelId).toBe('opus') expect(config!.defaultPermissionMode).toBe('bypassPermissions') - expect(config!.defaultEffort).toBe('high') + expect('defaultEffort' in config!).toBe(false) expect(config!.pickerShortcut).not.toBe('A') // must differ from freshclaude }) diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 857fe75b..e688c32d 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -1,6 +1,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { api, + getAgentChatCapabilities, + refreshAgentChatCapabilities, fetchSidebarSessionsSnapshot, getAgentTimelinePage, getAgentTurnBody, @@ -31,6 +33,15 @@ function mockJson(value: unknown) { } } +function mockJsonResponse(status: number, value: unknown) { + return { + ok: status >= 200 && status < 300, + status, + statusText: status === 503 ? 'Service Unavailable' : 'Error', + text: () => Promise.resolve(JSON.stringify(value)), + } +} + describe('visible-first read-model helpers', () => { beforeEach(() => { mockFetch.mockReset() @@ -128,6 +139,43 @@ describe('visible-first read-model helpers', () => { ) }) + it('preserves typed capability errors from non-2xx capability reads and refreshes', async () => { + mockFetch + .mockResolvedValueOnce(mockJsonResponse(503, { + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'Probe failed upstream', + retryable: true, + }, + })) + .mockResolvedValueOnce(mockJsonResponse(503, { + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload invalid', + retryable: false, + }, + })) + + await expect(getAgentChatCapabilities('freshclaude')).resolves.toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'Probe failed upstream', + retryable: true, + }, + }) + await expect(refreshAgentChatCapabilities('freshclaude')).resolves.toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload invalid', + retryable: false, + }, + }) + }) + it('rejects timeline requests that omit the pinned restore revision', async () => { await expect(getAgentTimelinePage('session-1', { priority: 'visible' }, { signal: new AbortController().signal })) .rejects diff --git a/test/unit/client/lib/session-type-utils.test.ts b/test/unit/client/lib/session-type-utils.test.ts index 0a6ac3ad..ad3ce10b 100644 --- a/test/unit/client/lib/session-type-utils.test.ts +++ b/test/unit/client/lib/session-type-utils.test.ts @@ -45,8 +45,9 @@ describe('buildResumeContent', () => { expect(content.provider).toBe('freshclaude') expect(content.resumeSessionId).toBe('abc-123') expect(content.initialCwd).toBe('/home/user/project') - expect(content.model).toBe('claude-opus-4-6') // default from provider config + expect(content.modelSelection).toBeUndefined() expect(content.permissionMode).toBe('bypassPermissions') // default from provider config + expect(content.effort).toBeUndefined() }) it('returns agent-chat content for kilroy sessionType', () => { @@ -120,26 +121,26 @@ describe('buildResumeContent', () => { sessionType: 'freshclaude', sessionId: 'abc-123', agentChatProviderSettings: { - defaultModel: 'claude-sonnet-4-20250514', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, defaultPermissionMode: 'default', - defaultEffort: 'max', + effort: 'turbo', }, }) expect(content.kind).toBe('agent-chat') if (content.kind !== 'agent-chat') throw new Error('expected agent-chat') - expect(content.model).toBe('claude-sonnet-4-20250514') + expect(content.modelSelection).toEqual({ kind: 'tracked', modelId: 'opus[1m]' }) expect(content.permissionMode).toBe('default') - expect(content.effort).toBe('max') + expect(content.effort).toBe('turbo') }) - it('applies default effort from provider config', () => { + it('does not apply a baked-in provider effort override', () => { const content = buildResumeContent({ sessionType: 'freshclaude', sessionId: 'abc-123', }) expect(content.kind).toBe('agent-chat') if (content.kind !== 'agent-chat') throw new Error('expected agent-chat') - expect(content.effort).toBe('high') // freshclaude default + expect(content.effort).toBeUndefined() }) it('preserves unknown sessionType as mode (was validated at creation)', () => { diff --git a/test/unit/client/lib/tab-registry-snapshot.test.ts b/test/unit/client/lib/tab-registry-snapshot.test.ts index acaf686a..27642209 100644 --- a/test/unit/client/lib/tab-registry-snapshot.test.ts +++ b/test/unit/client/lib/tab-registry-snapshot.test.ts @@ -37,6 +37,46 @@ describe('shouldKeepClosedTab', () => { }) describe('collectPaneSnapshots', () => { + it('serializes agent-chat selection strategies and explicit effort overrides', () => { + const node: PaneNode = { + type: 'leaf', + id: 'pane-agent', + content: { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req-agent', + status: 'idle', + resumeSessionId: '00000000-0000-4000-8000-000000000123', + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + permissionMode: 'default', + effort: 'turbo', + plugins: ['planner'], + }, + } + + const snapshots = collectPaneSnapshots(node, 'server-1') + + expect(snapshots).toEqual([{ + paneId: 'pane-agent', + kind: 'agent-chat', + title: undefined, + payload: { + provider: 'freshclaude', + resumeSessionId: '00000000-0000-4000-8000-000000000123', + sessionRef: { + provider: 'claude', + sessionId: '00000000-0000-4000-8000-000000000123', + serverInstanceId: 'server-1', + }, + initialCwd: undefined, + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + permissionMode: 'default', + effort: 'turbo', + plugins: ['planner'], + }, + }]) + }) + describe('extension content', () => { it('serializes extension pane content with correct kind and payload', () => { const node: PaneNode = { diff --git a/test/unit/client/sdk-message-handler.test.ts b/test/unit/client/sdk-message-handler.test.ts index b542ecbc..5bc25434 100644 --- a/test/unit/client/sdk-message-handler.test.ts +++ b/test/unit/client/sdk-message-handler.test.ts @@ -16,24 +16,17 @@ describe('sdk-message-handler', () => { _resetCancelledCreates() }) - it('dispatches setAvailableModels on sdk.models', () => { - const { dispatch, calls } = createMockDispatch() - const models = [ - { value: 'claude-opus-4-6', displayName: 'Opus 4.6', description: 'Most capable' }, - { value: 'claude-sonnet-4-5-20250929', displayName: 'Sonnet 4.5', description: 'Fast' }, - ] + it('ignores the obsolete sdk.models websocket path', () => { + const { dispatch } = createMockDispatch() const handled = handleSdkMessage(dispatch, { type: 'sdk.models', sessionId: 's1', - models, + models: [{ value: 'opus', displayName: 'Opus' }], }) - expect(handled).toBe(true) - expect(dispatch).toHaveBeenCalledOnce() - const action = calls[0] - expect(action.type).toBe('agentChat/setAvailableModels') - expect(action.payload.models).toEqual(models) + expect(handled).toBe(false) + expect(dispatch).not.toHaveBeenCalled() }) it('dispatches sessionCreated on sdk.created', () => { diff --git a/test/unit/client/store/agentChatThunks.test.ts b/test/unit/client/store/agentChatThunks.test.ts index e79546a8..7f175f8c 100644 --- a/test/unit/client/store/agentChatThunks.test.ts +++ b/test/unit/client/store/agentChatThunks.test.ts @@ -7,13 +7,17 @@ import agentChatReducer, { turnBodyReceived, } from '@/store/agentChatSlice' import { + fetchAgentChatCapabilities, loadAgentTimelineWindow, loadAgentTurnBody, + refreshAgentChatCapabilities, _resetAgentChatThunkControllers, } from '@/store/agentChatThunks' const getAgentTimelinePage = vi.fn() const getAgentTurnBody = vi.fn() +const getAgentChatCapabilities = vi.fn() +const refreshAgentChatCapabilitiesApi = vi.fn() vi.mock('@/lib/api', async () => { const actual = await vi.importActual('@/lib/api') @@ -21,6 +25,8 @@ vi.mock('@/lib/api', async () => { ...actual, getAgentTimelinePage: (...args: unknown[]) => getAgentTimelinePage(...args), getAgentTurnBody: (...args: unknown[]) => getAgentTurnBody(...args), + getAgentChatCapabilities: (...args: unknown[]) => getAgentChatCapabilities(...args), + refreshAgentChatCapabilities: (...args: unknown[]) => refreshAgentChatCapabilitiesApi(...args), } }) @@ -97,9 +103,104 @@ describe('agentChatThunks', () => { beforeEach(() => { getAgentTimelinePage.mockReset() getAgentTurnBody.mockReset() + getAgentChatCapabilities.mockReset() + refreshAgentChatCapabilitiesApi.mockReset() _resetAgentChatThunkControllers() }) + it('fetches capability catalogs into provider-scoped loading and success state', async () => { + let resolveResponse: (value: unknown) => void + getAgentChatCapabilities.mockReturnValue(new Promise((resolve) => { + resolveResponse = resolve + })) + + const store = makeStore() + const promise = store.dispatch(fetchAgentChatCapabilities('freshclaude') as any) + + expect(store.getState().agentChat.capabilitiesByProvider.freshclaude).toEqual({ + status: 'loading', + capabilities: undefined, + error: undefined, + }) + + resolveResponse!({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + }) + + await promise + + expect(getAgentChatCapabilities).toHaveBeenCalledWith('freshclaude', {}) + expect(store.getState().agentChat.capabilitiesByProvider.freshclaude).toEqual({ + status: 'succeeded', + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_234, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Latest Opus track', + supportsEffort: true, + supportedEffortLevels: ['turbo', 'warp'], + supportsAdaptiveThinking: true, + }, + ], + }, + error: undefined, + }) + }) + + it('preserves typed failures from non-2xx capability refresh responses', async () => { + refreshAgentChatCapabilitiesApi.mockRejectedValue({ + status: 503, + message: 'Service Unavailable', + details: { + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload invalid', + retryable: false, + }, + }, + }) + + const store = makeStore() + const response = await store.dispatch(refreshAgentChatCapabilities('kilroy') as any) + + expect(response).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload invalid', + retryable: false, + }, + }) + expect(refreshAgentChatCapabilitiesApi).toHaveBeenCalledWith('kilroy', {}) + expect(store.getState().agentChat.capabilitiesByProvider.kilroy).toEqual({ + status: 'failed', + capabilities: undefined, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload invalid', + retryable: false, + }, + }) + }) + it('loads a visible timeline window and hydrates the most recent turn body', async () => { getAgentTimelinePage.mockResolvedValue({ sessionId: 'sess-1', diff --git a/test/unit/client/store/panesPersistence.test.ts b/test/unit/client/store/panesPersistence.test.ts index f671bb79..94a1bf18 100644 --- a/test/unit/client/store/panesPersistence.test.ts +++ b/test/unit/client/store/panesPersistence.test.ts @@ -742,6 +742,45 @@ describe('version 5 migration (drop claude-chat panes)', () => { const result = loadPersistedPanes() expect(result!.layouts.tab1.content.kind).toBe('terminal') }) + + it('migrates legacy agent-chat model and effort fields into selection strategies', () => { + localStorage.setItem('freshell.layout.v3', JSON.stringify({ + version: 3, + tabs: { tabs: [{ id: 'tab1', title: 'Tab 1' }], activeTabId: 'tab1' }, + panes: { + version: 6, + layouts: { + tab1: { + type: 'leaf', + id: 'pane1', + content: { + kind: 'agent-chat', + provider: 'freshclaude', + createRequestId: 'req1', + status: 'idle', + model: 'claude-opus-4-6', + effort: 'high', + }, + }, + }, + activePane: { tab1: 'pane1' }, + paneTitles: {}, + paneTitleSetByUser: {}, + }, + tombstones: [], + })) + + const persisted = loadPersistedPanes() + const content = (persisted!.layouts.tab1 as any).content + + expect(persisted!.version).toBe(PANES_SCHEMA_VERSION) + expect(content.model).toBeUndefined() + expect(content.modelSelection).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(content.effort).toBe('high') + }) }) import { BROWSER_PREFERENCES_STORAGE_KEY } from '../../../../src/store/storage-keys' diff --git a/test/unit/client/store/persistedState.test.ts b/test/unit/client/store/persistedState.test.ts index c1e314f0..96f9dd25 100644 --- a/test/unit/client/store/persistedState.test.ts +++ b/test/unit/client/store/persistedState.test.ts @@ -44,6 +44,10 @@ describe('persistedState parsers', () => { }) describe('parsePersistedPanesRaw', () => { + it('bumps the panes schema version for selection-strategy persistence', () => { + expect(PANES_SCHEMA_VERSION).toBe(7) + }) + it('returns null for invalid JSON', () => { expect(parsePersistedPanesRaw('{')).toBeNull() }) diff --git a/test/unit/client/store/settingsThunks.test.ts b/test/unit/client/store/settingsThunks.test.ts index 264b4796..77fe85c6 100644 --- a/test/unit/client/store/settingsThunks.test.ts +++ b/test/unit/client/store/settingsThunks.test.ts @@ -400,4 +400,48 @@ describe('settingsThunks', () => { expect(store.getState().settings.settings.codingCli.providers.codex?.model).toBeUndefined() expect(store.getState().settings.settings.codingCli.providers.codex?.sandbox).toBeUndefined() }) + + it('preserves nested agent-chat clears by converting them into API clear sentinels', async () => { + const store = makeStore() + const initialServerSettings = store.getState().settings.serverSettings + store.dispatch(setServerSettings({ + ...initialServerSettings, + agentChat: { + ...initialServerSettings.agentChat, + providers: { + ...initialServerSettings.agentChat.providers, + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'turbo', + }, + }, + }, + })) + + apiPatch.mockResolvedValue({}) + + await store.dispatch(saveServerSettingsPatch({ + agentChat: { + providers: { + freshclaude: { + modelSelection: undefined, + effort: undefined, + }, + }, + }, + })) + + expect(apiPatch).toHaveBeenCalledWith('/api/settings', { + agentChat: { + providers: { + freshclaude: { + modelSelection: null, + effort: null, + }, + }, + }, + }) + expect(store.getState().settings.settings.agentChat.providers.freshclaude?.modelSelection).toBeUndefined() + expect(store.getState().settings.settings.agentChat.providers.freshclaude?.effort).toBeUndefined() + }) }) diff --git a/test/unit/server/agent-chat-capability-registry.supportedModels.probe.ts b/test/unit/server/agent-chat-capability-registry.supportedModels.probe.ts new file mode 100644 index 00000000..91b15919 --- /dev/null +++ b/test/unit/server/agent-chat-capability-registry.supportedModels.probe.ts @@ -0,0 +1,80 @@ +import { query } from '@anthropic-ai/claude-agent-sdk' +import { pathToFileURL } from 'node:url' + +import { AgentChatCapabilitiesSchema } from '../../../shared/agent-chat-capabilities.js' +import { + normalizeAgentChatCapabilityCatalog, +} from '../../../server/agent-chat-capability-registry.js' +import { createClaudeSdkOptions } from '../../../server/sdk-bridge.js' + +const PROBE_ENV = 'FRESHELL_RUN_LIVE_SUPPORTED_MODELS_PROBE' +const PROBE_TIMEOUT_MS = 10_000 + +export async function runLiveSupportedModelsProbe() { + const abortController = new AbortController() + const probeQuery = query({ + prompt: (async function* emptyPrompt() {})(), + options: createClaudeSdkOptions({ abortController }), + }) + let timeoutId: ReturnType | undefined + + try { + const rawModels = await Promise.race([ + Promise.resolve(probeQuery.supportedModels()), + new Promise((_, reject) => { + timeoutId = setTimeout(() => { + abortController.abort() + reject(new Error(`supportedModels() probe timed out after ${PROBE_TIMEOUT_MS}ms`)) + }, PROBE_TIMEOUT_MS) + }), + ]) + + const normalizedModels = normalizeAgentChatCapabilityCatalog(rawModels) + const parsed = AgentChatCapabilitiesSchema.parse({ + provider: 'freshclaude', + fetchedAt: Date.now(), + models: normalizedModels, + }) + + return { + rawModels, + normalizedModels, + parsed, + } + } finally { + if (timeoutId !== undefined) { + clearTimeout(timeoutId) + } + await Promise.resolve(probeQuery.close()) + } +} + +async function main() { + if (process.env[PROBE_ENV] !== '1') { + console.log( + `${PROBE_ENV} is not set; skipping live supportedModels() differential probe.`, + ) + return + } + + const result = await runLiveSupportedModelsProbe() + console.log(JSON.stringify({ + rawModels: result.rawModels, + normalizedModels: result.normalizedModels, + parsedModelIds: result.parsed.models.map((model) => model.id), + }, null, 2)) +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + main().catch((error) => { + console.error(JSON.stringify({ + error: error instanceof Error + ? { + message: error.message, + stack: error.stack, + } + : String(error), + }, null, 2)) + process.exitCode = 1 + }) +} diff --git a/test/unit/server/agent-chat-capability-registry.test.ts b/test/unit/server/agent-chat-capability-registry.test.ts new file mode 100644 index 00000000..faed255b --- /dev/null +++ b/test/unit/server/agent-chat-capability-registry.test.ts @@ -0,0 +1,332 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { AgentChatCapabilityRegistry } from '../../../server/agent-chat-capability-registry.js' + +describe('AgentChatCapabilityRegistry', () => { + let now = 1_000 + let closeMock: ReturnType + let supportedModelsMock: ReturnType + let queryFactory: ReturnType + + beforeEach(() => { + now = 1_000 + closeMock = vi.fn().mockResolvedValue(undefined) + supportedModelsMock = vi.fn() + queryFactory = vi.fn(() => ({ + supportedModels: supportedModelsMock, + close: closeMock, + })) + }) + + it('normalizes model capabilities, closes the probe query, and preserves sdk runtime metadata', async () => { + supportedModelsMock.mockResolvedValue([ + { + value: 'opus', + displayName: 'opus', + description: 'Primary track', + supportedEffortLevels: [' low ', 'medium'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + display_name: 'haiku', + supported_effort_levels: [], + supports_adaptive_thinking: false, + }, + ]) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const result = await registry.getCapabilities('freshclaude') + + expect(queryFactory).toHaveBeenCalledTimes(1) + expect(closeMock).toHaveBeenCalledTimes(1) + expect(result).toEqual({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_000, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: 'Primary track', + supportsEffort: true, + supportedEffortLevels: ['low', 'medium'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: undefined, + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + }) + + it('normalizes effort support from supportedEffortLevels so the shared contract stays self-consistent', async () => { + supportedModelsMock.mockResolvedValue([ + { + value: 'opus', + displayName: 'opus', + supportsEffort: false, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + value: 'haiku', + displayName: 'haiku', + supportsEffort: true, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ]) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const result = await registry.getCapabilities('freshclaude') + + expect(result).toEqual({ + ok: true, + capabilities: { + provider: 'freshclaude', + fetchedAt: 1_000, + models: [ + { + id: 'opus', + displayName: 'Opus', + description: undefined, + supportsEffort: true, + supportedEffortLevels: ['turbo'], + supportsAdaptiveThinking: true, + }, + { + id: 'haiku', + displayName: 'Haiku', + description: undefined, + supportsEffort: false, + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ], + }, + }) + expect(closeMock).toHaveBeenCalledTimes(1) + }) + + it('coalesces concurrent refreshes, reuses successful cache within ttl, and refreshes again after expiry', async () => { + let resolveProbe: ((value: unknown[]) => void) | undefined + supportedModelsMock.mockImplementation(() => new Promise((resolve) => { + resolveProbe = resolve + })) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const pendingA = registry.refreshCapabilities('freshclaude') + const pendingB = registry.refreshCapabilities('kilroy') + + expect(queryFactory).toHaveBeenCalledTimes(1) + + resolveProbe?.([ + { + value: 'opus', + displayName: 'opus', + supportedEffortLevels: ['high'], + supportsAdaptiveThinking: true, + }, + ]) + + const [first, second] = await Promise.all([pendingA, pendingB]) + expect(first).toMatchObject({ ok: true, capabilities: { provider: 'freshclaude' } }) + expect(second).toMatchObject({ ok: true, capabilities: { provider: 'kilroy' } }) + + await registry.getCapabilities('freshclaude') + expect(queryFactory).toHaveBeenCalledTimes(1) + + now += 5_001 + supportedModelsMock.mockResolvedValue([ + { + value: 'haiku', + displayName: 'haiku', + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ]) + + const refreshed = await registry.getCapabilities('freshclaude') + + expect(queryFactory).toHaveBeenCalledTimes(2) + expect(refreshed).toMatchObject({ + ok: true, + capabilities: { + provider: 'freshclaude', + models: [{ id: 'haiku' }], + }, + }) + }) + + it('keeps the last successful catalog after a failed refresh', async () => { + supportedModelsMock.mockResolvedValueOnce([ + { + value: 'opus', + displayName: 'opus', + supportedEffortLevels: ['high'], + supportsAdaptiveThinking: true, + }, + ]) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const first = await registry.getCapabilities('freshclaude') + expect(first).toMatchObject({ + ok: true, + capabilities: { models: [{ id: 'opus' }] }, + }) + + supportedModelsMock.mockRejectedValueOnce(new Error('probe failed')) + + const refresh = await registry.refreshCapabilities('freshclaude') + expect(refresh).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'probe failed', + retryable: true, + }, + }) + + const cached = await registry.getCapabilities('freshclaude') + expect(cached).toMatchObject({ + ok: true, + capabilities: { models: [{ id: 'opus' }] }, + }) + }) + + it('times out a hung probe, closes it, and lets a later retry succeed', async () => { + vi.useFakeTimers() + try { + supportedModelsMock.mockImplementation(() => new Promise(() => {})) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + probeTimeoutMs: 100, + }) + + const timedOut = registry.refreshCapabilities('freshclaude') + await vi.advanceTimersByTimeAsync(100) + + await expect(timedOut).resolves.toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PROBE_FAILED', + message: 'Capability probe timed out after 100ms', + retryable: true, + }, + }) + + const firstCall = queryFactory.mock.calls[0]?.[0] as + | { options?: { abortController?: AbortController } } + | undefined + expect(firstCall?.options?.abortController?.signal.aborted).toBe(true) + expect(closeMock).toHaveBeenCalledTimes(1) + + supportedModelsMock.mockResolvedValueOnce([ + { + value: 'haiku', + displayName: 'haiku', + supportedEffortLevels: [], + supportsAdaptiveThinking: false, + }, + ]) + + const retried = await registry.refreshCapabilities('freshclaude') + expect(queryFactory).toHaveBeenCalledTimes(2) + expect(retried).toMatchObject({ + ok: true, + capabilities: { + provider: 'freshclaude', + models: [{ id: 'haiku' }], + }, + }) + } finally { + vi.useRealTimers() + } + }) + + it('rejects malformed upstream payloads with a typed error', async () => { + supportedModelsMock.mockResolvedValue([ + { + displayName: 'Missing id', + supportedEffortLevels: ['high'], + }, + ]) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const result = await registry.refreshCapabilities('freshclaude') + + expect(result).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload is missing a model id', + retryable: false, + }, + }) + expect(closeMock).toHaveBeenCalledTimes(1) + }) + + it('rejects malformed nested effort metadata with a typed error', async () => { + supportedModelsMock.mockResolvedValue([ + { + value: 'opus', + displayName: 'opus', + supportedEffortLevels: 'turbo', + }, + ]) + + const registry = new AgentChatCapabilityRegistry({ + queryFactory, + now: () => now, + ttlMs: 5_000, + }) + + const result = await registry.refreshCapabilities('freshclaude') + + expect(result).toEqual({ + ok: false, + error: { + code: 'CAPABILITY_PAYLOAD_INVALID', + message: 'Capability payload has invalid supported effort levels for opus', + retryable: false, + }, + }) + expect(closeMock).toHaveBeenCalledTimes(1) + }) +}) diff --git a/test/unit/server/config-store.test.ts b/test/unit/server/config-store.test.ts index ef4576c6..11a08c81 100644 --- a/test/unit/server/config-store.test.ts +++ b/test/unit/server/config-store.test.ts @@ -1124,10 +1124,19 @@ describe('ConfigStore', () => { await store.load() const updated = await store.patchSettings({ - agentChat: { providers: { freshclaude: { defaultModel: 'claude-sonnet-4-5-20250929' } } }, + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + }, + }, + }, }) - expect(updated.agentChat?.providers?.freshclaude?.defaultModel).toBe('claude-sonnet-4-5-20250929') + expect(updated.agentChat?.providers?.freshclaude?.modelSelection).toEqual({ + kind: 'tracked', + modelId: 'opus[1m]', + }) }) it('patchSettings deep-merges agentChat providers without clobbering other keys', async () => { @@ -1135,15 +1144,54 @@ describe('ConfigStore', () => { await store.load() await store.patchSettings({ - agentChat: { providers: { freshclaude: { defaultModel: 'claude-opus-4-6', defaultEffort: 'high' } } }, + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + effort: 'high', + }, + }, + }, }) const updated = await store.patchSettings({ agentChat: { providers: { freshclaude: { defaultPermissionMode: 'default' } } }, }) - expect(updated.agentChat?.providers?.freshclaude?.defaultModel).toBe('claude-opus-4-6') + expect(updated.agentChat?.providers?.freshclaude?.modelSelection).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) expect(updated.agentChat?.providers?.freshclaude?.defaultPermissionMode).toBe('default') - expect(updated.agentChat?.providers?.freshclaude?.defaultEffort).toBe('high') + expect(updated.agentChat?.providers?.freshclaude?.effort).toBe('high') + }) + + it('patchSettings removes stored model selections and effort overrides when cleared', async () => { + const store = new ConfigStore() + await store.load() + + await store.patchSettings({ + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'turbo', + }, + }, + }, + }) + const updated = await store.patchSettings({ + agentChat: { + providers: { + freshclaude: { + modelSelection: undefined, + effort: undefined, + }, + }, + }, + }) + + expect(updated.agentChat?.providers?.freshclaude?.modelSelection).toBeUndefined() + expect(updated.agentChat?.providers?.freshclaude?.effort).toBeUndefined() }) it('defaultSettings includes empty agentChat providers', () => { @@ -1168,8 +1216,11 @@ describe('ConfigStore', () => { const store = new ConfigStore() const loaded = await store.load() - expect(loaded.settings.agentChat?.providers?.freshclaude?.defaultModel).toBe('claude-opus-4-6') - expect(loaded.settings.agentChat?.providers?.freshclaude?.defaultEffort).toBe('high') + expect(loaded.settings.agentChat?.providers?.freshclaude?.modelSelection).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(loaded.settings.agentChat?.providers?.freshclaude?.effort).toBe('high') expect((loaded.settings as any).freshclaude).toBeUndefined() }) @@ -1194,8 +1245,11 @@ describe('ConfigStore', () => { // agentChat value wins over legacy for overlapping keys expect(loaded.settings.agentChat?.providers?.freshclaude?.defaultPermissionMode).toBe('normal') // Legacy-only keys are preserved - expect(loaded.settings.agentChat?.providers?.freshclaude?.defaultModel).toBe('claude-opus-4-6') - expect(loaded.settings.agentChat?.providers?.freshclaude?.defaultEffort).toBe('high') + expect(loaded.settings.agentChat?.providers?.freshclaude?.modelSelection).toEqual({ + kind: 'exact', + modelId: 'claude-opus-4-6', + }) + expect(loaded.settings.agentChat?.providers?.freshclaude?.effort).toBe('high') // Legacy key removed expect((loaded.settings as any).freshclaude).toBeUndefined() }) diff --git a/test/unit/server/sdk-bridge-types.test.ts b/test/unit/server/sdk-bridge-types.test.ts index 9c5dc89e..f554b429 100644 --- a/test/unit/server/sdk-bridge-types.test.ts +++ b/test/unit/server/sdk-bridge-types.test.ts @@ -227,24 +227,24 @@ describe('SDK Protocol Types', () => { expect(result.success).toBe(true) }) - it('validates sdk.create with effort field', () => { + it('validates sdk.create with an unfamiliar effort field', () => { const msg = { type: 'sdk.create', requestId: 'req-1', - effort: 'high', + effort: 'turbo', } const result = BrowserSdkMessageSchema.safeParse(msg) expect(result.success).toBe(true) if (result.success) { - expect(result.data).toHaveProperty('effort', 'high') + expect(result.data).toHaveProperty('effort', 'turbo') } }) - it('rejects sdk.create with invalid effort value', () => { + it('rejects sdk.create with an empty effort value', () => { const msg = { type: 'sdk.create', requestId: 'req-1', - effort: 'turbo', + effort: '', } const result = BrowserSdkMessageSchema.safeParse(msg) expect(result.success).toBe(false) diff --git a/test/unit/server/sdk-bridge.test.ts b/test/unit/server/sdk-bridge.test.ts index b90ddd78..dc070aea 100644 --- a/test/unit/server/sdk-bridge.test.ts +++ b/test/unit/server/sdk-bridge.test.ts @@ -22,6 +22,7 @@ let mockSupportedModels: any[] = [ ] /** When set, supportedModels() rejects with this error */ let mockSupportedModelsError: Error | null = null +let mockSupportedModelsCallCount = 0 vi.mock('@anthropic-ai/claude-agent-sdk', () => ({ query: vi.fn(({ options }: any) => { @@ -46,9 +47,13 @@ vi.mock('@anthropic-ai/claude-agent-sdk', () => ({ ;(gen as any).streamInput = vi.fn() ;(gen as any).setPermissionMode = vi.fn().mockResolvedValue(undefined) ;(gen as any).setModel = vi.fn().mockResolvedValue(undefined) - ;(gen as any).supportedModels = mockSupportedModelsError - ? vi.fn().mockRejectedValue(mockSupportedModelsError) - : vi.fn().mockResolvedValue(mockSupportedModels) + ;(gen as any).supportedModels = vi.fn(async () => { + mockSupportedModelsCallCount += 1 + if (mockSupportedModelsError) { + throw mockSupportedModelsError + } + return mockSupportedModels + }) return gen }), })) @@ -66,6 +71,7 @@ describe('SdkBridge', () => { mockInterruptFn = undefined mockKeepStreamOpen = false mockStreamEndResolve = null + mockSupportedModelsCallCount = 0 mockSupportedModels = [ { value: 'claude-opus-4-6', displayName: 'Opus 4.6', description: 'Most capable' }, { value: 'claude-sonnet-4-5-20250929', displayName: 'Sonnet 4.5', description: 'Fast' }, @@ -857,7 +863,7 @@ describe('SdkBridge', () => { const received: any[] = [] bridge.subscribe(session.sessionId, (msg) => received.push(msg)) - // sdk.session.init + sdk.models (async) + sdk.assistant + // sdk.session.init + sdk.assistant expect(received.length).toBeGreaterThanOrEqual(2) expect(received[0].type).toBe('sdk.session.init') expect(received.find(m => m.type === 'sdk.assistant')).toBeDefined() @@ -1216,8 +1222,8 @@ describe('SdkBridge', () => { }) }) - describe('fetchAndBroadcastModels', () => { - it('broadcasts sdk.models after system/init', async () => { + describe('model capability broadcasts', () => { + it('does not broadcast sdk.models after system/init', async () => { mockKeepStreamOpen = true mockMessages.push({ type: 'system', @@ -1235,81 +1241,12 @@ describe('SdkBridge', () => { await new Promise(resolve => setTimeout(resolve, 200)) - const modelsMsg = received.find(m => m.type === 'sdk.models') - expect(modelsMsg).toBeDefined() - expect(modelsMsg.models).toHaveLength(2) - expect(modelsMsg.models[0].value).toBe('claude-opus-4-6') - }) - - it('uses cached models for subsequent sessions', async () => { - mockKeepStreamOpen = true - const initMsg = { - type: 'system', - subtype: 'init', - session_id: 'cli-123', - model: 'claude-sonnet-4-5-20250929', - cwd: '/tmp', - tools: ['Bash'], - uuid: 'test-uuid', - } - - // First session — triggers fetch - mockMessages.push(initMsg) - const s1 = await bridge.createSession({ cwd: '/tmp' }) - const r1: any[] = [] - bridge.subscribe(s1.sessionId, (msg) => r1.push(msg)) - await new Promise(resolve => setTimeout(resolve, 200)) - expect(r1.find(m => m.type === 'sdk.models')).toBeDefined() - - // Second session — should use cache (change mock to prove it's not re-fetched) - mockMessages.length = 0 - mockMessages.push({ ...initMsg, uuid: 'test-uuid-2' }) - mockSupportedModels = [{ value: 'different-model', displayName: 'Different', description: 'New' }] - const s2 = await bridge.createSession({ cwd: '/tmp' }) - const r2: any[] = [] - bridge.subscribe(s2.sessionId, (msg) => r2.push(msg)) - await new Promise(resolve => setTimeout(resolve, 200)) - - const modelsMsg = r2.find(m => m.type === 'sdk.models') - expect(modelsMsg).toBeDefined() - // Should be cached value, not the new mock - expect(modelsMsg.models[0].value).toBe('claude-opus-4-6') - }) - - it('formats raw model IDs into human-readable display names', async () => { - mockKeepStreamOpen = true - // Simulate SDK returning raw model IDs as displayName - mockSupportedModels = [ - { value: 'claude-opus-4-6', displayName: 'claude-opus-4-6', description: '' }, - { value: 'claude-sonnet-4-5-20250929', displayName: 'claude-sonnet-4-5-20250929', description: '' }, - { value: 'claude-haiku-4-5-20251001', displayName: 'claude-haiku-4-5-20251001', description: '' }, - ] - mockMessages.push({ - type: 'system', - subtype: 'init', - session_id: 'cli-123', - model: 'claude-sonnet-4-5-20250929', - cwd: '/tmp', - tools: ['Bash'], - uuid: 'test-uuid', - }) - - const session = await bridge.createSession({ cwd: '/tmp' }) - const received: any[] = [] - bridge.subscribe(session.sessionId, (msg) => received.push(msg)) - - await new Promise(resolve => setTimeout(resolve, 200)) - - const modelsMsg = received.find(m => m.type === 'sdk.models') - expect(modelsMsg).toBeDefined() - expect(modelsMsg.models[0].displayName).toBe('Opus 4.6') - expect(modelsMsg.models[1].displayName).toBe('Sonnet 4.5') - expect(modelsMsg.models[2].displayName).toBe('Haiku 4.5') + expect(received.find(m => m.type === 'sdk.session.init')).toBeDefined() + expect(received.find(m => m.type === 'sdk.models')).toBeUndefined() }) - it('handles supportedModels() failure gracefully', async () => { + it('does not call supportedModels during session init', async () => { mockKeepStreamOpen = true - mockSupportedModelsError = new Error('Not supported') mockMessages.push({ type: 'system', subtype: 'init', @@ -1326,9 +1263,8 @@ describe('SdkBridge', () => { await new Promise(resolve => setTimeout(resolve, 200)) - // Should get sdk.session.init but no sdk.models (failure was swallowed) expect(received.find(m => m.type === 'sdk.session.init')).toBeDefined() - expect(received.find(m => m.type === 'sdk.models')).toBeUndefined() + expect(mockSupportedModelsCallCount).toBe(0) }) }) diff --git a/test/unit/server/ws-handler-sdk.test.ts b/test/unit/server/ws-handler-sdk.test.ts index de17c3b8..743d9602 100644 --- a/test/unit/server/ws-handler-sdk.test.ts +++ b/test/unit/server/ws-handler-sdk.test.ts @@ -3596,11 +3596,11 @@ describe('WS Handler SDK Integration', () => { type: 'sdk.create', requestId: 'req-effort', cwd: '/tmp', - effort: 'max', + effort: 'turbo', }, 'sdk.created') expect(mockSdkBridge.createSession).toHaveBeenCalledWith( - expect.objectContaining({ effort: 'max' }), + expect.objectContaining({ effort: 'turbo' }), ) } finally { ws.close() @@ -3620,11 +3620,11 @@ describe('WS Handler SDK Integration', () => { ws.send(JSON.stringify({ type: 'sdk.set-model', sessionId: 'sdk-sess-1', - model: 'claude-sonnet-4-5-20250929', + model: 'opus[1m]', })) await vi.waitFor( - () => expect(mockSdkBridge.setModel).toHaveBeenCalledWith('sdk-sess-1', 'claude-sonnet-4-5-20250929'), + () => expect(mockSdkBridge.setModel).toHaveBeenCalledWith('sdk-sess-1', 'opus[1m]'), { timeout: 3000 }, ) } finally { diff --git a/test/unit/shared/settings.test.ts b/test/unit/shared/settings.test.ts index 95c03095..6ae7eeaf 100644 --- a/test/unit/shared/settings.test.ts +++ b/test/unit/shared/settings.test.ts @@ -25,6 +25,56 @@ describe('shared settings contract', () => { }) }) + it('accepts tracked and exact agent-chat model selections with dynamic effort strings', () => { + const parsed = buildServerSettingsPatchSchema().parse({ + agentChat: { + providers: { + freshclaude: { + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'ultra', + }, + kilroy: { + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + defaultPermissionMode: 'plan', + }, + }, + }, + }) + + expect(parsed.agentChat?.providers?.freshclaude).toEqual({ + modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, + effort: 'ultra', + }) + expect(parsed.agentChat?.providers?.kilroy).toEqual({ + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + defaultPermissionMode: 'plan', + }) + }) + + it('accepts empty effort clear sentinels while allowing omitted model selections', () => { + const schema = buildServerSettingsPatchSchema() + + expect(schema.safeParse({ + agentChat: { + providers: { + freshclaude: { + defaultPermissionMode: 'plan', + effort: 'ultra', + }, + }, + }, + }).success).toBe(true) + expect(schema.safeParse({ + agentChat: { + providers: { + freshclaude: { + effort: '', + }, + }, + }, + }).success).toBe(true) + }) + it('rejects representative local-only fields in the server patch schema', () => { const schema = buildServerSettingsPatchSchema() @@ -75,6 +125,24 @@ describe('shared settings contract', () => { expect(merged.agentChat.defaultPlugins).toEqual(['/custom/plugins/local-tools']) }) + it('migrates legacy defaultModel/defaultEffort values into exact selections and explicit effort overrides', () => { + const merged = mergeServerSettings(createDefaultServerSettings({ loggingDebug: false }), { + agentChat: { + providers: { + freshclaude: { + defaultModel: 'claude-opus-4-6', + defaultEffort: 'high', + } as any, + }, + }, + }) + + expect(merged.agentChat.providers.freshclaude).toEqual({ + modelSelection: { kind: 'exact', modelId: 'claude-opus-4-6' }, + effort: 'high', + }) + }) + it('mergeServerSettings preserves runtime CLI providers outside the built-in defaults', () => { const merged = mergeServerSettings(createDefaultServerSettings({ loggingDebug: false }), { codingCli: {