Skip to content

feat(cli): implement non-invasive UX Journey testing framework#23030

Open
NTaylorMullen wants to merge 3 commits into
mainfrom
ntm/visual-journey-sentinels
Open

feat(cli): implement non-invasive UX Journey testing framework#23030
NTaylorMullen wants to merge 3 commits into
mainfrom
ntm/visual-journey-sentinels

Conversation

@NTaylorMullen
Copy link
Copy Markdown
Collaborator

Summary

This PR introduces a robust "UX Journey" testing framework for Gemini CLI, enabling non-invasive verification of React component presence and visual state in the terminal.

Details

The framework allows for "White Box" testing of the terminal UI without requiring manual instrumentation (sentinels) of production components. Key features include:

  • Automatic Component Discovery: Leverages a modified Ink reconciler to automatically tag DOM nodes with their React owner component name (e.g., SuggestionsDisplay).
  • Deterministic Polling: Adds rig.waitForComponent('Name') to handle asynchronous UI transitions reliably.
  • SVG Audit Trail: Injects component metadata into SVG snapshots (e.g., <!-- component: SuggestionsDisplay -->), allowing for logical verification of "hidden" or flickering components in PR reviews.
  • Zero Mutation: Production components remain 100% untouched.

Related Issues

Part of the effort to improve visual testing reliability.

How to Validate

Run the new UX journey test:

npm test -w @google/gemini-cli -- src/ui/components/SuggestionsDisplay.ux.test.tsx

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
      • npm run

@NTaylorMullen NTaylorMullen requested a review from a team as a code owner March 19, 2026 00:07
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a foundational framework for non-invasive UX testing within the Gemini CLI. It allows developers to verify the presence and state of React components in the terminal UI without altering the production code. By integrating automatic component discovery, deterministic waiting mechanisms, and an SVG-based audit trail, the framework significantly enhances the reliability and maintainability of visual tests, ensuring a consistent user experience.

Highlights

  • New UX Journey Testing Framework: Introduced a robust 'UX Journey' testing framework for the Gemini CLI, enabling non-invasive verification of React component presence and visual state directly within the terminal UI.
  • Automatic Component Discovery: Implemented a mechanism that leverages a modified Ink reconciler to automatically tag DOM nodes with their React owner component name, facilitating 'White Box' testing without manual instrumentation.
  • Deterministic Polling for UI Transitions: Added a new rig.waitForComponent('Name') method to AppRig to reliably handle asynchronous UI transitions and ensure components are present before assertions.
  • SVG Audit Trail with Component Metadata: Enhanced SVG snapshot generation to inject component metadata (e.g., <!-- component: SuggestionsDisplay -->) as comments, allowing for logical verification of components, even if hidden or flickering.
  • New toVisuallyContain Custom Matcher: Created a new Vitest custom matcher, toVisuallyContain, which asserts the presence of a component in the Ink tree or its metadata within the SVG audit trail.
  • Zero Production Component Mutation: Ensured that the entire testing framework operates without modifying production components, maintaining the integrity of the application code.
  • Example UX Test Case: Added a new UX journey test for SuggestionsDisplay to demonstrate the usage and capabilities of the new framework, verifying its visual behavior when a '/' is typed.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@NTaylorMullen NTaylorMullen force-pushed the ntm/visual-journey-sentinels branch from 6de1e0c to cd37a59 Compare March 19, 2026 00:10
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a robust and non-invasive UX Journey testing framework, which is a significant improvement for UI testing reliability. The approach of using a modified reconciler to inject test metadata, combined with custom matchers and SVG audit trails, is well-executed and avoids polluting production code. The new waitForComponent and toVisuallyContain utilities provide a clean API for writing these tests. I have one suggestion to further improve the maintainability and type safety of the new test utilities.

Comment thread packages/cli/src/test-utils/AppRig.tsx Outdated
@NTaylorMullen NTaylorMullen force-pushed the ntm/visual-journey-sentinels branch 2 times, most recently from 998967d to daf5114 Compare March 19, 2026 00:19
@github-actions
Copy link
Copy Markdown

Size Change: -4 B (0%)

Total Size: 26.2 MB

Filename Size Change
./bundle/chunk-3HJTL4LL.js 0 B -3.63 MB (removed) 🏆
./bundle/chunk-BQ7ATBJT.js 0 B -13.5 MB (removed) 🏆
./bundle/core-A3LADOF7.js 0 B -41.4 kB (removed) 🏆
./bundle/devtoolsService-TNPY32P2.js 0 B -27.7 kB (removed) 🏆
./bundle/interactiveCli-VVTED4MG.js 0 B -1.61 MB (removed) 🏆
./bundle/oauth2-provider-XMIGBZBM.js 0 B -9.19 kB (removed) 🏆
./bundle/chunk-B6SHLMYP.js 13.5 MB +13.5 MB (new file) 🆕
./bundle/chunk-OKHLHC3T.js 3.63 MB +3.63 MB (new file) 🆕
./bundle/core-7VPDQY5Z.js 41.4 kB +41.4 kB (new file) 🆕
./bundle/devtoolsService-SO2YOWHH.js 27.7 kB +27.7 kB (new file) 🆕
./bundle/interactiveCli-SGLWZLYM.js 1.61 MB +1.61 MB (new file) 🆕
./bundle/oauth2-provider-UAMCXM5Z.js 9.19 kB +9.19 kB (new file) 🆕
ℹ️ View Unchanged
Filename Size
./bundle/chunk-34MYV7JD.js 2.45 kB
./bundle/chunk-37ZTTFQF.js 966 kB
./bundle/chunk-5AUYMPVF.js 858 B
./bundle/chunk-664ZODQF.js 124 kB
./bundle/chunk-DAHVX5MI.js 206 kB
./bundle/chunk-G4TMH6EN.js 1.95 MB
./bundle/chunk-IUUIT4SU.js 56.5 kB
./bundle/chunk-RJTRUG2J.js 39.8 kB
./bundle/devtools-36NN55EP.js 696 kB
./bundle/dist-T73EYRDX.js 356 B
./bundle/gemini.js 697 kB
./bundle/getMachineId-bsd-TXG52NKR.js 1.55 kB
./bundle/getMachineId-darwin-7OE4DDZ6.js 1.55 kB
./bundle/getMachineId-linux-SHIFKOOX.js 1.34 kB
./bundle/getMachineId-unsupported-5U5DOEYY.js 1.06 kB
./bundle/getMachineId-win-6KLLGOI4.js 1.72 kB
./bundle/memoryDiscovery-KKSSU3SI.js 922 B
./bundle/multipart-parser-KPBZEGQU.js 11.7 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/client/main.js 221 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/_client-assets.js 227 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/index.js 11.5 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/types.js 132 B
./bundle/sandbox-macos-permissive-open.sb 890 B
./bundle/sandbox-macos-permissive-proxied.sb 1.31 kB
./bundle/sandbox-macos-restrictive-open.sb 3.36 kB
./bundle/sandbox-macos-restrictive-proxied.sb 3.56 kB
./bundle/sandbox-macos-strict-open.sb 4.82 kB
./bundle/sandbox-macos-strict-proxied.sb 5.02 kB
./bundle/src-QVCVGIUX.js 47 kB
./bundle/tree-sitter-7U6MW5PS.js 274 kB
./bundle/tree-sitter-bash-34ZGLXVX.js 1.84 MB
./bundle/undici-4X2YZID5.js 360 B

compressed-size-action

@gemini-cli gemini-cli Bot added the status/need-issue Pull requests that need to have an associated issue. label Mar 19, 2026
Comment thread packages/cli/src/test-utils/customMatchers.ts Outdated
Comment thread packages/cli/src/test-utils/svg.ts Outdated
Comment thread packages/cli/src/test-utils/customMatchers.ts Outdated
…sting

- Expose Ink rootNode in AppRig and render utilities for tree traversal
- Add toVisuallyContain matcher to verify component presence in the Ink DOM
- Inject component metadata as comments into generated SVGs for auditing
- Add waitForComponent to AppRig for deterministic UI state synchronization
- Implement visual journey test for SuggestionsDisplay
…sting

- Enhance AppRig and matchers to support robust component discovery via node attributes and styles

- Update SuggestionsDisplay to support logical component tagging

- Fix act() warnings and stability issues in SuggestionsDisplay tests

- Refresh snapshots and rebase on origin/main
@NTaylorMullen NTaylorMullen force-pushed the ntm/visual-journey-sentinels branch from daf5114 to 05bf04c Compare March 31, 2026 23:47
@NTaylorMullen NTaylorMullen requested a review from a team as a code owner March 31, 2026 23:47
- Replace 'any' with 'TestableDOMNode' for Ink tree traversal in AppRig and custom matchers
- Rename 'toVisuallyContain' to 'toContainComponent' for clarity
- Remove SVG audit trail (component injection) to prevent snapshot churn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status/need-issue Pull requests that need to have an associated issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants