test: add browser agent integration tests by kunal-10-cloud · Pull Request #21151 · google-gemini/gemini-cli

kunal-10-cloud · 2026-03-04T20:49:48Z

Summary

Adds end-to-end integration tests for the browser agent to verify the full flow (prompt → browser_agent delegation → MCP → Chrome DevTools → result) that isn't covered by existing mocked unit tests.

Details

browser-agent.test.ts: Added 5 new integration tests (navigate+snapshot, screenshot, interaction, cleanup, sequential).
Added graceful skipping (describe.skipIf) on environments without Chrome/Chromium.
Fake Responses: Added 5 JSONL fake response files matching existing test patterns.
CI: Added "Ensure Chrome is available" step to the e2e_linux job in chained_e2e.yml.

Related Issues

Closes #21120

How to Validate

Run `npm run build`
Run `GEMINI_API_KEY=test npx vitest run --root ./integration-tests browser-agent.test.ts`
Verify all 5 tests pass locally.

Pre-Merge Checklist

gemini-code-assist · 2026-03-04T20:50:06Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the testing suite by introducing comprehensive end-to-end integration tests for the browser agent. These tests validate the entire workflow from user prompt to browser interaction via Chrome DevTools, ensuring robust functionality beyond what mocked unit tests can provide. The addition of these tests improves the reliability and stability of the browser agent feature by verifying its full operational flow.

Highlights

New Integration Tests: Added five new end-to-end integration tests for the browser agent, covering navigation, screenshot, interaction, cleanup, and sequential operations.
Environment Compatibility: Introduced describe.skipIf to gracefully skip browser agent tests on environments lacking Chrome/Chromium, ensuring test suite stability.
Fake Response Data: Created five new JSONL fake response files to support the new integration tests, simulating API interactions.
CI/CD Enhancement: Updated the e2e_linux CI job to include a step ensuring Chrome is available for testing, improving CI reliability for browser-dependent tests.

Changelog

integration-tests/browser-agent.cleanup.responses
- Added fake API responses for the browser agent cleanup test.
integration-tests/browser-agent.interaction.responses
- Added fake API responses for the browser agent interaction test.
integration-tests/browser-agent.navigate-snapshot.responses
- Added fake API responses for the browser agent navigation and snapshot test.
integration-tests/browser-agent.screenshot.responses
- Added fake API responses for the browser agent screenshot test.
integration-tests/browser-agent.sequential.responses
- Added fake API responses for the browser agent sequential operations test.
integration-tests/browser-agent.test.ts
- Added five new integration tests for the browser agent.
- Implemented logic to check for Chrome availability and skip tests if not found.
- Included test setup and teardown using TestRig for isolated test environments.

Ignored Files

Ignored by pattern: .github/workflows/** (1)
- .github/workflows/chained_e2e.yml

Activity

The author has added new tests to cover the browser agent's end-to-end flow.
Validation steps for running the tests locally have been provided in the PR description.
The author has confirmed validation on MacOS using npm run.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a valuable set of integration tests for the browser agent, verifying the end-to-end flow, which is a great addition for ensuring reliability. No security vulnerabilities were found in this change. Feedback includes a suggestion to make the Chrome availability check on macOS more robust to improve the developer experience when running these tests.

integration-tests/browser-agent.test.ts

kunal-10-cloud · 2026-03-04T21:01:11Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a valuable set of integration tests for the browser agent, verifying the end-to-end flow which is a great addition for ensuring reliability. The tests are well-structured, using a TestRig for setup and teardown, and correctly skip execution if Chrome is not available. I have one suggestion to improve the cleanup test by removing an unnecessary delay, which will make the test suite faster without compromising its effectiveness.

integration-tests/browser-agent.test.ts

kunal-10-cloud · 2026-03-04T21:23:57Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a valuable set of end-to-end integration tests for the browser agent, verifying the complete flow from prompt to browser interaction. The tests are well-structured, using a test rig and fake responses for deterministic execution. The inclusion of a check to skip tests gracefully on environments without Chrome is a good practice. My review includes one suggestion to enhance the Chrome availability check to include support for Windows, which would improve cross-platform test coverage.

integration-tests/browser-agent.test.ts

kunal-10-cloud · 2026-03-04T21:31:39Z

/gemini review

gemini-code-assist

Code Review

The original review comments proposed using statSync for Chrome detection on Windows. However, this directly contradicts the repository rule that mandates the use of asynchronous file system operations (e.g., fs.promises.stat) to avoid blocking the event loop. As such, both comments have been dropped to adhere to the established guidelines.

kunal-10-cloud · 2026-03-04T21:40:03Z

hii @jacob314, @gsquared94 can you please take some time to review this PR and let me know if any changes are required?

gsquared94 · 2026-03-05T08:15:02Z

Hey, thanks for these tests—they're a solid addition. Some issues:

Windows npx bug: In browserManager.ts, the browser agent uses command: 'npx'. On Windows, spawn won't find it unless you use npx.cmd. This likely means the feature is currently broken on Windows. Can you fix this along with your tests?
Chrome discovery: Standard Windows installs don't put Chrome in the PATH, so where chrome in the test will fail. We should probably check the standard C:\Program Files paths too.
Node 20 Compatibility: You're using import.meta.dirname, but the project supports Node 20.0.0. dirname was only added in 20.11.0. Better to use the standard fileURLToPath approach so we don't break the CLI for users on
older LTS releases.
CI: Can we add the "Ensure Chrome is available" step to the e2e_windows job in chained_e2e.yml?

Could you enable Actions on your fork and run the full E2E suite there and share a link to a successful run to help verify these changes.

- Fix Windows npx bug: use npx.cmd on win32 in browserManager.ts - Fix Chrome discovery: check C:\Program Files paths with fs.existsSync - Fix Node 20.0 compat: replace import.meta.dirname with fileURLToPath - Add Chrome availability step to e2e_windows CI job - Update browserManager unit tests for platform-conditional npx

This reverts commit 545b786.

kunal-10-cloud · 2026-03-05T10:23:22Z

Hey, thanks for these tests—they're a solid addition. Some issues:

Windows npx bug: In browserManager.ts, the browser agent uses command: 'npx'. On Windows, spawn won't find it unless you use npx.cmd. This likely means the feature is currently broken on Windows. Can you fix this along with your tests?

Chrome discovery: Standard Windows installs don't put Chrome in the PATH, so where chrome in the test will fail. We should probably check the standard C:\Program Files paths too.

Node 20 Compatibility: You're using import.meta.dirname, but the project supports Node 20.0.0. dirname was only added in 20.11.0. Better to use the standard fileURLToPath approach so we don't break the CLI for users on
older LTS releases.

CI: Can we add the "Ensure Chrome is available" step to the e2e_windows job in chained_e2e.yml?

Could you enable Actions on your fork and run the full E2E suite there and share a link to a successful run to help verify these changes.

@gsquared94 thank you for the feedback
i have implemented the necessary, kindly review it once
also attaching the link of workflows for the same to verify the changes
https://github.com/kunal-10-cloud/gemini-cli/actions

gsquared94

LGTM!

kunal-10-cloud · 2026-03-05T11:11:45Z

@gsquared94 really sorry, there was formatting issue due to which merge was blocked, have tested it now locally, could you please enable the merge

test: add browser agent integration tests

e3d4d6d

kunal-10-cloud requested review from a team as code owners March 4, 2026 20:49

gemini-code-assist bot reviewed Mar 4, 2026

View reviewed changes

integration-tests/browser-agent.test.ts Outdated Show resolved Hide resolved

Merge branch 'main' into test/browser-agent-integration-tests

f74c45c

gemini-cli bot added the area/platform Issues related to Build infra, Release mgmt, Testing, Eval infra, Capacity, Quota mgmt label Mar 4, 2026

gemini-code-assist bot reviewed Mar 4, 2026

View reviewed changes

integration-tests/browser-agent.test.ts Outdated Show resolved Hide resolved

kunal-10-cloud added 4 commits March 5, 2026 02:47

Merge branch 'main' into test/browser-agent-integration-tests

0b095d9

fix:updated commit on gemini bot's suggestion

9754d75

fix:removed wait time as per gemini bot suggestion

e40ff1c

test: remove explicit timeout delay per PR review

0cf2226

gemini-code-assist bot reviewed Mar 4, 2026

View reviewed changes

integration-tests/browser-agent.test.ts Show resolved Hide resolved

test: add windows support for chrome availability check

5a17505

gemini-code-assist bot reviewed Mar 4, 2026

View reviewed changes

Merge branch 'main' into test/browser-agent-integration-tests

37b8f16

kunal-10-cloud added 4 commits March 5, 2026 15:13

ci: temporarily enable workflow on fork for E2E verification

545b786

Revert "ci: temporarily enable workflow on fork for E2E verification"

f088a6a

This reverts commit 545b786.

Merge branch 'main' into test/browser-agent-integration-tests

b1aecfb

gsquared94 approved these changes Mar 5, 2026

View reviewed changes

gsquared94 enabled auto-merge March 5, 2026 10:37

chore: fix prettier formatting issues

b9d1388

auto-merge was automatically disabled March 5, 2026 11:08
Head branch was pushed to by a user without write access

kunal-10-cloud requested a review from gsquared94 March 5, 2026 11:12

gsquared94 enabled auto-merge March 5, 2026 11:17

gsquared94 approved these changes Mar 5, 2026

View reviewed changes

sehoon38 approved these changes Mar 5, 2026

View reviewed changes

gsquared94 added this pull request to the merge queue Mar 5, 2026

Merged via the queue into google-gemini:main with commit 09259a8 Mar 5, 2026
27 checks passed

struckoff pushed a commit to struckoff/gemini-cli that referenced this pull request Mar 6, 2026

test: add browser agent integration tests (google-gemini#21151)

1f994ee

gemini-code-assist bot mentioned this pull request Mar 11, 2026

Changelog for v0.34.0-preview.0 #21965

Merged

kunal-10-cloud added a commit to kunal-10-cloud/gemini-cli that referenced this pull request Mar 12, 2026

test: add browser agent integration tests (google-gemini#21151)

4d5b9f1

liamhelmer pushed a commit to badal-io/gemini-cli that referenced this pull request Mar 12, 2026

test: add browser agent integration tests (google-gemini#21151)

1bf1d23

yashodipmore pushed a commit to yashodipmore/geemi-cli that referenced this pull request Mar 21, 2026

test: add browser agent integration tests (google-gemini#21151)

20718cb

Conversation

kunal-10-cloud commented Mar 4, 2026

Summary

Details

Related Issues

How to Validate

Pre-Merge Checklist

Uh oh!

gemini-code-assist bot commented Mar 4, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

kunal-10-cloud commented Mar 4, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

kunal-10-cloud commented Mar 4, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

kunal-10-cloud commented Mar 4, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

kunal-10-cloud commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gsquared94 commented Mar 5, 2026

Uh oh!

kunal-10-cloud commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gsquared94 left a comment

Choose a reason for hiding this comment

Uh oh!

kunal-10-cloud commented Mar 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kunal-10-cloud commented Mar 4, 2026 •

edited

Loading

kunal-10-cloud commented Mar 5, 2026 •

edited

Loading