feat(core): add tool name validation in TOML policy files

[allenhutchison]

Summary

Fixes #12119

When a user misspells a tool name in a TOML policy file (e.g., toolName = "run_shell" instead of "run_shell_command"), the rule silently does nothing. This adds validation that warns about unrecognized tool names with "did you mean?" suggestions.

• Built-in tool validation at TOML load time: Uses Levenshtein distance (threshold ≤ 3) to detect likely typos of built-in tool names while avoiding false positives for dynamically registered tools (agent tools, custom tools)
• MCP tool validation after server connection: Cross-references policy rules against actually discovered MCP server tools in McpClient.discover(), warning about typos like people.getxMe → people.getMe
• Severity-aware error reporting: Adds severity field to PolicyFileError so warnings display as warnings (not errors) in the UI via coreEvents.emitFeedback
• Non-blocking: Rules still load normally regardless of warnings

Files changed

File	Change
packages/core/src/policy/toml-loader.ts	Add tool_name_warning error type, severity field, validateToolName() for built-in tools, validateMcpPolicyToolNames() for MCP tools
packages/core/src/policy/config.ts	Update formatPolicyError and error emission to use severity
packages/core/src/tools/mcp-client.ts	Call validateMcpPolicyToolNames after MCP tool discovery
packages/core/src/policy/toml-loader.test.ts	Add getErrors/getWarnings helpers, update 7 existing tests, add 18 new tests

Test plan

• All 49 tests pass in toml-loader.test.ts
• Pre-commit hooks (prettier, eslint) pass
• Manual test: misspelled built-in tool name ("grob") shows warning with suggestion ("glob")
• Manual test: dynamic/agent tool name ("delegate_to_agent") does NOT produce false positive
• Manual test: misspelled MCP tool name ("people.getxMe") shows warning after server connects

[gemini-code-assist]

Summary of Changes

Hello @allenhutchison, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the robustness of policy file processing by adding comprehensive validation for tool names specified in TOML policy configurations. It addresses a silent failure mode where misspelled tool names would lead to rules not being applied. By introducing both built-in and MCP-specific tool name validation, with intelligent typo detection and severity-aware feedback, the system now proactively informs users of potential misconfigurations without blocking policy loading.

Highlights

• Built-in Tool Name Validation: Introduced validation for tool names in TOML policy files, providing 'did you mean?' suggestions for likely typos of built-in tools using Levenshtein distance.
• MCP Tool Name Validation: Implemented post-connection validation for MCP (Multi-Cloud Platform) tool names, cross-referencing policy rules against actually discovered MCP server tools.
• Severity-Aware Error Reporting: Enhanced policy error reporting with a 'severity' field, allowing warnings to be displayed distinctly from errors in the UI via coreEvents.emitFeedback.
• Non-Blocking Warnings: Ensured that policy rules continue to load and function normally even when warnings are present, making the validation process non-blocking.

Changelog

• packages/core/src/policy/config.ts
  • Updated formatPolicyError to display 'warning' or 'error' based on the error's severity.
  • Modified coreEvents.emitFeedback to use the error's severity property when emitting feedback.
• packages/core/src/policy/toml-loader.test.ts
  • Added getErrors and getWarnings helper functions to filter PolicyLoadResult errors by severity.
  • Updated 7 existing tests to use the new getErrors helper for assertions.
  • Added 18 new tests covering various scenarios for built-in and MCP tool name validation, including typos, wildcards, and dynamic tools.
• packages/core/src/policy/toml-loader.ts
  • Added tool_name_warning to PolicyFileErrorType and a severity field to the PolicyFileError interface.
  • Introduced MAX_TYPO_DISTANCE constant and validateToolName function for built-in tool name validation using Levenshtein distance.
  • Integrated built-in tool name validation into loadPoliciesFromToml for both rule and safety_checker sections.
  • Exported validateMcpPolicyToolNames function to perform validation of MCP tool names against discovered tools.
• packages/core/src/tools/mcp-client.ts
  • Imported validateMcpPolicyToolNames from toml-loader.ts.
  • Called validateMcpPolicyToolNames after MCP tool discovery to emit warnings for misspelled MCP tool names found in policy rules.

Activity

• All 49 tests in toml-loader.test.ts passed.
• Pre-commit hooks (prettier, eslint) passed.
• Manual tests were conducted to verify warnings for misspelled built-in tool names, correct handling of dynamic/agent tool names, and warnings for misspelled MCP tool names after server connection.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Size Change: +3.95 kB (+0.02%)

Total Size: 25.8 MB

Filename	Size	Change
./bundle/gemini.js	25.3 MB	+3.95 kB (+0.02%)

ℹ️ View Unchanged

Filename	Size
./bundle/node_modules/@google/gemini-cli-devtools/dist/client/main.js	221 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/_client-assets.js	227 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/index.js	11.5 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/types.js	132 B
./bundle/sandbox-macos-permissive-open.sb	890 B
./bundle/sandbox-macos-permissive-proxied.sb	1.31 kB
./bundle/sandbox-macos-restrictive-open.sb	3.36 kB
./bundle/sandbox-macos-restrictive-proxied.sb	3.56 kB
./bundle/sandbox-macos-strict-open.sb	4.82 kB
./bundle/sandbox-macos-strict-proxied.sb	5.02 kB

_{compressed-size-action}

[gemini-code-assist]

Code Review

This pull request introduces a valuable feature to validate tool names in TOML policy files, warning users about potential typos using Levenshtein distance. This improves user experience by preventing silent failures from misspelled tool names. The implementation correctly handles built-in tools at load time and MCP tools after server discovery. I've found one high-severity issue where a typo resembling an MCP tool name format could be missed by the validation logic. My review includes a suggestion to address this gap. Otherwise, the changes are well-implemented and thoroughly tested.

[Abhijit-2592]

LGTM with a small NIT