Skip to content

Add behavioral evals for web tool selection (google_web_search vs web_fetch) #23483

@PewterZz

Description

@PewterZz

Summary

The eval suite has no tests covering how the agent chooses between google_web_search and web_fetch. These are the two web tools available to the agent but they serve different purposes:

  • google_web_search — for open-ended queries where the agent needs to find information
  • web_fetch — for fetching a specific URL the user has already provided

Without evals, regressions in this distinction go undetected.

Expected behavior

  • When asked for current information with no URL, agent should use google_web_search
  • When given a specific URL, agent should use web_fetch, not search
  • When the answer is in local files, agent should use neither web tool

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentIssues related to Core Agent, Tools, Memory, Sub-Agents, Hooks, Agent Qualitystatus/need-triageIssues that need to be triaged by the triage automation.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions