Skip to content

[router][tool call] Clean up redundant detect_format and has_tool_markers#11270

Merged
slin1237 merged 4 commits intomainfrom
chang/stream
Oct 6, 2025
Merged

[router][tool call] Clean up redundant detect_format and has_tool_markers#11270
slin1237 merged 4 commits intomainfrom
chang/stream

Conversation

@CatherineSue
Copy link
Collaborator

@CatherineSue CatherineSue commented Oct 6, 2025

Motivation

Out of the 11 tool parsers, 8 of them has the same implementation of detect_format and has_tool_markers. This PR removes the redundancy.

Modifications

  • Replace detect_format with has_tool_markers
  • Update all references

Accuracy Tests

Benchmarking and Profiling

Checklist

- Renamed detect_format to has_tool_markers throughout the codebase
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @CatherineSue, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request streamlines the tool parsing mechanism by eliminating redundant code for detecting tool markers across multiple parser implementations. By consolidating the detect_format and has_tool_markers methods into a single, consistently named has_tool_markers function within the ToolParser trait, the change enhances code maintainability, clarity, and consistency in how tool calls are identified within the router.

Highlights

  • Refactored ToolParser Trait: The detect_format method within the ToolParser trait has been renamed to has_tool_markers to provide a more accurate description of its function: checking for the presence of tool call markers in text.
  • Consolidated Tool Marker Detection Logic: Redundant has_tool_markers implementations across various tool parsers (DeepSeek, GLM-4 MoE, GPT-OSS, Kimi K2, Mistral, Qwen, Step3) have been removed. Their logic has been integrated into the single, renamed has_tool_markers method, reducing code duplication.
  • Updated All References: All internal and external call sites that previously invoked detect_format, including those in the gRPC router logic and unit tests, have been updated to correctly use the new has_tool_markers method.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request does a great job of cleaning up the tool parsers by replacing the redundant detect_format method with has_tool_markers. The changes are clear and improve the overall code structure. However, I've identified a logic error in the KimiK2Parser implementation that seems to have been oversimplified during the refactoring. This change would likely cause a test failure and affect tool call detection for that parser. My detailed comment provides a suggested fix.

@slin1237 slin1237 merged commit 466992b into main Oct 6, 2025
35 checks passed
@slin1237 slin1237 deleted the chang/stream branch October 6, 2025 21:04
ch-tiger1 pushed a commit to ch-tiger1/sglang that referenced this pull request Oct 9, 2025
lpc0220 pushed a commit to lpc0220/sglang that referenced this pull request Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments