[router][grpc] Add dependencies in Cargo.toml to support chat template rendering

[CatherineSue]

Motivation

The current grpc router would fail with tool call request.

Error:

2025-10-08 20:01:48 ERROR http_request{method=POST uri=/v1/chat/completions version=HTTP/1.1 request_id="chatcmpl-pWPzLmWXND9b39GTogjW5XIm"}: sglang_router_rs::routers::grpc::utils: src/routers/grpc/utils.rs:441: Failed to apply chat template: Failed to render template: unknown filter: filter tojson is unknown (in chat:60)

RCA:
The model being tested is Llama-3.1-8B-Instruct. Its chat template requires tojson filter to parse tool call. See https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct?chat_template=default#L38

This is available in json and builtins features in minijinja crate.

Modifications

• Add json and builtins for minijinja in Cargo.toml
• Remove duplicate has_tool_call in json_parser.rs and llama_parser.rs

Accuracy Tests

non-stream, tool_choice = "auto"

non-stream, tool_choice = "required"

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @CatherineSue, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This PR addresses a critical bug in the gRPC router preventing proper tool call handling for certain models by enabling required minijinja features and streamlining tool detection logic in json_parser.rs and llama_parser.rs.

Highlights

• Intent: This pull request aims to resolve an issue where the gRPC router fails to process tool call requests, specifically for models like Llama-3.1-8B-Instruct, due to a missing tojson filter in chat template rendering. The tojson filter is required for parsing tool calls and is enabled by adding specific features to the minijinja crate.
• Changes: The primary change involves updating sgl-router/Cargo.toml to include the json and builtins features for the minijinja dependency, enabling the necessary tojson filter. Additionally, the PR refactors tool call detection logic in sgl-router/src/tool_parser/parsers/json_parser.rs and sgl-router/src/tool_parser/parsers/llama_parser.rs. This includes removing a duplicate has_tool_call method and simplifying the has_tool_markers function in both parsers to make tool call detection more general by removing the explicit check for "name" within the tool call text.
• Reviewer Activity: None

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

Thank you for this contribution! The primary change to enable tojson and builtins features for minijinja is a great fix for the tool-calling issue with Llama-3.1 models.

I've reviewed the refactoring part of your changes where you remove the duplicate has_tool_call functions. While this is a good cleanup, I've found a couple of potential regressions in the new implementation of has_tool_markers that could cause tool calls to be missed during streaming. My comments below provide more details and suggestions to address this. Please take a look!