Skip to content

Update tool parser and related documentation #11223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JustinTong0323 merged 5 commits into from
Oct 7, 2025
Merged

Update tool parser and related documentation #11223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fd18079
Update tool parser and related documentation
JustinTong0323 Oct 5, 2025
4df3784
Merge branch 'main' into xinyuan-update-tool-call-parser
JustinTong0323 Oct 5, 2025
9e5dbea
upd
JustinTong0323 Oct 5, 2025
fcb55d7
Merge branch 'main' into xinyuan-update-tool-call-parser
JustinTong0323 Oct 6, 2025
eed2bfc
update order
JustinTong0323 Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions docs/advanced_features/tool_parser.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,18 @@
"\n",
"| Parser | Supported Models | Notes |\n",
"|---|---|---|\n",
"| `deepseekv3` | DeepSeek-v3 (e.g., `deepseek-ai/DeepSeek-V3-0324`) | Recommend adding `--chat-template ./examples/chat_template/tool_chat_template_deepseekv3.jinja` to launch command. |\n",
"| `deepseekv31` | DeepSeek-V3.1 and DeepSeek-V3.2 (e.g. `deepseek-ai/DeepSeek-V3.1`, `deepseek-ai/DeepSeek-V3.2-Exp`) | Recommend adding `--chat-template ./examples/chat_template/tool_chat_template_deepseekv31.jinja` (Or ..deepseekv32.jinja for DeepSeek-V3.2) to launch command. |\n",
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Oct 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The path ..deepseekv32.jinja in the note is a bit ambiguous and could be a typo. For clarity and to avoid user confusion, it's better to provide the full relative path from the project root, similar to the other examples. Also, "Or" is typically not capitalized in parentheses.

| `deepseekv31` | DeepSeek-V3.1 and DeepSeek-V3.2 (e.g. `deepseek-ai/DeepSeek-V3.1`, `deepseek-ai/DeepSeek-V3.2-Exp`) | Recommend adding `--chat-template ./examples/chat_template/tool_chat_template_deepseekv31.jinja` (or `./examples/chat_template/tool_chat_template_deepseekv32.jinja` for DeepSeek-V3.2) to launch command. |\n

"| `glm` | GLM series (e.g. `zai-org/GLM-4.6`) | |\n",
"| `gpt-oss` | GPT-OSS (e.g., `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `lmsys/gpt-oss-120b-bf16`, `lmsys/gpt-oss-20b-bf16`) | The gpt-oss tool parser filters out analysis channel events and only preserves normal text. This can cause the content to be empty when explanations are in the analysis channel. To work around this, complete the tool round by returning tool results as `role=\"tool\"` messages, which enables the model to generate the final content. |\n",
"| `kimi_k2` | `moonshotai/Kimi-K2-Instruct` | |\n",
"| `llama3` | Llama 3.1 / 3.2 / 3.3 (e.g. `meta-llama/Llama-3.1-8B-Instruct`, `meta-llama/Llama-3.2-1B-Instruct`, `meta-llama/Llama-3.3-70B-Instruct`) | |\n",
"| `llama4` | Llama 4 (e.g. `meta-llama/Llama-4-Scout-17B-16E-Instruct`) | |\n",
"| `mistral` | Mistral (e.g. `mistralai/Mistral-7B-Instruct-v0.3`, `mistralai/Mistral-Nemo-Instruct-2407`, `mistralai/Mistral-7B-v0.3`) | |\n",
"| `qwen25` | Qwen 2.5 (e.g. `Qwen/Qwen2.5-1.5B-Instruct`, `Qwen/Qwen2.5-7B-Instruct`) and QwQ (i.e. `Qwen/QwQ-32B`) | For QwQ, reasoning parser can be enabled together with tool call parser. See [reasoning parser](https://docs.sglang.ai/backend/separate_reasoning.html). |\n",
"| `deepseekv3` | DeepSeek-v3 (e.g., `deepseek-ai/DeepSeek-V3-0324`) | |\n",
"| `gpt-oss` | GPT-OSS (e.g., `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `lmsys/gpt-oss-120b-bf16`, `lmsys/gpt-oss-20b-bf16`) | The gpt-oss tool parser filters out analysis channel events and only preserves normal text. This can cause the content to be empty when explanations are in the analysis channel. To work around this, complete the tool round by returning tool results as `role=\"tool\"` messages, which enables the model to generate the final content. |\n",
"| `kimi_k2` | `moonshotai/Kimi-K2-Instruct` | |\n",
"| `pythonic` | Llama-3.2 / Llama-3.3 / Llama-4 | Model outputs function calls as Python code. Requires `--tool-call-parser pythonic` and is recommended to use with a specific chat template. |\n"
"| `pythonic` | Llama-3.2 / Llama-3.3 / Llama-4 | Model outputs function calls as Python code. Requires `--tool-call-parser pythonic` and is recommended to use with a specific chat template. |\n",
"| `qwen` | Qwen series (e.g. `Qwen/Qwen3-Next-80B-A3B-Instruct`, `Qwen/Qwen3-VL-30B-A3B-Thinking`) except Qwen3-Coder| |\n",
"| `qwen3_coder` | Qwen3-Coder (e.g. `Qwen/Qwen3-Coder-30B-A3B-Instruct`) | |\n",
"| `step3` | Step-3 | |\n"
]
},
{
Expand Down
14 changes: 8 additions & 6 deletions python/sglang/srt/function_call/function_call_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,19 @@ class FunctionCallParser:
"""

ToolCallParserEnum: Dict[str, Type[BaseFormatDetector]] = {
"llama3": Llama32Detector,
"qwen25": Qwen25Detector,
"mistral": MistralDetector,
"deepseekv3": DeepSeekV3Detector,
"deepseekv31": DeepSeekV31Detector,
"pythonic": PythonicDetector,
"glm": Glm4MoeDetector,
"glm45": Glm4MoeDetector,
"gpt-oss": GptOssDetector,
"kimi_k2": KimiK2Detector,
"llama3": Llama32Detector,
"mistral": MistralDetector,
"pythonic": PythonicDetector,
"qwen": Qwen25Detector,
"qwen25": Qwen25Detector,
"qwen3_coder": Qwen3CoderDetector,
"glm45": Glm4MoeDetector,
"step3": Step3Detector,
"gpt-oss": GptOssDetector,
}

def __init__(self, tools: List[Tool], tool_call_parser: str):
Expand Down
8 changes: 7 additions & 1 deletion python/sglang/srt/server_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,13 @@ def __post_init__(self):
self._handle_other_validations()

def _handle_deprecated_args(self):
pass
# handle deprecated tool call parsers
deprecated_tool_call_parsers = {"qwen25": "qwen", "glm45": "glm"}
Copy link
Collaborator

@CatherineSue CatherineSue Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw qwen25 still exists in the above map.

Copy link
Collaborator Author

@JustinTong0323 JustinTong0323 Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, here is just for the warning... Maybe remove in future

if self.tool_call_parser in deprecated_tool_call_parsers:
logger.warning(
f"The tool_call_parser '{self.tool_call_parser}' is deprecated. Please use '{deprecated_tool_call_parsers[self.tool_call_parser]}' instead."
)
self.tool_call_parser = deprecated_tool_call_parsers[self.tool_call_parser]
Comment on lines +532 to +536
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Oct 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve readability and avoid multiple dictionary lookups, you can store the new parser name in a variable.

Additionally, the deprecated_tool_call_parsers dictionary is created on each call. Since this is a constant mapping, consider defining it as a module-level constant (e.g., _DEPRECATED_TOOL_CALL_PARSERS) outside this method to avoid repeated creation and improve performance.

Suggested change
if self.tool_call_parser in deprecated_tool_call_parsers:
logger.warning(
f"The tool_call_parser '{self.tool_call_parser}' is deprecated. Please use '{deprecated_tool_call_parsers[self.tool_call_parser]}' instead."
)
self.tool_call_parser = deprecated_tool_call_parsers[self.tool_call_parser]
if self.tool_call_parser in deprecated_tool_call_parsers:
new_parser = deprecated_tool_call_parsers[self.tool_call_parser]
logger.warning(
f"The tool_call_parser '{self.tool_call_parser}' is deprecated. Please use '{new_parser}' instead."
)
self.tool_call_parser = new_parser


def _handle_missing_default_values(self):
if self.tokenizer_path is None:
Expand Down
Loading