Fix Responses API tool type constraint to support function tools.

[Leoyzen]

Motivation

This PR fixes the type constraint issue in SGLang's Response API ResponseTool class that prevented users from using custom function calls via the standard OpenAI Responses API format.

Previously, the ResponseTool class only accepted type="web_search_preview" or type="code_interpreter", causing pydantic validation errors when users tried to provide custom function tools with type="function". This severely limited the usability of SGLang's Responses API, as it didn't match OpenAI's actual API specification which supports both built-in tools and custom function tools.

Example error users encountered:

pydantic_core._pydantic_core.ValidationError: 1 validation error for ResponsesRequest
tools.0.type
  Input should be 'web_search_preview' or 'code_interpreter' [type=literal_error, input_value='function', input_type=str]

This fix using a Union type with discriminator to support both built-in and function tools while maintaining type safety.

Related issue: #10038

Modifications

Core Changes

1. Updated /python/sglang/srt/entrypoints/openai/protocol.py

   • Added ResponseBuiltinTool class to handle built-in tools (web_search_preview, code_interpreter)
   • Added ResponseFunctionTool class to handle custom function tools with name, description, parameters, and strict fields
   • Replaced simple ResponseTool class with a Union type alias using discriminator for safe type handling
   • Import dependencies: typing_extensions.Annotated for Union type annotation

2. Updated /python/sglang/srt/entrypoints/harmony_utils.py

   • Modified get_developer_message() function to accept both list[Tool] (OpenAI type) and list["ResponseTool"] (SGLang type)
   • Added TYPE_CHECKING import to avoid circular import issues
   • The function now correctly processes function tools and extracts their name, description, and parameters attributes

Test Changes

3. Created /test/registered/openai_server/basic/test_response_tool_fix.py
   • Added comprehensive test suite with 5 test cases:
     • test_response_tool_function_type_rejected(): Validates function tool acceptance
     • test_builtin_tools_work(): Ensures backward compatibility with built-in tools
     • test_mixed_tools(): Tests mixing built-in and function tools
     • test_empty_tools(): Verifies empty tool lists work correctly
     • test_function_tool_with_full_schema(): Tests complete function tool schema with nested parameters
   • Added register_cpu_ci(est_time=2, suite="default") marker for CI integration
   • Placed in test/registered/openai_server/basic/ to follow project conventions

Accuracy Tests

N/A - This change only affects request validation and data models. It does not modify model inference, kernel logic, or output generation logic. The existing test suite verifies correct parsing and validation behavior.

Benchmarking and Profiling

N/A - No performance impact. The change uses pydantic's built-in Union type handling with discriminator, which has comparable performance to the previous Literal type constraint. Execution time of ResponsesRequest validation remains ~1.6 seconds as verified by test suite.

Checklist

• Format your code according to Format code with pre-commit.
• Add unit tests according to Run and add unit tests.
• Update documentation according to Write documentations.
  • Documentation update may be needed to reflect the improved Responses API tool support
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Summary of Changes

Hello @Leoyzen, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical type constraint issue within SGLang's Responses API, which previously prevented the use of custom function tools. By adopting a more flexible Pydantic model using Union and a discriminator, the API now fully supports both built-in and user-defined function tools, aligning with standard OpenAI API specifications. This change significantly enhances the API's versatility and enables developers to integrate custom functionalities seamlessly.

Highlights

• Expanded Tool Support: The SGLang Responses API now supports custom function tools (type="function") in addition to existing built-in tools (web_search_preview, code_interpreter), resolving a previous limitation.
• Pydantic Type Refinement: The ResponseTool class was refactored using a Pydantic Union type with a discriminator, Annotated, and TypeAlias to correctly handle different tool structures while maintaining type safety and aligning with OpenAI's API specification.
• Enhanced get_developer_message: The get_developer_message function in harmony_utils.py was updated to correctly process both OpenAI's Tool type and SGLang's new ResponseTool type, ensuring broader compatibility.
• Comprehensive Test Coverage: A new test file, test_response_tool_fix.py, was added with five dedicated test cases to validate the correct parsing and acceptance of various tool configurations, including function tools, built-in tools, mixed lists, and empty tool lists.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a great improvement, fixing a key limitation in the Responses API by allowing type="function" for tools. The use of a discriminated union in protocol.py is well-implemented, and the new tests provide good coverage for the Pydantic models.

However, I've found a critical issue in harmony_utils.py where the code doesn't correctly handle the two different structures for function tools (openai.Tool vs. ResponseFunctionTool), which would lead to a runtime error. My review includes a code suggestion to fix this.

[Leoyzen]

Also fix

• [Bug] Error when using responses api of GPT-OSS model with previous_response_id. #15735
• [ENHANCEMENT] v3.37+ forces native tool calling (XML selector removed) → SGLang openai/gpt-oss-120b (HarmonyParser) returns 500 structure_info not used with HarmonyParser; v3.36.16 works with XML RooCodeInc/Roo-Code#10319
• [Feature] tool_choice required for gpt-oss-120b #9980

[antoine-roux]

I'm very interested in this PR 👍

[JustinTong0323]

/tag-and-rerun-ci

[LuYanFCP]

Thank you very much for your contribution. I have found some issues while using it

1. The tool information was not rendered into the chat_template, resulting in the inability to call it.
2. Output to a structure that has not been converted into response tool output.
   I have fixed these issues and submitted the PR to your branch @Leoyzen
   please check feat: add response request tool render in chat_template and restructure output Leoyzen/sglang#1

[Leoyzen]

@JustinTong0323 Please take a look at this pr.

[Leoyzen]

/gemini review

[gemini-code-assist]

Code Review

This pull request effectively resolves the type constraint issue in the ResponseTool class, enabling the use of custom function tools in the Responses API. The introduction of ResponseBuiltinTool and ResponseFunctionTool with a discriminated union is a clean and robust solution. The accompanying tests are thorough and cover the new functionality well. My review includes a suggestion to refactor some duplicated code for better maintainability and a fix for a minor issue in one of the new tests.

[arieller]

This fix worked for me, thanks!

[bmkor]

Is this issue fixed in the latest build? May I know?