Clarify support for tool calling and OpenAI Responses API

[mickvangelderen]

I'm trying out sglang to serve gpt-oss-120b instead of vLLM which I've been using so far because the documentation seemed to suggest that sglang supports tool calling and the OpenAI Responses API, although I can't find it anymore. I remember there being a compatibility table somewhere a few days ago that suggested tool calling was supported with the Responses API (but not chat?).

It would be helpful if the documentation is updated with an overview of what is supported exactly. In my testing I noticed that currently the built in tools are supported but not user defined client side tools: Input should be 'web_search_preview' or 'code_interpreter' [type=literal_error, input_value='function', input_type=str].

[2025-09-04 19:42:42] ERROR:    Exception in ASGI application                                                                                                                           │
│ Traceback (most recent call last):                                                                                                                                                      │
│   File "/usr/local/lib/python3.12/dist-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi                                                                              │
│     result = await app(  # type: ignore[func-returns-value]                                                                                                                             │
│              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                             │
│   File "/usr/local/lib/python3.12/dist-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__                                                                              │
│     return await self.app(scope, receive, send)                                                                                                                                         │
│            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                         │
...
│   File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/http_server.py", line 1049, in v1_responses_request                                                                         │
│     request_obj = ResponsesRequest(**request)                                                                                                                                           │
│                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                           │
│   File "/usr/local/lib/python3.12/dist-packages/pydantic/main.py", line 253, in __init__                                                                                                    │
│     validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)                                                                                              │
│                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                              │
│ pydantic_core._pydantic_core.ValidationError: 1 validation error for ResponsesRequest                                                                                                   │
│ tools.0.type                                                                                                                                                                            │
│   Input should be 'web_search_preview' or 'code_interpreter' [type=literal_error, input_value='function', input_type=str]                                                               │
│     For further information visit https://errors.pydantic.dev/2.11/v/literal_error          

[Swipe4057]

how do you start the server?

[JustinTong0323]

I think tool calling is supported with the chat api, not the responses.

[bfroemel]

sglang/python/sglang/srt/entrypoints/openai/protocol.py

Line 717 in f40038f

type: Literal["web_search_preview", "code_interpreter"] = Field(

But probably more is needed than just removing the Literal check...

I think tool calling is supported with the chat api, not the responses.

Could someone maybe offer a short openai Python snippet to demonstrate (custom/mcp) gpt-oss tool-calling via sglang? I'd also prefer the much more powerful responses API (https://platform.openai.com/docs/guides/migrate-to-responses), but I take anything at this point :)

[mickvangelderen]

how do you start the server?

@Swipe4057

I'm using the sglang docker image @ latest, image id:

docker.io/lmsysorg/sglang@sha256:323057ec2fbac78c47b043ac80cf756724e201b43067fb1d441eba1e7f97d01d

With the following args:

python -m sglang.launch_server --model-path openai/gpt-oss-120b --host 0.0.0.0 --port 80 --download-dir /remote/vllm/ --tp-size 2 --tool-call-parser gpt-oss

[segamza]

I'm also interested in this question, is there any progress?

[edv-sml]

same here... tool-calling is sometimes possible but most of the time not.. also using the latest image.
had some problems with vllm 0.10.1 too then switcht to sglang. sglang did it better but not perfect. now i tried vllm 0.11.0 - works perfectly. but sadly vllm uses more vram than sglang and i can use my gpt-oss-120b with only 32k context window with vllm.

with sglang i can use the whole 131k context window with my 96gb vram - so hoping that the sglang team will implement the fix soon.

i'll stick to sglang because of the better performance and less vram usage compared to vllm.

tried tool-calling with roo-code, claude-cli, kilo code, cline everywhere the same problems with sglang and the harmony/tool-call parser

[bfroemel]

@CatherineSue seeing your extensive contributions at least in this related PR #8837 could you maybe take a look at this issue? Many thanks!

In case someone just wants to test gpt-oss-120b's tool calling capabilities: try llama.cpp with codex-cli (but chat completions api). simply amazing and to my knowledge the best experience you can have right now locally with that model (and simple tools that have only required parameters or few/mutual exclusive optional parameters).

Unfortunately there is more to tool calling than is immediately obvious (harmony, json schema-aware sampling of tool calls) + subtle gotchas (e.g., even with llama.cpp tool calls may be produced incorrectly, if optional parameters are generated out of order - a problem if a tool is more complex like goose's developer__text_editor, str_replace).

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.