Skip to content

[Bug] GRPC Router issue with MCP Tool handling #12669

@ankrovv

Description

@ankrovv

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

When invoking GPT-OSS-120B through the SGLang Router (gRPC mode) with an MCP tool, the response ends after the last mcp_call output item and does not produce a final "type": "message" item summarizing or integrating the tool results.

According to the OpenAI Responses API behavior, the router should perform a second iteration after the tool call, feeding the tool outputs back into the model to generate the final output message.

Reproduction

Request

curl -X POST http://localhost:8082/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "/raid/models/openai/gpt-oss-120b",
    "input": "Using the DeepWiki MCP server, tell me: what transport protocols does the 2025-03-26 version of the MCP spec support?",
    "tool_choice": "auto",
    "tools": [
      {
        "type": "mcp",
        "server_url": "https://mcp.deepwiki.com/mcp",
        "server_label": "deepwiki",
        "require_approval": "never"
      }
    ]
  }'

Output

"output": [
  { "type": "mcp_list_tools", ... },
  { "type": "reasoning", ... },
  { "type": "mcp_call", ... }
#missing final output response
]

Environment

Python: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
CUDA available: True
GPU 0,1,2,3,4,5,6,7: NVIDIA A100-SXM4-80GB
GPU 0,1,2,3,4,5,6,7 Compute Capability: 8.0
CUDA_HOME: /usr/local/cuda-12.2
NVCC: Cuda compilation tools, release 12.2, V12.2.140
CUDA Driver Version: 550.54.14
PyTorch: 2.8.0+cu128
sglang: 0.5.4.post2
sgl_kernel: 0.3.16.post4
flashinfer_python: 0.5.0
triton: 3.4.0
transformers: 4.57.1
torchao: 0.9.0
numpy: 1.26.4
aiohttp: 3.9.3
fastapi: 0.109.2
hf_transfer: 0.1.9
huggingface_hub: 0.36.0
interegular: 0.3.3
modelscope: 1.31.0
orjson: 3.11.4
outlines: 0.1.11
packaging: 25.0
psutil: 5.9.8
pydantic: 2.12.3
python-multipart: 0.0.20
pyzmq: 27.1.0
uvicorn: 0.27.1
uvloop: 0.19.0
vllm: Module Not Found
xgrammar: 0.1.25
openai: 2.6.1
tiktoken: 0.12.0
anthropic: 0.72.0
litellm: Module Not Found
decord2: 2.0.0
NVIDIA Topology:
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 NIC0 NIC1 NIC2 NIC3 NIC4 NIC5 NIC6 NIC7 NIC8 NIC9 NIC10 NIC11 NIC12 NIC13NIC14 NIC15 NIC16 NIC17 CPU Affinity NUMA Affinity GPU NUMA ID
GPU0 X NV12 NV12 NV12 NV12 NV12 NV12 NV12 SYS SYS SYS SYS SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS 48-63,176-191 3 N/A
GPU1 NV12 X NV12 NV12 NV12 NV12 NV12 NV12 SYS SYS SYS SYS SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS 48-63,176-191 3 N/A
GPU2 NV12 NV12 X NV12 NV12 NV12 NV12 NV12 SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS 16-31,144-159 1 N/A
GPU3 NV12 NV12 NV12 X NV12 NV12 NV12 NV12 SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS 16-31,144-159 1 N/A
GPU4 NV12 NV12 NV12 NV12 X NV12 NV12 NV12 SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PXB PXB 112-127,240-254 7 N/A
GPU5 NV12 NV12 NV12 NV12 NV12 X NV12 NV12 SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PXB PXB 112-127,240-254 7 N/A
GPU6 NV12 NV12 NV12 NV12 NV12 NV12 X NV12 SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS 80-95,208-223 5 N/A
GPU7 NV12 NV12 NV12 NV12 NV12 NV12 NV12 X SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PXB PXB SYS SYS SYS SYS SYS 80-95,208-223 5 N/A
NIC0 SYS SYS SYS SYS SYS SYS SYS SYS X SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC1 SYS SYS PXB PXB SYS SYS SYS SYS SYS X PIX PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC2 SYS SYS PXB PXB SYS SYS SYS SYS SYS PIX X PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC3 SYS SYS PXB PXB SYS SYS SYS SYS SYS PXB PXB X PIX SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC4 SYS SYS PXB PXB SYS SYS SYS SYS SYS PXB PXB PIX X SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC5 PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS X PIX PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC6 PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PIX X PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC7 PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB X PIX SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC8 PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PIX X SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC9 SYS SYS SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS X PIX PXB PXB SYS SYS SYS SYS SYS
NIC10 SYS SYS SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS PIX X PXB PXB SYS SYS SYS SYS SYS
NIC11 SYS SYS SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB X PIX SYS SYS SYS SYS SYS
NIC12 SYS SYS SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PIX X SYS SYS SYS SYS SYS
NIC13 SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS X SYS SYS SYS SYS
NIC14 SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS X PIX PXB PXB
NIC15 SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PIX X PXB PXB
NIC16 SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB X PIX
NIC17 SYS SYS SYS SYS PXB PXB SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS SYS PXB PXB PIX X

Legend:

X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks

NIC Legend:

NIC0: mlx5_0
NIC1: mlx5_1
NIC2: mlx5_2
NIC3: mlx5_3
NIC4: mlx5_4
NIC5: mlx5_5
NIC6: mlx5_6
NIC7: mlx5_7
NIC8: mlx5_8
NIC9: mlx5_9
NIC10: mlx5_10
NIC11: mlx5_11
NIC12: mlx5_12
NIC13: mlx5_13
NIC14: mlx5_14
NIC15: mlx5_15
NIC16: mlx5_16
NIC17: mlx5_17

ulimit soft: 1024

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions