Skip to content

[router][grpc] Fix background tasks stored with wrong id#11945

Merged
slin1237 merged 1 commit intomainfrom
chang/responses
Oct 22, 2025
Merged

[router][grpc] Fix background tasks stored with wrong id#11945
slin1237 merged 1 commit intomainfrom
chang/responses

Conversation

@CatherineSue
Copy link
Collaborator

@CatherineSue CatherineSue commented Oct 22, 2025

Motivation

Background tasks were completing successfully but storing responses with the wrong ID:

  • Background response ID: resp_b99907a8-77bd-4c94-8de5-3de9191a11a4
  • Stored response ID: chatcmpl-1e093101-3e87-4913-a3de-0945e93402a5 (internal gRPC request ID)

This caused GET requests to return the original queued response instead of the completed one.

Root Cause:

The chat_to_responses() function was using chat_resp.id (the internal gRPC request ID) instead of the background response_id parameter.

Modifications

  1. Created chat_to_responsesd() that accepts an optional response_id_override
  2. Updated all call sites to use the new function and pass the response_id when available

Accuracy Tests

Before

Get response always shows the status is queued
Screenshot 2025-10-21 at 6 29 01 PM

After

Fixed. Able to get full response
Screenshot 2025-10-21 at 6 28 50 PM

Benchmarking and Profiling

Checklist

@gemini-code-assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@slin1237 slin1237 merged commit 590bc4b into main Oct 22, 2025
59 of 66 checks passed
@slin1237 slin1237 deleted the chang/responses branch October 22, 2025 01:38
xjpang pushed a commit to xjpang/sglang that referenced this pull request Oct 22, 2025
@slin1237 slin1237 mentioned this pull request Oct 23, 2025
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments