Skip to content

[router] Fix flaky router e2e tests #13306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
slin1237 merged 1 commit into from
Nov 15, 2025
Merged

[router] Fix flaky router e2e tests #13306

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions sgl-router/py_test/integration_mock/test_circuit_breaker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,28 @@ def test_circuit_breaker_disable_flag(router_manager, mock_workers):
"disable_retries": True,
},
)
r = requests.post(
f"{rh.url}/v1/completions",
json={
"model": "test-model",
"prompt": "x",
"max_tokens": 1,
"stream": False,
},
timeout=3,
)
assert r.status_code == 500

saw_500 = False
for _ in range(8):
r = requests.post(
f"{rh.url}/v1/completions",
json={
"model": "test-model",
"prompt": "x",
"max_tokens": 1,
"stream": False,
},
timeout=3,
)
if r.status_code == 500:
# Worker starts, continue to check
saw_500 = True
break
assert (
r.status_code == 503
), "Should only see 503 when waiting for worker to start"

assert saw_500
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Nov 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The final assertion assert saw_500 lacks a descriptive message, making test failures harder to debug. Please add a message, similar to the assertion on line 46.

Additionally, this retry logic is duplicated across multiple tests. Consider extracting it into a shared helper function to improve maintainability. The magic number 8 could also be defined as a constant.

Suggested change
assert saw_500
assert saw_500, "Worker did not return status 500 after retries"



@pytest.mark.integration
Expand Down
32 changes: 21 additions & 11 deletions sgl-router/py_test/integration_mock/test_retries.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,25 @@ def test_disable_retries_surfaces_failure(router_manager, mock_workers):
},
)

r = requests.post(
f"{rh.url}/v1/completions",
json={
"model": "test-model",
"prompt": "x",
"max_tokens": 1,
"stream": False,
},
timeout=5,
)
assert r.status_code == 500
saw_500 = False
for _ in range(8):
r = requests.post(
f"{rh.url}/v1/completions",
json={
"model": "test-model",
"prompt": "x",
"max_tokens": 1,
"stream": False,
},
timeout=5,
)
if r.status_code == 500:
# Worker starts, continue to check
saw_500 = True
break
assert (
r.status_code == 503
), "Should only see 503 when waiting for worker to start"

assert saw_500
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Nov 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This assertion would be more informative with a failure message to aid in debugging. As noted in the review for test_circuit_breaker.py, this retry logic is also a candidate for refactoring into a shared helper function to reduce code duplication.

Suggested change
assert saw_500
assert saw_500, "Worker did not return status 500 after retries"

# mock_workers fixture handles cleanup
Loading