[Feature] sglang-router shouldn't be ready when there is no worker ready or registered.

[gongwei-130]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

currently sglang-router would be ready to take traffic no matter there is any worker ready behind it or not. Usually worker would take longer time to to start, router shouldn't be ready unless there is at least one worker or 1 set of pd worker in pd disagg setup.

Related resources

No response

[zhyncs]

At least one pair P:D is ready.

[slin1237]

that's not how gateway works
how do you define ready when you have different model family and serving mode behind the gateway
which pair, or which model then should be deemed more important

[slin1237]

for example, when you deploy Nginx, Istio gateway or whatever gateway
Would they be not ready if the ingress resource isn't ready?
what is needed is probably continues checking for working during startup and after
until they are removed explicitly

[zhyncs]

@slin1237

how do you define ready

When server shows The server is fired up and ready to roll!

[slin1237]

#12252