[router] Add mTLS Support for Router-to-Worker Communication

[slin1237]

Motivation

This PR adds mutual TLS (mTLS) authentication support for secure communication between the SGLang Model Gateway and worker nodes. This enables the router to authenticate itself to workers using client certificates and verify worker identities using custom Certificate Authorities (CAs).

In production deployments, especially in enterprise or multi-tenant environments, secure communication between the gateway and workers is critical. Typically this is done via service mesh. This feature is dedicated when service mesh is no an option.

Modification

Configuration (Python)

Added three new CLI arguments in router_args.py:

• --client-cert-path: Path to the router's client certificate for authenticating to workers
• --client-key-path: Path to the router's private key corresponding to the client certificate
• --ca-cert-path: Path(s) to CA certificate(s) for verifying worker TLS certificates (supports multiple CAs)

Core Implementation (Rust)

Configuration (config/types.rs, lib.rs):

• Added client_identity field to store combined certificate + private key (PEM format)
• Added ca_certificates field to store one or more CA certificates (PEM format)
• Certificates are loaded and validated during router initialization
• Validation ensures both cert and key are provided together or neither

HTTP Client (server.rs):

• Modified HTTP client builder to configure mTLS when certificates are provided
• Client identity is set using reqwest::Identity::from_pem()
• CA certificates are added using add_root_certificate()
• Added informative logging when mTLS features are enabled

Usage Example

  python -m sglang_router.launch_router \
    --worker-urls https://worker1:8000 https://worker2:8000 \
    --client-cert-path /path/to/router-cert.pem \
    --client-key-path /path/to/router-key.pem \
    --ca-cert-path /path/to/ca1.pem /path/to/ca2.pem

Known Limitations

The current implementation creates a single HTTP client for all workers, which works well for
deployments where:

• All workers share the same CA
• All workers accept the same client certificate (single security domain)

For multi-domain deployments (e.g., different model families with different CAs), the
architecture would need refactoring to support per-worker HTTP clients. See the detailed FIXME
comment in server.rs:801 for the required changes.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @slin1237, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces mutual TLS (mTLS) support for secure communication between the SGLang router and its worker nodes. This enhancement is crucial for production environments, particularly where a service mesh is not utilized, by allowing the router to authenticate itself to workers using client certificates and to verify worker identities against specified Certificate Authorities. The changes involve adding new configuration options in Python and integrating the mTLS setup into the Rust core, specifically within the HTTP client builder.

Highlights

• mTLS Configuration via CLI: New command-line arguments (--client-cert-path, --client-key-path, --ca-cert-path) have been added to configure mTLS for router-to-worker communication, allowing specification of client certificates and CA certificates for verification.
• Rust Core Implementation for mTLS: The RouterConfig and Router structs in Rust now include fields to store the combined client identity (certificate and key) and CA certificates. Validation ensures that both client certificate and key are provided together if mTLS is enabled.
• Secure HTTP Client Configuration: The HTTP client responsible for communicating with worker nodes is now configured to use mTLS when certificate paths are provided. This enables mutual authentication, where the router authenticates itself to workers and verifies worker identities using the specified CA certificates.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces mTLS support for secure communication between the router and workers, which is a valuable security enhancement. The implementation is well-structured, adding new CLI arguments in Python and handling certificate loading and client configuration in Rust. My review identified a critical issue in the Python argument parsing that prevents CA certificates from being loaded correctly. Additionally, I've pointed out two high-severity error handling issues in the Rust code that could cause the server to panic on startup if certificate files are malformed. Addressing these points will significantly improve the robustness of this new feature.