[Piecewise] Use same global graph memory pool as the main cuda graph …

[byjiang1996]

…runner's memory pool

Motivation

As titled

Accuracy Tests and Benchmarking and Profiling

from sglang.srt.entrypoints.http_server import launch_server
from sglang.srt.server_args import prepare_server_args

if __name__ == "__main__":
    # Simulate CLI arguments (excluding the script name)
    args = [
        "--model-path",
        "Qwen/Qwen3-8B",
        "--attention-backend",
        "fa3",
        # "--disable-cuda-graph",
        "--enable-piecewise-cuda-graph",
        "--tp",
        "2"
    ]
    server_args = prepare_server_args(args)
    launch_server(server_args)

When both decode cuda graph and piecewise cuda graph is enabled

python3 benchmark/gsm8k/bench_sglang.py 

Accuracy: 0.955
Invalid: 0.000
Latency: 5.809 s
Output throughput: 4113.320 token/s

When only piecewise cuda graph is enabled but decode cuda graph is disabled

python3 benchmark/gsm8k/bench_sglang.py 

Accuracy: 0.955
Invalid: 0.000
Latency: 15.619 s
Output throughput: 1538.560 token/s

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[ByronHsu]

/tag-and-rerun-ci