Add DeepSeek-V4-Pro FP8 MI355X SGLang benchmark

[Oseltamivir]

Summary

• Add dsv4-fp8-mi355x-sglang benchmark config for DeepSeek-V4-Flash FP8 on AMD MI355X with SGLang
• New benchmark script benchmarks/single_node/dsv4_fp8_mi355x.sh with DSv4-specific env vars and server args from sgl-project/sglang#23608
• Image: lmsysorg/sglang:deepseek-v4-mi350 (custom build from PR branch), Model: sgl-project/DeepSeek-V4-Flash-FP8
• Compressed attention backend, DP attention enabled, TP=4/DP=4, CUDA graph disabled
• Search space: TP=4, concurrency 4-64 for 1k1k and 8k1k

Test plan

• generate_sweep_configs.py full-sweep --config-files .github/configs/amd-master.yaml --model-prefix dsv4 generates 10 entries
• pytest utils/matrix_logic/ -v — 149/149 tests pass
• Verify benchmark runs on MI355X hardware

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[claude]

🟡 Two SGLang flags in the new script use non-canonical abbreviated forms: --dp $TP on line 57 (should be --dp-size) and --max-running-request 256 on line 62 (should be --max-running-requests, plural). Every other SGLang benchmark script in this repo uses the canonical --dp-size/--data-parallel-size and --max-running-requests (e.g. dsr1_fp8_b200.sh, glm5_fp8_b200.sh, qwen3.5_fp8_b200.sh, multi_node/amd_utils/server.sh). These work today only because argparse's default allow_abbrev=True resolves them as unambiguous prefix matches — they would silently break on a future SGLang release that adds another --dp-* flag (e.g. --dp-lb-port) or on a build that sets allow_abbrev=False, either via an "ambiguous option" error at server startup or by not applying the intended 256 running-request cap / DP=TP setting.

Extended reasoning...

The bug

At benchmarks/single_node/dsv4_fp8_mi355x.sh:57 and :62, two SGLang launch_server flags are written in non-canonical abbreviated form:

--tensor-parallel-size $TP \
--dp $TP \                    # line 57 — should be --dp-size $TP
...
--max-running-request 256 \    # line 62 — should be --max-running-requests 256

SGLang's ServerArgs registers exactly one option for each: --dp-size (aliased as --data-parallel-size) and --max-running-requests (plural). Neither --dp nor --max-running-request is a registered alias. Every other benchmark script in this repo uses the canonical names — multi_node/amd_utils/server.sh uses --dp-size, 25+ single-node scripts use --data-parallel-size, and 30+ scripts use --max-running-requests (plural). Within this very file, line 56 already uses the full --tensor-parallel-size form, so --dp on line 57 is inconsistent even locally.

Why it happens to work today

Python's argparse defaults to allow_abbrev=True. As long as SGLang registers only one option starting with each prefix, argparse resolves --dp → --dp-size and --max-running-request → --max-running-requests as unambiguous prefix matches, and the values are applied. So on the current custom image lmsysorg/sglang:deepseek-v4-mi350 this most likely runs successfully.

Why it is still worth fixing

The other verifier correctly pointed out that the breakage scenarios are hypothetical. That objection is fair as stated — argparse would raise ambiguous option rather than silently pick the wrong flag, and today's behavior is correct. But the script is a new, one-line-each fix and sits on top of a custom fork build from sgl-project/sglang#23608, not a stable release. Two concrete fragilities remain:

1. Upstream SGLang could add a sibling flag at any time. If a future version introduces any --dp-* option (e.g. --dp-lb-port, --dp-balance-mode) or --max-running-request-* option, argparse will raise ambiguous option: --dp could match --dp-size, --dp-lb-port at server startup, and the benchmark will silently fail to launch. Since this script's image is a PR-branch build that will be rebased onto upstream, this is a real risk window, not purely hypothetical.
2. A build with allow_abbrev=False would reject both flags. The linked PR #23608 could set this on its parser; if it ever does, the server fails with unrecognized arguments: --dp --max-running-request 256.

Step-by-step proof the canonical forms are required

1. Grep upstream python/sglang/srt/server_args.py: only --dp-size / --data-parallel-size and --max-running-requests are registered. No --dp alias. No --max-running-request alias.
2. Grep this repo for --dp (bare): only hit is dsv4_fp8_mi355x.sh:57. All 25+ other scripts use --dp-size or --data-parallel-size.
3. Grep this repo for --max-running-request (singular): only hit is dsv4_fp8_mi355x.sh:62. All 30+ other scripts use --max-running-requests (plural).
4. Concrete failure example: if upstream SGLang merges a --dp-lb-port flag next release and this image is rebuilt, running this script produces:

   sglang.launch_server: error: ambiguous option: --dp could match --dp-size, --dp-lb-port
   

   and the benchmark aborts before wait_for_server_ready completes.

Fix

Two trivial renames on lines 57 and 62:

-    --dp $TP \
+    --dp-size $TP \
...
-    --max-running-request 256 \
+    --max-running-requests 256 \

This brings the script in line with every other benchmark in benchmarks/ and removes both fragilities.