[Feature] NVIDIA DGX Spark (GB10, sm_121a) Support Tracking

[yvbbrjdr]

This issue tracks the progress of adding support for the NVIDIA DGX Spark system (GB10, sm_121a).

The benchmark results published in the LMSYS blog post, along with the Docker image tag for Spark (lmsysorg/sglang:spark), were produced using a custom SGLang snapshot from my personal development branch: main...yvbbrjdr:sglang:spark.

The branch currently includes several temporary workarounds that need to be properly addressed before it can be merged into main. These include:

1. Outdated base: The branch is approximately two weeks old, and rebasing onto main may not succeed cleanly or work properly.
2. PyTorch compatibility: The official PyTorch release does not yet support CUDA 13.0, so a nightly build was used in a custom Dockerfile.
3. Triton issue: Running GPT-OSS models triggers PTXAS compilation error: '.tile::gather4 with destination state space as .shared::cluster' not supported on target 'sm_121a' triton-lang/triton#8335, which remains unresolved.
4. FP8 kernel dispatch: FP8 CUTLASS kernels currently fail to dispatch on GB10 (sm_121a). As a temporary workaround, they are disabled on this branch, causing PyTorch to fall back to legacy FP8 inference kernels with reduced performance.
5. Dependency status: All external dependencies (except for sgl-kernel, which must be rebuilt for sm_121a) have been disabled due to unknown compatibility. It’s unclear which git tags or commits of these dependencies are compatible with the GB10 architecture.

Additionally, @johnnynunez has opened related PRs for CUDA 13 and FA4 support, which may help resolve some of the above issues: #11299, #11606 (Thank you!)

[yvbbrjdr]

My next step: Rebase the branch onto main and resolve any additional issues introduced during the process, ensuring we have a clean baseline to continue development and integration work. Also make sure all issues raised above still hold.

[johnnynunez]

My next step: Rebase the branch onto main and resolve any additional issues introduced during the process, ensuring we have a clean baseline to continue development and integration work. Also make sure all issues raised above still hold.

pytorch 2.9.0 comes today that resolve some issues with cu13

[Kh4L]

Also relevant PR #11287

[LuYanFCP]

Sglang NvFP4 Kernel in DGX Spark will assertion failed like

relative issue: #11725

[Azure-Tang]

Hi, I tried to run moe nvfp4 model on spark, it can decode for a while but then crush with error:

# my run command
python3 -m sglang.launch_server \
    --model-path GLM-4.5-Air-nvfp4  \
    --host 0.0.0.0 --port 30000 \
    --disable-radix-cache \
    --chunked-prefill-size 1024 \
    --max-running-requests 20 \
    --cuda-graph-max-bs 10 \
    --max-total-tokens 32000 \
    --mem-fraction-static 0.75 \
    --moe-runner-backend flashinfer_cutlass

Does anyone know how to solve this? It seems like error from flashinfer.

[eugr]

@yvbbrjdr - Any progress on this? The fork / spark-specific container works with gpt-oss-120b, but is missing some other important improvements. For instance, this one: #12724 which prevents the spark version from compiling CUDA graphs on dual Spark configuration.

I applied the patch locally, but it would be good to merge Spark support into main at some point.

Looks like the Spark version uses a custom Triton build too - most likely the performance boost comes from those changes - any plans to merge those into the triton main?

[shaunsingh]

Question, will it be possible to enable trtllm MHA & MLA on the spark? errors saying it is reserved for Blackwell GPUs as it stands, but the spark is blackwell-based

[eugr]

Any update on this?

[ganisback]

any updates?

[eugr]

@yvbbrjdr - have I missed any PRs? I don't see any relevant commits...

[johnnynunez]

@yvbbrjdr - have I missed any PRs? I don't see any relevant commits...

What is not working for you?

[eugr]

@johnnynunez - well, the changes they've made to speed up gpt-oss on Spark are not merged.

Also the official Docker image would still not run on Spark, although I didn't check in the last two weeks.