Skip to content

[AMD] Add DLLM support for AMD GPUs with LLaDA2 testing#15560

Merged
mickqian merged 6 commits intosgl-project:mainfrom
sunxxuns:test-dllm-amd
Jan 3, 2026
Merged

[AMD] Add DLLM support for AMD GPUs with LLaDA2 testing#15560
mickqian merged 6 commits intosgl-project:mainfrom
sunxxuns:test-dllm-amd

Conversation

@sunxxuns
Copy link
Collaborator

@sunxxuns sunxxuns commented Dec 21, 2025

enable basic text diffusion model for amd cards.

Launch Command

python -m sglang.launch_server \
--model-path inclusionAI/LLaDA2.0-mini \
--dllm-algorithm LowConfidence \
--trust-remote-code \
--mem-fraction-static 0.9 \
--max-running-requests 1 \
--attention-backend triton \
--host 0.0.0.0 \
--port 30000

Test Command

curl -s http://127.0.0.1:30000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "inclusionAI/LLaDA2.0-mini",
    "prompt": "Write a 1-sentence summary of what SGLang is.",
    "temperature": 0.2,
    "max_tokens": 64,
    "stream": false
  }' | jq .

@gemini-code-assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions bot added the amd label Dec 21, 2025
@sunxxuns sunxxuns force-pushed the test-dllm-amd branch 13 times, most recently from ee83ee2 to 20269e3 Compare December 21, 2025 06:49
@sunxxuns sunxxuns changed the title [AMD] Test DLLM (LLaDA2) on MI300 GPUs [AMD] Add DLLM support for AMD GPUs with LLaDA2 testing Dec 21, 2025
@ClawSeven
Copy link
Contributor

@sunxxuns Hi, great work on adding dLLM support for AMD! Your code can serve as a reference for other hardware implementations. I’ve added the link to the dLLM roadmap so others can follow your approach.

Copy link
Collaborator

@yushengsu-thu yushengsu-thu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main added parts are in CI/D and will not affect the functions on other hardware. but might need to rebase

@sunxxuns sunxxuns enabled auto-merge (squash) January 2, 2026 00:52
@sunxxuns
Copy link
Collaborator Author

sunxxuns commented Jan 2, 2026

The main added parts in on CI/D and will not affect the functions on other hardware. but might need to rebase

rebased, pls merge if no issue to avoid next rebase.

@mickqian mickqian disabled auto-merge January 3, 2026 02:05
@zyzshishui
Copy link
Contributor

/rerun-failed-ci

@mickqian mickqian merged commit 8b869e3 into sgl-project:main Jan 3, 2026
215 of 229 checks passed
yingluosanqian pushed a commit to yingluosanqian/sglang that referenced this pull request Jan 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments