Dsv32 opt2 by ZhengdQin · Pull Request #38 · ZhengdQin/sglang

ZhengdQin · 2026-01-16T06:14:35Z

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.
Work with maintainers to merge your PR. See the PR Merge Process

…4440) Co-authored-by: key4ng <rukeyang@gmail.com>

…r chunked execution on H200 (sgl-project#14325)

…ct#14441) Co-authored-by: key4ng <rukeyang@gmail.com>

Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com> Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>

…ion (sgl-project#8287) Signed-off-by: Kay Yan <kay.yan@daocloud.io>

…roject#14446) Co-authored-by: key4ng <rukeyang@gmail.com>

Co-authored-by: Chang Su <chang.s.su@oracle.com>

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: Yueming Yuan <yy28@illinois.edu>

sgl-project#14432)

…ct#14421) Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

…gl-project#14463)

…#14484)

…t#14472)

…d-fields (sgl-project#13996) Co-authored-by: dev <devnull@example.com> Co-authored-by: Mick <mickjagger19@icloud.com>

…roject#14364) Co-authored-by: niehen6174 <niehen.6174@gmail.com> Co-authored-by: Mick <mickjagger19@icloud.com>

… pipeline (sgl-project#14499)

…gl-project#14456)

Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>

…ng params as default behavior (sgl-project#14471)

…sgl-project#14856)

…4877) Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>

…roject#14880)

…low-up) (sgl-project#14853)

…roject#13641) Co-authored-by: jianyingzhu <53300651@qq.com> Co-authored-by: Jianying <53503712+jianyingzhu@users.noreply.github.com> Co-authored-by: root <root@2u2g-spr-0417.ipp4a1.colossus.nvidia.com> Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>

…4878)

slin1237 and others added 30 commits December 4, 2025 10:16

[model-gateway] extract conversation out of oai router (sgl-project#1…

7f3308b

…4440) Co-authored-by: key4ng <rukeyang@gmail.com>

[DeepseekV3.2][NSA][Indexer] Fix PAGED top-k transform for NSA indexe…

7dfcc78

…r chunked execution on H200 (sgl-project#14325)

[model-gateway] move oai header util to router header util (sgl-proje…

d8faf2f

…ct#14441) Co-authored-by: key4ng <rukeyang@gmail.com>

[FIX] trtllm-moe-fp4-renorm for Qwen series models (sgl-project#14350)

922756a

add doc for quantized kv cache (sgl-project#14348)

88d1bab

Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com> Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>

fix: Correct environment variable syntax in docker-compose configurat…

0e6441b

…ion (sgl-project#8287) Signed-off-by: Kay Yan <kay.yan@daocloud.io>

[model-gateway] move all responses api event from oai to proto (sgl-p…

eb85fa6

…roject#14446) Co-authored-by: key4ng <rukeyang@gmail.com>

[model-gateway] add mistral 3 image processor (sgl-project#14445)

29c6c2e

Co-authored-by: Chang Su <chang.s.su@oracle.com>

[model-gateway] grpc to leverage event type (sgl-project#14450)

c1006fd

Co-authored-by: Chang Su <chang.s.su@oracle.com>

ministral3 (sgl-project#14251)

6d37e70

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: Yueming Yuan <yy28@illinois.edu>

[Bug] fix not desired disable fused share experts caused by rocm logic (

2ecee75

sgl-project#14432)

Rename secrets.WHL_TOKEN -> secrets.GH_PAT_FOR_WHL_RELEASE (sgl-proje…

b5d3998

…ct#14421) Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

[diffusion] improve: further optimize model load (sgl-project#13836)

fa0ca97

Add CI permissions for user 'yushengsu-thu' (sgl-project#14468)

532037d

[ez] Fix typing (sgl-project#14473)

41429a8

Add AMD stage support to /rerun-stage command and fix related bugs (s…

4c5074e

…gl-project#14463)

Add YAMY1234 to CI Permission (sgl-project#14475)

80a575e

clean up gemlite usage (sgl-project#14444)

b76e303

[diffusion] chore: further improve model searching logic (sgl-project…

beec8ee

…#14484)

[diffusion] fix: fix bug about pin memory when offloading (sgl-projec…

46b05ef

…t#14472)

[diffusion] cli: add argument --adjust-frames and --override-protecte…

7c744d1

…d-fields (sgl-project#13996) Co-authored-by: dev <devnull@example.com> Co-authored-by: Mick <mickjagger19@icloud.com>

dockerfile: add runtime stage + ubuntu 24.04 (sgl-project#13861)

498ea41

[diffusion] fix: fix CLIP text encoder attention mask not used (sgl-p…

35ba6fe

…roject#14364) Co-authored-by: niehen6174 <niehen.6174@gmail.com> Co-authored-by: Mick <mickjagger19@icloud.com>

Enable RadixCache for Mamba2 models (sgl-project#13584)

2ce121a

[diffusion] fix: Fix profiler trace missing Python stack in diffusion…

5347732

… pipeline (sgl-project#14499)

support GLM-V vision model dp (sgl-project#14097)

8fce9e7

[misc] add model arch and type to server info and use it for harmony (s…

7235a7f

…gl-project#14456)

Add Mistral Large 3 Eagle Support (sgl-project#14466)

205f041

Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>

Add Mistral Large 3 to nightly CI tests (sgl-project#14459)

6628098

[diffusion] chore: set allowing overriding protected fields of sampli…

a890456

…ng params as default behavior (sgl-project#14471)

Kangyan-Zhou and others added 18 commits December 10, 2025 23:17

Only count limitations for previous runs that reaches the test stages (…

2856624

…sgl-project#14856)

[CI][BUG] fix ib setup for disaggregation hicache test (sgl-project#1…

8348725

…4877) Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>

[Fix] Remove unused import from test_disaggregation_hicache.py (sgl-p…

a076d75

…roject#14880)

fix: adding temporary bypass for nightly tests (sgl-project#14876)

e52cf30

Avoid deleting entire cache for missing shards (sgl-project#14754 fol…

f85460f

…low-up) (sgl-project#14853)

Tiny add more error info for bench_serving (sgl-project#14827)

a368df2

Tiny support range ratio in GSP in bench serving (sgl-project#14828)

45eeeb9

[NPU] adapt dsv3.2 nsa prefill context parallel (sgl-project#14541)

388018a

[diffusion] feat: support sageattn & sageattn3 backend (sgl-project#1…

5d804a3

…4878)

dsv32 multistream opt

8f980dc

clean code

8e78e2e

delete renormalize in topk

a88f8a2

dsv32 use batch_matmul_transpose in MTP

12afaab

modify comment

28cd3b5

Support dynamic w8a8

bf4dee5

dsv3 support ascend_fuseep

9391ed0

rebase modify

f6f61a2

github-actions bot added documentation Improvements or additions to documentation quant deepseek model-gateway sgl-kernel dependencies Multi-modal lora speculative-decoding amd hicache labels Jan 16, 2026

ZhengdQin merged commit 9f0a745 into main Jan 16, 2026
50 of 66 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Dsv32 opt2#38

Dsv32 opt2#38
ZhengdQin merged 705 commits intomainfrom
dsv32_opt2

ZhengdQin commented Jan 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Comments

Conversation

ZhengdQin commented Jan 16, 2026

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants