[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE

[ch-wan]

Motivation

Modifications

Accuracy Test

Benchmark & Profiling

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[trevor-m]

This commit is causing 0 accuracy with modelopt FP4 moe. Will try to debug

[zhyncs]

Hi @trevor-m sorry for this. And thanks for your help!

[hebiao064]

do we have 5/N?

[ch-wan]

@trevor-m Sorry for causing inconvenience. I was trying to test DeepSeek-FP4 before merging this PR but confronted issues for loading the model.

  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 637, in load_model
    self.model = get_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/__init__.py", line 22, in get_model
    return loader.load_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 559, in load_model
    model = _initialize_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 171, in _initialize_model
    quant_config = _get_quantization_config(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 127, in _get_quantization_config
    quant_config = get_quant_config(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/weight_utils.py", line 211, in get_quant_config
    if config["producer"]["name"] == "modelopt":
KeyError: 'producer'

Here is my command:

python3 -m sglang.launch_server  --model-path Barrrrry/DeepSeek-R1-W4AFP8   --context-length 8192  --tp 2 --ep 2   --trust-remote-code   --enable-ep-moe

Could you share how to run this model so that I can avoid a similar issue happen again?

[trevor-m]

@trevor-m Sorry for causing inconvenience. I was trying to test DeepSeek-FP4 before merging this PR but confronted issues for loading the model.

  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 637, in load_model
    self.model = get_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/__init__.py", line 22, in get_model
    return loader.load_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 559, in load_model
    model = _initialize_model(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 171, in _initialize_model
    quant_config = _get_quantization_config(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/loader.py", line 127, in _get_quantization_config
    quant_config = get_quant_config(
  File "/sgl-workspace/sglang/python/sglang/srt/model_loader/weight_utils.py", line 211, in get_quant_config
    if config["producer"]["name"] == "modelopt":
KeyError: 'producer'

Here is my command:

python3 -m sglang.launch_server  --model-path Barrrrry/DeepSeek-R1-W4AFP8   --context-length 8192  --tp 2 --ep 2   --trust-remote-code   --enable-ep-moe

Could you share how to run this model so that I can avoid a similar issue happen again?

No worries! Here is a command python3 -m sglang.launch_server --model-path nvidia/DeepSeek-R1-0528-FP4 --trust-remote-code --quantization modelopt_fp4 --tp 4 --enable-flashinfer-cutlass-moe --enable-ep-moe --ep-size 4
`