Skip to content

Fix quantization and nightly tests#4258

Merged
merrymercy merged 4 commits intomainfrom
pr-fix-compressed-fp8
Mar 10, 2025
Merged

Fix quantization and nightly tests#4258
merrymercy merged 4 commits intomainfrom
pr-fix-compressed-fp8

Conversation

@merrymercy
Copy link
Contributor

@merrymercy merrymercy commented Mar 10, 2025 •
edited
Loading

Monkey patch functions like AWQMoEMethod from vllm to align the arguments

python -m sglang.launch_server --model-path cognitivecomputations/DeepSeek-V3-AWQ --tp-size 8

@merrymercy merrymercy merged commit 00d25a7 into main Mar 10, 2025
22 of 23 checks passed
@merrymercy merrymercy deleted the pr-fix-compressed-fp8 branch March 10, 2025 10:06
aoshen524 pushed a commit to aoshen524/sglang that referenced this pull request Mar 10, 2025
@merrymercy @ShenAo1111
Fix quantization and nightly tests (sgl-project#4258)
95145b6
@Fridge003 Fridge003 mentioned this pull request Mar 13, 2025
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@zhyncs zhyncs Awaiting requested review from zhyncs

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@merrymercy