feat: Improve LoRA compatibility by adding unified format detection and diffusers-based normalization#14659
Merged
mickqian merged 6 commits intosgl-project:mainfrom Dec 13, 2025
Merged
Conversation
Contributor
Summary of ChangesHello @MikukuOvO, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the SGLang multimodal generation pipeline's ability to handle a wide array of LoRA (Low-Rank Adaptation) checkpoint formats. By implementing a robust detection and normalization layer, it ensures that LoRAs from various sources, which often have inconsistent naming conventions, can be seamlessly integrated and utilized. This change aims to improve the overall reliability and extensibility of LoRA loading without affecting inference performance or accuracy. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a significant improvement to LoRA compatibility in SGLang by adding a unified format adapter. The new lora_format_adapter module intelligently detects and normalizes various LoRA checkpoint formats into a diffusers-compatible structure, which will greatly enhance robustness when loading LoRAs from different ecosystems. The implementation is well-designed, modular, and robustly handles dependencies and potential conversion failures. The integration into the LoRAPipeline is clean, and the addition of a comprehensive regression test suite with real-world checkpoints is a fantastic step to ensure correctness and prevent future regressions. Overall, this is an excellent contribution. I have one critical suggestion for the new test file to ensure its portability.
MikukuOvO
force-pushed
the
feature-lora-adapter
branch
from
e2a6ee5 to
84cb1fb
Compare
Collaborator
|
/tag-and-rerun-ci |
mickqian
reviewed
Dec 9, 2025
python/sglang/multimodal_gen/runtime/pipelines_core/lora_format_adapter.py
Outdated
Show resolved
Hide resolved
python/sglang/multimodal_gen/runtime/pipelines_core/lora_format_adapter.py
Show resolved
Hide resolved
MikukuOvO
force-pushed
the
feature-lora-adapter
branch
from
84cb1fb to
477006f
Compare
Contributor
Author
|
Addressed all the comments. Thanks for the review! |
Collaborator
|
have you tested them locally? |
Contributor
Author
I've test the |
Collaborator
|
I mean the actual LoRAs with these formats |
MikukuOvO
force-pushed
the
feature-lora-adapter
branch
3 times, most recently
from
3ddf2f7 to
21ba69c
Compare
Contributor
Author
Hi Mick, I've added the test for xlabs-ai, wan, qwen, and z-image, results as follows: black-forest-labs/FLUX.1-dev & XLabs-AI/flux-RealismLora[12-10 04:58:02] Rank 0: LoRA adapter XLabs-AI/flux-RealismLora applied to 152 layerswithout lora
with lora
Qwen/Qwen-Image & lymy-ai/qwen-image-realism-lora[12-10 18:47:11] Rank 0: LoRA adapter flymy-ai/qwen-image-realism-lora applied to 240 layerswithout lora
with lora
Wan-AI/Wan2.2-I2V-A14B-Diffusers & lightx2v/Wan2.2-Distill-Loras[12-10 18:59:06] Rank 0: LoRA adapter lightx2v/Wan2.2-Distill-Loras applied to 400 layersChoose flux generated image as input image, similar results Tongyi-MAI/Z-Image-Turbo & wcde/Z-Image-Turbo-DeJPEG-Lora[12-10 18:50:27] Rank 0: LoRA adapter wcde/Z-Image-Turbo-DeJPEG-Lora applied to 240 layersWhile Tongyi get error in the following generation, seems due to the unmatched reasons in the File "/workspace/sglang/python/sglang/multimodal_gen/runtime/models/dits/zimage.py", line 66, in forward
self.mlp[0].weight.dtype
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1964, in __getattr__
raise AttributeError(
AttributeError: 'BaseLayerWithLoRA' object has no attribute 'weight'
[12-10 18:50:28] Failed to generate output for prompt 1: Error executing request mocked_fake_id_for_offline_generate: 'BaseLayerWithLoRA' object has no attribute 'weight' |
Contributor
Author
Complete log |
…xlabs-ai & standard)
…x, non-diffusers support
…, kohya-flux, etc.)
MikukuOvO
force-pushed
the
feature-lora-adapter
branch
2 times, most recently
from
ef4c92b to
65145fa
Compare
…abs-FLUX, Kohya-FLUX, non-diffusers SD, and more
MikukuOvO
force-pushed
the
feature-lora-adapter
branch
from
65145fa to
009cbde
Compare
Contributor
Author
|
I’ve verified that this issue is already fixed after rebasing onto the latest main (from #14543). |
Collaborator
|
/rerun-failed-ci |
1 similar comment
Collaborator
|
/rerun-failed-ci |
Liwansi
added a commit
to iforgetmyname/sglang
that referenced
this pull request
Dec 13, 2025
…n_eagle3_npu * 'main' of https://github.com/sgl-project/sglang: (25 commits) [NPU] perf update with kvcache nz & w4a8 quant (sgl-project#14423) [PP Prefill][NIXL] Fix PP mode transfer completion tracking to wait for all ranks (sgl-project#15027) Fix GLM-4.6 tool calls don't support streaming output for arguments i… (sgl-project#13989) feature: adding nightly wheel workflow and indexer (sgl-project#14924) [diffusion] feat: Improve LoRA compatibility by adding unified format detection and diffusers-based normalization (sgl-project#14659) [Fix] Disable trtllm moe backend for draft model for a qucik fix (sgl-project#15002) [diffusion] fix: use NDRotaryEmbedding in flux_2 (sgl-project#15034) Mistral Large 3 NVFP4 support (sgl-project#14485) call check_quantized_moe_compatibility after initialize (sgl-project#13876) Add sgl_router_attempt_http_responses_total for single attempt information (sgl-project#15037) Add error code in prometheus metrics and add X-SMG-Error-Code header (sgl-project#15036) Provide more fine grained error reason for reqwest error (sgl-project#15032) Tiny change http router response format to unify (sgl-project#15031) Tiny unify grpc existing error responses into new format (sgl-project#15030) Add `code` field and unify error responses for router (sgl-project#15028) Super tiny remove unused log_request (sgl-project#15035) Fix decode OOM caused by retraction (sgl-project#14939) [CI]Add gb200 runner back (sgl-project#15024) Add a special label for b200 CI runner that can run kernel tests (sgl-project#15033) Fix regression caused by fa3 block_table (sgl-project#15009) ... # Conflicts: # python/sglang/srt/hardware_backend/npu/attention/ascend_backend.py
Prozac614
pushed a commit
to Prozac614/sglang
that referenced
this pull request
Dec 17, 2025
… detection and diffusers-based normalization (sgl-project#14659)
GuoYechang
pushed a commit
to GuoYechang/sglang
that referenced
this pull request
Jan 13, 2026
… detection and diffusers-based normalization (sgl-project#14659)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This PR introduces a complete LoRA format adapter for multimodal generation in SGLang.
LoRA checkpoints across the community vary widely in structure (Kohya/A1111, XLabs FLUX, Wan2.1/Comfy, Qwen-Image, Z-Image, etc.). Without a unified adapter, these heterogeneous naming schemes make it difficult to robustly load LoRAs into SGLang’s diffusers-based pipelines.
This PR provides:
Modifications
1. Added
lora_format_adaptermoduleA new adapter module is implemented to provide a two-stage normalization pipeline for external LoRA checkpoints.
Stage 1 — Format Detection
The adapter inspects the raw LoRA
state_dictand classifies it into one of several known ecosystems based on key patterns:This establishes a consistent and extensible abstraction for LoRA format recognition.
Stage 2 — Normalization via diffusers converters
Based on the detected format, the adapter sequentially tries the appropriate converter functions exposed in:
diffusers.loaders.lora_conversion_utilsThe adapter:
dictand(dict, …)return typesThis produces a uniform, diffusers-compatible LoRA state_dict that the SGLang pipeline can reliably consume.
2. Integrated the adapter into LoRA loading pipeline
After LoRA weights are loaded inside
lora_pipeline, the new adapter is now invoked automatically.All external LoRAs now go through a consistent:
detect → normalize
flow before being applied to the model.
This ensures broader compatibility with real-world LoRA checkpoints and reduces pipeline fragility.
3. Added unit tests for the adapter
A standalone regression test suite is added. It:
This provides confidence that the adapter works across diverse LoRA ecosystems and helps catch regressions early.
Accuracy Tests
This PR does not modify model execution or kernels.
Only LoRA loading behavior changes, and tests verify consistent normalization with no accuracy impact.
Benchmarking and Profiling
This PR does not affect runtime inference performance.
Normalization occurs only once at LoRA load time and has negligible overhead.
Checklist