[vllm_omni, rollout] fix: forward-compat with vllm-omni multi-LoRA API rename#6077
Conversation
…I rename vllm-omni upstream (verl-project#2309) renames OmniDiffusionSamplingParams.lora_request and .lora_scale to the plural list-form lora_requests / lora_scales, in order to support multi-LoRA composition. That rename breaks the current vllm_omni_async_server.py which builds sampling_kwargs with the singular key and would fail on newer vllm-omni releases. Populate whichever of the two field names the installed vllm-omni exposes, detected via the OmniDiffusionSamplingParams dataclass __dataclass_fields__ at runtime. This keeps working with the currently-pinned vllm-omni 0.18.0 (singular) while not breaking once rollout upgrades past the rename. AI-assisted change: drafted with Claude Code, reviewed and defended by the human submitter end-to-end. Not duplicating any open PR (searched for "vllm_omni_async_server", "vllm_omni lora", "vllm_omni multi-lora" on 2026-04-20, no results). Tests: no new tests - the existing tests/workers/rollout/rollout_vllm/test_vllm_omni_generate.py path is unchanged under the singular API (which is what the pinned 0.18.0 exposes); the new plural branch will be exercised once the CI pin is bumped to a post-rename vllm-omni release. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: ultranationalism <www913363043@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request introduces forward-compatibility for vllm-omni versions 0.19 and above by dynamically mapping 'lora_request' to the new plural 'lora_requests' field. Feedback suggests extending this logic to include 'lora_scale' (mapping it to 'lora_scales'), as the current implementation only addresses the request field despite the code comments mentioning both.
| if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__: | ||
| sampling_kwargs["lora_requests"] = [lora_request] | ||
| else: | ||
| sampling_kwargs["lora_request"] = lora_request |
There was a problem hiding this comment.
The PR description and the added code comments (lines 174-175) explicitly mention that both lora_request and lora_scale are being renamed to their plural forms (lora_requests and lora_scales) in vllm-omni >= 0.19. However, the current implementation only handles the forward-compatibility for lora_request.
If a user passes lora_scale in sampling_params, it will be moved to extra_args by the loop at lines 167-171 when running on a newer version of the library. This value will then be ignored by the OmniDiffusionSamplingParams constructor. To ensure full forward-compatibility as intended, lora_scale should also be mapped to lora_scales (as a single-element list) when the plural API is detected.
| if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__: | |
| sampling_kwargs["lora_requests"] = [lora_request] | |
| else: | |
| sampling_kwargs["lora_request"] = lora_request | |
| if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__: | |
| sampling_kwargs["lora_requests"] = [lora_request] | |
| if "lora_scale" in extra_args: | |
| sampling_kwargs["lora_scales"] = [extra_args.pop("lora_scale")] | |
| else: | |
| sampling_kwargs["lora_request"] = lora_request |
Address gemini-code-assist review on verl-project#6077: the prior commit only mapped the singular lora_request to lora_requests, but the commit message and code comment explicitly called out both lora_request AND lora_scale as renamed in vllm-omni >=0.19. Handle lora_scale symmetrically so callers who pass it via sampling_params do not have it silently swallowed into extra_args on the post-rename API. On the old singular API this is a no-op: "lora_scale" is a primitive- default float field on OmniDiffusionSamplingParams, so the existing hasattr loop already routes it into sampling_kwargs. On the new plural API the key does not match any dataclass field, so the loop puts it into extra_args; when a lora_request is present we now lift it back out into the plural lora_scales list. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: ultranationalism <www913363043@gmail.com>
|
Addressed gemini-code-assist review in f3d95ce — |
What does this PR do?
Makes
vllm_omni_async_server.generate()forward-compatible with the upcoming vllm-omni multi-LoRA API rename (vllm-project/vllm-omni#2309), which renamesOmniDiffusionSamplingParams.lora_request: LoRARequest | None→lora_requests: list[LoRARequest]OmniDiffusionSamplingParams.lora_scale: float→lora_scales: list[float]to support multi-LoRA composition on a single diffusion request.
Today VERL constructs sampling params with the singular key:
That
TypeErrors the moment rollout installs a vllm-omni release containing the rename, because the dataclass no longer has alora_requestfield. This PR detects which field name the installed vllm-omni exposes at runtime viaOmniDiffusionSamplingParams.__dataclass_fields__and populates the correct one — keeping the adapter working on both the currently-pinnedvllm-omni==0.18.0(singular) and any post-rename release (plural, one element list wrapping the single LoRA that this adapter currently supports).Checklist Before Starting
gh pr list --repo volcengine/verl --state all --search \"vllm_omni_async_server\"— no resultsgh pr list --repo volcengine/verl --state open --search \"vllm_omni lora\"— no resultsgh issue list --repo volcengine/verl --state all --search \"vllm_omni multi-lora\"— no results[vllm_omni, rollout] fix: …Test
CI pins
vllm-omni==0.18.0(singular API). Under that pin, the branch selected at runtime is theelsebranch, which is byte-identical to the pre-patch behavior, so all existing tests intests/workers/rollout/rollout_vllm/test_vllm_omni_generate.pycontinue to hold without modification.The plural branch activates once rollout bumps its pin past the rename; at that point the same test file exercises the new path. No additional unit tests are added in this PR because the rename hasn't landed in a released vllm-omni yet — pinning a test against a future API shape would require vendoring it.
Manually verified the dataclass-field probe on both branches of vllm-omni:
API and Usage Example
No public VERL API change. The rollout continues to accept exactly one LoRA adapter per request (same
self.lora_as_adapter/VLLM_LORA_INT_IDplumbing as today); the patch only affects how that single adapter is handed toOmniDiffusionSamplingParams.Design & Code Changes
Single file touched:
verl/workers/rollout/vllm_rollout/vllm_omni_async_server.py— 8 inserted, 1 deleted.AI assistance disclosure
This PR was drafted with Claude Code. The human submitter reviewed every changed line and defends the change end-to-end.
Checklist Before Submitting
recipesubmodule.Signed-off-by: ultranationalism www913363043@gmail.com