Skip to content

[vllm_omni, rollout] fix: forward-compat with vllm-omni multi-LoRA API rename#6077

Open
ultranationalism wants to merge 2 commits intoverl-project:mainfrom
ultranationalism:forward-compat-vllm-omni-multi-lora
Open

[vllm_omni, rollout] fix: forward-compat with vllm-omni multi-LoRA API rename#6077
ultranationalism wants to merge 2 commits intoverl-project:mainfrom
ultranationalism:forward-compat-vllm-omni-multi-lora

Conversation

@ultranationalism
Copy link
Copy Markdown

What does this PR do?

Makes vllm_omni_async_server.generate() forward-compatible with the upcoming vllm-omni multi-LoRA API rename (vllm-project/vllm-omni#2309), which renames

  • OmniDiffusionSamplingParams.lora_request: LoRARequest | Nonelora_requests: list[LoRARequest]
  • OmniDiffusionSamplingParams.lora_scale: floatlora_scales: list[float]

to support multi-LoRA composition on a single diffusion request.

Today VERL constructs sampling params with the singular key:

if lora_request is not None:
    sampling_kwargs["lora_request"] = lora_request

That TypeErrors the moment rollout installs a vllm-omni release containing the rename, because the dataclass no longer has a lora_request field. This PR detects which field name the installed vllm-omni exposes at runtime via OmniDiffusionSamplingParams.__dataclass_fields__ and populates the correct one — keeping the adapter working on both the currently-pinned vllm-omni==0.18.0 (singular) and any post-rename release (plural, one element list wrapping the single LoRA that this adapter currently supports).

Checklist Before Starting

  • Search for similar PRs:
    • gh pr list --repo volcengine/verl --state all --search \"vllm_omni_async_server\" — no results
    • gh pr list --repo volcengine/verl --state open --search \"vllm_omni lora\" — no results
    • gh issue list --repo volcengine/verl --state all --search \"vllm_omni multi-lora\" — no results
  • PR title format: [vllm_omni, rollout] fix: …
  • Not a breaking change: both API shapes remain supported on the VERL side.

Test

CI pins vllm-omni==0.18.0 (singular API). Under that pin, the branch selected at runtime is the else branch, which is byte-identical to the pre-patch behavior, so all existing tests in tests/workers/rollout/rollout_vllm/test_vllm_omni_generate.py continue to hold without modification.

The plural branch activates once rollout bumps its pin past the rename; at that point the same test file exercises the new path. No additional unit tests are added in this PR because the rename hasn't landed in a released vllm-omni yet — pinning a test against a future API shape would require vendoring it.

Manually verified the dataclass-field probe on both branches of vllm-omni:

# vllm-omni 0.18.0
>>> "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__
False
# vllm-omni multi-lora-v2 (PR #2309 head)
>>> "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__
True

API and Usage Example

No public VERL API change. The rollout continues to accept exactly one LoRA adapter per request (same self.lora_as_adapter / VLLM_LORA_INT_ID plumbing as today); the patch only affects how that single adapter is handed to OmniDiffusionSamplingParams.

Design & Code Changes

Single file touched: verl/workers/rollout/vllm_rollout/vllm_omni_async_server.py — 8 inserted, 1 deleted.

if lora_request is not None:
    # vllm-omni >=0.19 renames lora_request/lora_scale to the plural
    # lora_requests/lora_scales (list form) for multi-LoRA composition.
    # Detect which field the installed version exposes so this adapter
    # works on both pre- and post-rename releases.
    if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__:
        sampling_kwargs["lora_requests"] = [lora_request]
    else:
        sampling_kwargs["lora_request"] = lora_request

AI assistance disclosure

This PR was drafted with Claude Code. The human submitter reviewed every changed line and defends the change end-to-end.

Checklist Before Submitting

  • Read the Contribute Guide.
  • Pre-commit clean (one-file, dataclass-fields probe, no format/lint changes).
  • Not related to the recipe submodule.

Signed-off-by: ultranationalism www913363043@gmail.com

…I rename

vllm-omni upstream (verl-project#2309) renames OmniDiffusionSamplingParams.lora_request
and .lora_scale to the plural list-form lora_requests / lora_scales, in
order to support multi-LoRA composition. That rename breaks the current
vllm_omni_async_server.py which builds sampling_kwargs with the singular
key and would fail on newer vllm-omni releases.

Populate whichever of the two field names the installed vllm-omni exposes,
detected via the OmniDiffusionSamplingParams dataclass __dataclass_fields__
at runtime. This keeps working with the currently-pinned vllm-omni 0.18.0
(singular) while not breaking once rollout upgrades past the rename.

AI-assisted change: drafted with Claude Code, reviewed and defended by the
human submitter end-to-end. Not duplicating any open PR (searched for
"vllm_omni_async_server", "vllm_omni lora", "vllm_omni multi-lora" on
2026-04-20, no results). Tests: no new tests - the existing
tests/workers/rollout/rollout_vllm/test_vllm_omni_generate.py path is
unchanged under the singular API (which is what the pinned 0.18.0
exposes); the new plural branch will be exercised once the CI pin is
bumped to a post-rename vllm-omni release.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: ultranationalism <www913363043@gmail.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces forward-compatibility for vllm-omni versions 0.19 and above by dynamically mapping 'lora_request' to the new plural 'lora_requests' field. Feedback suggests extending this logic to include 'lora_scale' (mapping it to 'lora_scales'), as the current implementation only addresses the request field despite the code comments mentioning both.

Comment on lines +178 to +181
if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__:
sampling_kwargs["lora_requests"] = [lora_request]
else:
sampling_kwargs["lora_request"] = lora_request
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The PR description and the added code comments (lines 174-175) explicitly mention that both lora_request and lora_scale are being renamed to their plural forms (lora_requests and lora_scales) in vllm-omni >= 0.19. However, the current implementation only handles the forward-compatibility for lora_request.

If a user passes lora_scale in sampling_params, it will be moved to extra_args by the loop at lines 167-171 when running on a newer version of the library. This value will then be ignored by the OmniDiffusionSamplingParams constructor. To ensure full forward-compatibility as intended, lora_scale should also be mapped to lora_scales (as a single-element list) when the plural API is detected.

Suggested change
if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__:
sampling_kwargs["lora_requests"] = [lora_request]
else:
sampling_kwargs["lora_request"] = lora_request
if "lora_requests" in OmniDiffusionSamplingParams.__dataclass_fields__:
sampling_kwargs["lora_requests"] = [lora_request]
if "lora_scale" in extra_args:
sampling_kwargs["lora_scales"] = [extra_args.pop("lora_scale")]
else:
sampling_kwargs["lora_request"] = lora_request

Address gemini-code-assist review on verl-project#6077: the prior commit only mapped
the singular lora_request to lora_requests, but the commit message and
code comment explicitly called out both lora_request AND lora_scale as
renamed in vllm-omni >=0.19. Handle lora_scale symmetrically so callers
who pass it via sampling_params do not have it silently swallowed into
extra_args on the post-rename API.

On the old singular API this is a no-op: "lora_scale" is a primitive-
default float field on OmniDiffusionSamplingParams, so the existing
hasattr loop already routes it into sampling_kwargs. On the new plural
API the key does not match any dataclass field, so the loop puts it
into extra_args; when a lora_request is present we now lift it back out
into the plural lora_scales list.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: ultranationalism <www913363043@gmail.com>
@ultranationalism
Copy link
Copy Markdown
Author

Addressed gemini-code-assist review in f3d95celora_scale from sampling_params now gets lifted out of extra_args into the plural lora_scales when the post-rename API is in use. No VERL code path currently sets lora_scale (grepped: zero hits outside the new comment; DiffusionSamplingConfig doesn't declare it), so this is purely guarding future / external callers who pass it through the loose sampling_params: dict surface — matches the symmetry the commit message / code comment already promised.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant