Skip to content

Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile#1184

Merged
afierka-intel merged 1 commit intohabana_mainfrom
dev/anko/modal_dict_fix
May 16, 2025
Merged

Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile#1184
afierka-intel merged 1 commit intohabana_mainfrom
dev/anko/modal_dict_fix

Conversation

@anko-intel
Copy link
Copy Markdown

@anko-intel anko-intel commented Apr 29, 2025 •
edited by github-actions Bot
Loading

It fixes issue with compilation in Dynamo. Original version failed with:

torch._dynamo.exc.InternalTorchDynamoError: AttributeError: 'dict' object has no attribute 'node'
...
    from user code:
..
         File "./vllm/model_executor/layers/linear.py", line 1469, in sync_weight_attrs
           for k in (set(src_param.__dict__.keys()) -

@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented Apr 29, 2025

/run-gaudi-tests

michalkuligowski
michalkuligowski reviewed Apr 30, 2025
View reviewed changes
Comment thread vllm/model_executor/layers/linear.py
@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented May 8, 2025

/run-gaudi-tests

1 similar comment
@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented May 9, 2025

/run-gaudi-tests

@michalkuligowski
Copy link
Copy Markdown

michalkuligowski commented May 9, 2025

@kzawora-intel @madamczyk-intel I think we could merge it since we have PR on upstream?

@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented May 14, 2025

/run-gaudi-tests

@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented May 15, 2025

@michalkuligowski, upstream version vllm-project#17844 was merged.

@anko-intel anko-intel force-pushed the dev/anko/modal_dict_fix branch from df53121 to 951eb97 Compare May 15, 2025 08:38
@anko-intel
Copy link
Copy Markdown
Author

anko-intel commented May 15, 2025

/run-gaudi-tests

@afierka-intel afierka-intel merged commit 387b93a into habana_main May 16, 2025
46 checks passed
@afierka-intel afierka-intel deleted the dev/anko/modal_dict_fix branch May 16, 2025 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michalkuligowski michalkuligowski michalkuligowski left review comments

@afierka-intel afierka-intel afierka-intel approved these changes

@bkowalskiINTEL bkowalskiINTEL bkowalskiINTEL approved these changes

@kzawora-intel kzawora-intel Awaiting requested review from kzawora-intel kzawora-intel is a code owner

@madamczyk-intel madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel is a code owner

@mgawarkiewicz-intel mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel is a code owner

@vivekgoe vivekgoe Awaiting requested review from vivekgoe

@xuechendi xuechendi Awaiting requested review from xuechendi xuechendi is a code owner

@jikunshang jikunshang Awaiting requested review from jikunshang jikunshang is a code owner

@mswiniarsk mswiniarsk Awaiting requested review from mswiniarsk mswiniarsk is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@anko-intel @michalkuligowski @afierka-intel @bkowalskiINTEL