[SPEC_DECODE][V0] fix for spec decode eagle after rebase#1150

Merged

xuechendi merged 4 commits intohabana_mainfrom

dev/rebase_spec_decode_fix

Apr 30, 2025

xuechendi commented Apr 24, 2025 •

edited by github-actions Bot

Loading

Now all spec decode tests are passed.

This PR not only fixed the rebase error, also refactored spec decode impl, includes

remove unnecessary change to common codes: batch_expansion.py
remove hpu_draft_model_runner.py and fix the previous_hidden_states in hpu_model_runner


          fix for spec decode eagle

604bb4a

Signed-off-by: Chendi Xue <chendi.xue@intel.com>

xuechendi requested review from afierka-intel, kzawora-intel, madamczyk-intel, mgawarkiewicz-intel, michalkuligowski and vivekgoe as code owners

April 24, 2025 03:44


          fix the return_hidden_states didn't be set issue

158b050

Signed-off-by: Chendi Xue <chendi.xue@intel.com>

Author

xuechendi commented Apr 24, 2025

@michalkuligowski , please check. I have verified in my own env.
Meanwhile, I have started spec_decode_upstream PR, so the code change here is aligned with upstream PR.
I reverted HPU if_statement in common shared spec decode path, so future rebase can be easier.

xuechendi commented

View reviewed changes

vllm/model_executor/layers/vocab_parallel_embedding.py

xuechendi commented

View reviewed changes

vllm/spec_decode/batch_expansion.py

xuechendi commented

View reviewed changes

vllm/spec_decode/draft_model_runner.py

xuechendi commented

View reviewed changes

vllm/worker/hpu_model_runner.py

xuechendi commented

View reviewed changes

vllm/worker/hpu_model_runner.py

xuechendi commented

View reviewed changes

vllm/worker/hpu_model_runner.py

xuechendi commented

View reviewed changes

vllm/worker/hpu_model_runner.py

xuechendi commented

View reviewed changes

vllm/worker/model_runner_base.py

xuechendi commented

View reviewed changes

vllm/worker/hpu_worker.py

michalkuligowski approved these changes

View reviewed changes


          Merge branch 'habana_main' into dev/rebase_spec_decode_fix

f49a112

michalkuligowski commented Apr 24, 2025

/run-gaudi-tests

xuechendi closed this

xuechendi reopened this

Author

xuechendi commented Apr 24, 2025 •

edited

Loading

@michalkuligowski , I saw failed UT seems went timeout because the shutdown blocked the resource release


          Merge branch 'habana_main' into dev/rebase_spec_decode_fix

1d5feb3

xuechendi enabled auto-merge (squash)

April 25, 2025 16:22

michalkuligowski commented Apr 28, 2025

/run-gaudi-tests

michalkuligowski commented Apr 29, 2025

I retriggered failed tests

michalkuligowski commented Apr 30, 2025

/skip-gaudi-tests due to false negative in mulimodal tests

xuechendi merged commit 2a06ffa into habana_main

39 of 41 checks passed

xuechendi deleted the dev/rebase_spec_decode_fix branch

April 30, 2025 14:00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

michalkuligowski michalkuligowski approved these changes

kzawora-intel Awaiting requested review from kzawora-intel kzawora-intel is a code owner

madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel is a code owner

mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel is a code owner

vivekgoe Awaiting requested review from vivekgoe

afierka-intel Awaiting requested review from afierka-intel afierka-intel is a code owner

Labels

None yet