Skip to content

habana_main rebase v4#85

Merged
kzawora-intel merged 169 commits intohabana_mainfrom
private/kzawora/rebase_v4
Jul 17, 2024
Merged

habana_main rebase v4#85
kzawora-intel merged 169 commits intohabana_mainfrom
private/kzawora/rebase_v4

Conversation

@kzawora-intel
Copy link
Copy Markdown

@kzawora-intel kzawora-intel commented Jul 3, 2024

No description provided.

sirejdua and others added 30 commits July 2, 2024 07:20
@sirejdua @sirejdua-db
[Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (v…
15aba08 
…llm-project#6050)

Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
@andoorve
[Core] Pipeline Parallel Support (vllm-project#4412)
c5832d2 
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
@robertgshaw2-redhat
Update conftest.py (vllm-project#6076)
4d26d80
@robertgshaw2-redhat @mgoin
[ Misc ] Refactor MoE to isolate Fp8 From Mixtral (vllm-project#5970)
7c008c5 
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
@Qubitium
[CORE] Quantized lm-head Framework (vllm-project#4442)
ee93f4f 
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
@mzusman @ErezSC42
@tomeras91 @zhuohan123
[Model] Jamba support (vllm-project#4115)
9d6a8da 
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
@youkaichao
[hardware][misc] introduce platform abstraction (vllm-project#6080)
482045e
@DarkLight1337 @xwjiang2010 @ywang96
[Core] Dynamic image size support for VLMs (vllm-project#5276)
9831aec 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
@rkooo567
[CI] Fix base url doesn't strip "/" (vllm-project#6087)
d18bab3
@njhill
[BugFix] Avoid unnecessary Ray import warnings (vllm-project#6079)
d830656
@youkaichao
[misc][distributed] error on invalid state (vllm-project#6092)
f666207
@ywang96 @DarkLight1337
[VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (vllm-…
3a86b54 
…project#6091)

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@ywang96
[Doc] Fix Mock Import (vllm-project#6094)
f1c7813
@ywang96
[Bugfix] Fix compute_logits in Jamba (vllm-project#6093)
7cd2ebb
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into HEAD
e3482dc
@kzawora-intel
habana components refactor
6a5effb
@kzawora-intel
fix is_prompt for mixtral
bca41a1
@mgoin
[Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (vllm-pro…
47f0954 
…ject#5975)
@youkaichao
[core][distributed] custom allreduce when pp size > 1 (vllm-project#6117
3c6325f 
)
@xwjiang2010 @ywang96
[vlm] Remove vision language config. (vllm-project#6089)
d9e98f4 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@robertgshaw2-redhat
[ Misc ] Clean Up CompressedTensorsW8A8 (vllm-project#6113)
62963d1
@youkaichao
[doc][misc] bump up py version in installation doc (vllm-project#6119)
966fe72
@youkaichao
[core][distributed] support n layers % pp size != 0 (vllm-project#6115)
3de6e6a
@tjohnson31415 @njhill
[Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (vll…
1dab9bc 
…m-project#6109)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
@andoorve
[Distributed][Core] Support Py39 and Py38 for PP (vllm-project#6120)
0ed646b 
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
@DarkLight1337
[CI/Build] Cleanup VLM tests (vllm-project#6107)
3dd5070
@gshtras @hongxiayang
[ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash at…
56b325e 
…tention and naive flash attention (vllm-project#6043)

Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into private/kzawora/reb…
b309c26 
…ase_v4
@kzawora-intel
restore HPU autodetection
717c0ce
@kzawora-intel
add WA for model loader
c5cd04a
ywang96 and others added 25 commits July 15, 2024 17:29
@ywang96
[VLM] Minor space optimization for ClipVisionModel (vllm-project#6436)
6ae1597
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into private/kzawora/reb…
c49d033 
…ase_v4
@tzielinski-habana
Added docstring and assertion to warmup_range
08ba388
@youkaichao
[doc][distributed] add suggestion for distributed inference (vllm-pro…
94b82e8 
…ject#6418)
@kzawora-intel
fix api mismatches
6bed248
@kzawora-intel
add assert for attn type
03dbee5
@tlrmchlsmth
[Kernel] Use CUTLASS kernels for the FP8 layers with Bias (vllm-proje…
c8fd97f 
…ct#6270)
@Pernekhan
[Misc] Use 0.0.9 version for flashinfer (vllm-project#6447)
a63a4c6 
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>
@tdoublep @cyang49
[Bugfix] Add custom Triton cache manager to resolve MoE MP issue (vll…
eaec4b9 
…m-project#6140)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
@tdoublep
[Bugfix] use float32 precision in samplers/test_logprobs.py for compa…
4ef95b0 
…ring with HF (vllm-project#6409)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@simon-mo
bump version to v0.5.2 (vllm-project#6433)
64fdc08
@youkaichao
[misc][distributed] fix pp missing layer condition (vllm-project#6446)
4cf256a
@kzawora-intel
multi-hpu fixes
8c58a66
@kzawora-intel
minor formatting stuff
d7afbf2
@kzawora-intel
fix sampling metadata for prefill
2b2549c
@WoosukKwon
[Docs] Add Google Cloud to sponsor list (vllm-project#6450)
3dee97b
@WoosukKwon
[Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (vllm-proj…
ec9933f 
…ect#6289)
@WoosukKwon @khluu
[CI/Build][TPU] Add TPU CI test (vllm-project#6277)
4552e37 
Co-authored-by: kevin <kevin@anyscale.com>
@khluu
Pin sphinx-argparse version (vllm-project#6453)
d6f3b3d 
Signed-off-by: kevin <kevin@anyscale.com>
@mzusman
[BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cl…
9ad32da 
…eanup bug (vllm-project#6425)

Co-authored-by: Mor Zusman <morz@ai21.com>
@g-eoj
[Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (v…
d92b3c5 
…llm-project#6419)
@WoosukKwon
[Docs] Announce 5th meetup (vllm-project#6458)
37d7766
@DarkLight1337
[CI/Build] vLLM cache directory for images (vllm-project#6444)
d970115
@kzawora-intel
bump ray version for hpu
e911fd8
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into private/kzawora/reb…
202d0b9 
…ase_v4
madamczyk-intel
madamczyk-intel approved these changes Jul 17, 2024
View reviewed changes
Copy link
Copy Markdown

@madamczyk-intel madamczyk-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@kzawora-intel kzawora-intel merged commit 291bee7 into habana_main Jul 17, 2024
@kzawora-intel kzawora-intel added the habana Issues or PRs submitted by Habana Labs label Sep 20, 2024
@kzawora-intel kzawora-intel deleted the private/kzawora/rebase_v4 branch October 7, 2024 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@madamczyk-intel madamczyk-intel madamczyk-intel approved these changes

Assignees

No one assigned

Labels

habana Issues or PRs submitted by Habana Labs rebase

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.