Open
Conversation
junpuf
force-pushed
the
vllm-dsv4-custom-build
branch
2 times, most recently
from
98f05a3 to
550ce92
Compare
Pin vLLM source to zyongye/vllm@3602f14f (dsv4 branch) which adds [New Model] Support DeepseekV4. Changes: - Add docker/vllm/versions.env with custom VLLM_REPO/VLLM_REF - Update image configs to point to the custom commit - Add EXTRA_BUILD_ARGS forwarding in build_image.sh - Add SETUPTOOLS_SCM_PRETEND_VERSION build-arg in Dockerfile - Update Dockerfile requirements/build.txt -> requirements/build/cuda.txt - Update workflows to source versions.env and include vllm-ref-short in tags - Disable non-essential tests (sanity/telemetry/upstream/endpoint)
junpuf
force-pushed
the
vllm-dsv4-custom-build
branch
from
550ce92 to
4af7695
Compare
Align with upstream Dockerfile changes: - FlashInfer bumped to 0.6.8.post1 (required by new source ref) - Mooncake changes are Ubuntu/apt-get specific, not applicable to AL2023
Create separate build artifacts instead of modifying the mainline image: - docker/vllm/Dockerfile.dsv4.amzn2023 - docker/vllm/versions.dsv4.env - .github/config/image/vllm-dsv4-ec2-amzn2023.yml - .github/workflows/pr-vllm-dsv4-ec2-amzn2023.yml Revert all changes to existing files (Dockerfile.amzn2023, image configs, workflows). Only shared change: EXTRA_BUILD_ARGS forwarding in build_image.sh.
junpuf
changed the title
junpuf
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build AL2023 vLLM image with DeepseekV4 model support.
Source:
zyongye/vllm@3602f14fWheel version:
0.20.1.dev0+amzn2023.dsv4.3602f14fTest Plan