[feat] add ascend readme and docker release#8700
Merged
zhyncs merged 12 commits intosgl-project:mainfrom Aug 12, 2025
Merged
Conversation
pkking
requested review from
ByronHsu,
HaiShaw,
merrymercy and
zhyncs
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Summary of Changes
Hello @pkking, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
I've initiated work on adding Docker release capabilities, specifically targeting NPU hardware. This pull request introduces a new Dockerfile that outlines the complete environment setup required to build and run applications on NPU devices, including the installation of various deep learning frameworks and custom kernels. The goal is to streamline the deployment process for NPU-accelerated workloads.
Highlights
- New Dockerfile for NPU Support: I've introduced a new Dockerfile (
docker/Dockerfile.npu) to enable the building and pushing of Docker images tailored for NPU (Neural Processing Unit) hardware. This is a foundational step towards supporting NPU-specific deployments. - Comprehensive NPU Environment Setup: The new Dockerfile sets up a comprehensive environment, including the installation of essential development tools, PyTorch with NPU adapters, vLLM, Triton-Ascend, and SGLang. It also integrates a custom SGLang kernel for NPU, ensuring all necessary dependencies are pre-configured within the image.
- Integration of Custom NPU Kernel: The Dockerfile includes specific steps to clone and build
sgl-kernel-npuand installdeep-ep, which are crucial for leveraging NPU capabilities with SGLang. This ensures that the custom kernel is correctly compiled and linked within the Docker environment.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a new Dockerfile for NPU hardware. The Dockerfile is functional but has several areas for improvement. My review focuses on a critical security issue regarding hardcoded credentials in URLs, and several medium-severity issues related to Docker best practices for optimizing image size and build efficiency. Specifically, I've suggested removing sensitive credentials, combining multiple RUN instructions for apt and pip commands, and cleaning up cloned git repositories after use. These changes will result in a more secure and leaner Docker image.
21 tasks
ping1jing2
changed the title
pkking
commented
Aug 6, 2025
pkking
commented
Aug 7, 2025
Contributor
Author
|
LGTM |
ping1jing2
reviewed
Aug 10, 2025
ping1jing2
reviewed
Aug 10, 2025
ping1jing2
reviewed
Aug 10, 2025
ping1jing2
reviewed
Aug 10, 2025
ping1jing2
reviewed
Aug 10, 2025
ping1jing2
reviewed
Aug 10, 2025
6 tasks
merrymercy
reviewed
Aug 11, 2025
merrymercy
reviewed
Aug 11, 2025
merrymercy
reviewed
Aug 11, 2025
Signed-off-by: mywaaagh_admin <pkwarcraft@gmail.com>
Signed-off-by: lichaoran <pkwarcraft@gmail.com>
pkking
force-pushed
the
main
branch
3 times, most recently
from
d2b4b13 to
d91192e
Compare
Signed-off-by: mywaaagh_admin <pkwarcraft@gmail.com>
ping1jing2
changed the title
Collaborator
|
lgtm |
|
@iforgetmyname I am using 8 * Ascend 910B to deploy the GLM4.5-Air (106B) model, but it reports OOM, could you help have a check? thanks.
|
Alcanderian
approved these changes
Aug 12, 2025
Collaborator
let me check it. if you don't use this image, will you also encounter the oom? |
Collaborator
Hey @thincal could you please open an issue here so that we can track it？ |
Alcanderian
added
ready-to-merge
narutolhy
pushed a commit
to narutolhy/sglang
that referenced
this pull request
Aug 17, 2025
Signed-off-by: mywaaagh_admin <pkwarcraft@gmail.com> Signed-off-by: lichaoran <pkwarcraft@gmail.com> Co-authored-by: Even Zhou <even.y.zhou@outlook.com> Co-authored-by: ronnie_zheng <zl19940307@163.com>
MahmoudAshraf97
pushed a commit
to MahmoudAshraf97/sglang
that referenced
this pull request
Sep 8, 2025
Signed-off-by: mywaaagh_admin <pkwarcraft@gmail.com> Signed-off-by: lichaoran <pkwarcraft@gmail.com> Co-authored-by: Even Zhou <even.y.zhou@outlook.com> Co-authored-by: ronnie_zheng <zl19940307@163.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
In the past, we only had images for GPU and AMD, but this PR would try to build and push docker image for NPU hardware
Modifications
Add two new workflow and NPU related Dockerfile, both docker images will be published to offical registry:
sglang:main-cann8.2.rc1.alpha003-a3sglang:v0.4.10.post1-cann8.2.rc1.alpha003-a3NOTE: we use metadata-action to generate image tags and labels, so the workflow rely on push tag event
FOR END USER
the image name schema:
sglang-{ver|main}-{cann_ver}-{device_type}, the os and python inherit fromcannimage, default to the latestubuntuandpythonversionFOR DEVELOPER
The released image will use the specific sgl-kernel-npu tag, if a new kernel-npu tag is needed, please edit the release docker workflow
Accuracy Test
NO NEED
Benchmark & Profiling
NO NEED
Checklist