Skip to content

Commit 051b82b

Browse files
CSWYF3634076CSWYF3634076
authored
[Docs] add qwen25-vl docs (#5243)
* [Docs] add qwen25-vl docs * [Docs] add qwen25-vl docs * [Docs] add qwen25-vl docs
1 parent 5a67a6d commit 051b82b

File tree

3 files changed

+268
-0
lines changed

3 files changed

+268
-0
lines changed

‎docs/get_started/quick_start_qwen25_vl.md‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
[简体中文](../zh/get_started/quick_start_qwen25_vl.md)
2+
3+
# Deploy Qwen2.5-VL in 10 Minutes
4+
5+
Before deployment, ensure your environment meets the following requirements:
6+
7+
- GPU Driver ≥ 535
8+
- CUDA ≥ 12.3
9+
- cuDNN ≥ 9.5
10+
- Linux X86_64
11+
- Python ≥ 3.10
12+
13+
This guide uses the lightweight Qwen2.5-VL model for demonstration, which can be deployed on most hardware configurations. Docker deployment is recommended.
14+
15+
For more information about how to install FastDeploy, refer to the [installation document](installation/README.md).
16+
17+
## 1. Launch Service
18+
19+
Please download the qwen25-vl model in advance: such as [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)
20+
21+
Add the following configuration in `config.json`
22+
```text
23+
"rope_3d": true,
24+
"freq_allocation": 16
25+
```
26+
27+
After installing FastDeploy, execute the following command in the terminal to start the service. For the configuration method of the startup command, refer to [Parameter Description](../parameters.md)
28+
29+
```
30+
export ENABLE_V1_KVCACHE_SCHEDULER=1
31+
python -m fastdeploy.entrypoints.openai.api_server \
32+
--model You/Path/Qwen2.5-VL-7B-Instruct \
33+
--port 8180 \
34+
--metrics-port 8181 \
35+
--engine-worker-queue-port 8182 \
36+
--max-model-len 32768 \
37+
--max-num-seqs 32
38+
```
39+
40+
> 💡 Note: In the path specified by ```--model```, if the subdirectory corresponding to the path does not exist in the current directory, it will try to query whether AIStudio has a preset model based on the specified model name (such as ```Qwen/Qwen2.5-VL-7B-Instruct```). If it exists, it will automatically start downloading. The default download path is: ```~/xx```. For instructions and configuration on automatic model download, see [Model Download](../supported_models.md).
41+
```--max-model-len``` indicates the maximum number of tokens supported by the currently deployed service.
42+
```--max-num-seqs``` indicates the maximum number of concurrent processing supported by the currently deployed service.
43+
44+
**Related Documents**
45+
- [Service Deployment](../online_serving/README.md)
46+
- [Service Monitoring](../online_serving/metrics.md)
47+
48+
## 2. Request the Service
49+
After starting the service, the following output indicates successful initialization:
50+
51+
```shell
52+
api_server.py[line:91] Launching metrics service at http://0.0.0.0:8181/metrics
53+
api_server.py[line:94] Launching chat completion service at http://0.0.0.0:8180/v1/chat/completions
54+
api_server.py[line:97] Launching completion service at http://0.0.0.0:8180/v1/completions
55+
INFO: Started server process [13909]
56+
INFO: Waiting for application startup.
57+
INFO: Application startup complete.
58+
INFO: Uvicorn running on http://0.0.0.0:8180 (Press CTRL+C to quit)
59+
```
60+
61+
### Health Check
62+
63+
Verify service status (HTTP 200 indicates success):
64+
65+
```shell
66+
curl -i http://0.0.0.0:8180/health
67+
```
68+
69+
### cURL Request
70+
Send requests as follows:
71+
72+
```shell
73+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
74+
-H "Content-Type: application/json" \
75+
-d '{
76+
"messages": [
77+
{"role": "user", "content": "Rewrite Li Bai's 'Quiet Night Thoughts' as a modern poem"}
78+
]
79+
}'
80+
```
81+
82+
For image inputs:
83+
84+
```shell
85+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
86+
-H "Content-Type: application/json" \
87+
-d '{
88+
"messages": [
89+
{"role": "user", "content": [
90+
{"type":"image_url", "image_url": {"url":"https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
91+
{"type":"text", "text":"From which era does the artifact in the image originate?"}
92+
]}
93+
]
94+
}'
95+
```
96+
97+
For video inputs:
98+
99+
```shell
100+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
101+
-H "Content-Type: application/json" \
102+
-d '{
103+
"messages": [
104+
{"role": "user", "content": [
105+
{"type":"video_url", "video_url": {"url":"https://bj.bcebos.com/v1/paddlenlp/datasets/paddlemix/demo_video/example_video.mp4"}},
106+
{"type":"text", "text":"How many apples are in the scene?"}
107+
]}
108+
]
109+
}'
110+
```
111+
112+
### Python Client (OpenAI-compatible API)
113+
114+
FastDeploy's API is OpenAI-compatible. You can also use Python for streaming requests:
115+
116+
```python
117+
import openai
118+
host = "0.0.0.0"
119+
port = "8180"
120+
client = openai.Client(base_url=f"http://{host}:{port}/v1", api_key="null")
121+
122+
response = client.chat.completions.create(
123+
model="null",
124+
messages=[
125+
{"role": "user", "content": [
126+
{"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
127+
{"type": "text", "text": "From which era does the artifact in the image originate?"},
128+
]},
129+
],
130+
stream=True,
131+
)
132+
for chunk in response:
133+
if chunk.choices[0].delta:
134+
print(chunk.choices[0].delta.content, end='')
135+
print('\n')
136+
```

‎docs/zh/get_started/quick_start_qwen25_vl.md‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
[English](../../get_started/quick_start_qwen25_vl.md)
2+
3+
# 10分钟完成 Qwen2.5-VL 模型部署
4+
5+
本文档讲解如何部署Qwen2.5-VL模型，在开始部署前，请确保你的硬件环境满足如下条件：
6+
7+
- GPU驱动 >= 535
8+
- CUDA >= 12.3
9+
- CUDNN >= 9.5
10+
- Linux X86_64
11+
- Python >= 3.10
12+
13+
为了快速在各类硬件部署，本文档采用 ```Qwen2.5-VL``` 模型作为示例，可在大部分硬件上完成部署。
14+
15+
安装FastDeploy方式参考[安装文档](./installation/README.md)。
16+
## 1. 启动服务
17+
请提前下载Qwen2.5-VL模型，例如 [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)
18+
19+
在`config.json`中增加下面的配置项
20+
```text
21+
"rope_3d": true,
22+
"freq_allocation": 16
23+
```
24+
25+
安装FastDeploy后，在终端执行如下命令，启动服务，其中启动命令配置方式参考[参数说明](../parameters.md)
26+
27+
```shell
28+
export ENABLE_V1_KVCACHE_SCHEDULER=1
29+
python -m fastdeploy.entrypoints.openai.api_server \
30+
--model You/Path/Qwen2.5-VL-7B-Instruct \
31+
--port 8180 \
32+
--metrics-port 8181 \
33+
--engine-worker-queue-port 8182 \
34+
--max-model-len 32768 \
35+
--max-num-seqs 32
36+
```
37+
38+
>💡 注意：在 ```--model``` 指定的路径中，若当前目录下不存在该路径对应的子目录，则会尝试根据指定的模型名称（如 ```Qwen/Qwen2.5-VL-7B-Instruct```）查询AIStudio是否存在预置模型，若存在，则自动启动下载。默认的下载路径为：```~/xx```。关于模型自动下载的说明和配置参阅[模型下载](../supported_models.md)。
39+
```--max-model-len``` 表示当前部署的服务所支持的最长Token数量。
40+
```--max-num-seqs``` 表示当前部署的服务所支持的最大并发处理数量。
41+
42+
**相关文档**
43+
44+
- [服务部署配置](../online_serving/README.md)
45+
- [服务监控metrics](../online_serving/metrics.md)
46+
47+
## 2. 用户发起服务请求
48+
49+
执行启动服务指令后，当终端打印如下信息，说明服务已经启动成功。
50+
51+
```
52+
api_server.py[line:91] Launching metrics service at http://0.0.0.0:8181/metrics
53+
api_server.py[line:94] Launching chat completion service at http://0.0.0.0:8180/v1/chat/completions
54+
api_server.py[line:97] Launching completion service at http://0.0.0.0:8180/v1/completions
55+
INFO: Started server process [13909]
56+
INFO: Waiting for application startup.
57+
INFO: Application startup complete.
58+
INFO: Uvicorn running on http://0.0.0.0:8180 (Press CTRL+C to quit)
59+
```
60+
61+
FastDeploy提供服务探活接口，用以判断服务的启动状态，执行如下命令返回 ```HTTP/1.1 200 OK``` 即表示服务启动成功。
62+
63+
```shell
64+
curl -i http://0.0.0.0:8180/health
65+
```
66+
67+
通过如下命令发起服务请求
68+
69+
```shell
70+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
71+
-H "Content-Type: application/json" \
72+
-d '{
73+
"messages": [
74+
{"role": "user", "content": "把李白的静夜思改写为现代诗"}
75+
]
76+
}'
77+
```
78+
79+
输入包含图片时，按如下命令发起请求
80+
81+
```shell
82+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
83+
-H "Content-Type: application/json" \
84+
-d '{
85+
"messages": [
86+
{"role": "user", "content": [
87+
{"type":"image_url", "image_url": {"url":"https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
88+
{"type":"text", "text":"图中的文物属于哪个年代?"}
89+
]}
90+
]
91+
}'
92+
```
93+
94+
输入包含视频时，按如下命令发起请求
95+
96+
```shell
97+
curl -X POST "http://0.0.0.0:8180/v1/chat/completions" \
98+
-H "Content-Type: application/json" \
99+
-d '{
100+
"messages": [
101+
{"role": "user", "content": [
102+
{"type":"video_url", "video_url": {"url":"https://bj.bcebos.com/v1/paddlenlp/datasets/paddlemix/demo_video/example_video.mp4"}},
103+
{"type":"text", "text":"画面中有几个苹果?"}
104+
]}
105+
]
106+
}'
107+
```
108+
109+
FastDeploy服务接口兼容OpenAI协议，可以通过如下Python代码发起服务请求。
110+
```python
111+
import openai
112+
host = "0.0.0.0"
113+
port = "8180"
114+
client = openai.Client(base_url=f"http://{host}:{port}/v1", api_key="null")
115+
116+
response = client.chat.completions.create(
117+
model="null",
118+
messages=[
119+
{"role": "user", "content": [
120+
{"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
121+
{"type": "text", "text": "图中的文物属于哪个年代?"},
122+
]},
123+
],
124+
stream=True,
125+
)
126+
for chunk in response:
127+
if chunk.choices[0].delta:
128+
print(chunk.choices[0].delta.content, end='')
129+
print('\n')
130+
```

‎mkdocs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ plugins:
5959
ERNIE-4.5-300B-A47B: ERNIE-4.5-300B-A47B快速部署
6060
ERNIE-4.5-VL-424B-A47B: ERNIE-4.5-VL-424B-A47B快速部署
6161
Quick Deployment For QWEN: Qwen3-0.6b快速部署
62+
Quick Deployment For QWEN2.5-VL: Qwen2.5-VL系列快速部署
6263
Online Serving: 在线服务
6364
OpenAI-Compatible API Server: 兼容 OpenAI 协议的服务化部署
6465
Monitor Metrics: 监控Metrics
@@ -122,6 +123,7 @@ nav:
122123
- ERNIE-4.5-300B-A47B: get_started/ernie-4.5.md
123124
- ERNIE-4.5-VL-424B-A47B: get_started/ernie-4.5-vl.md
124125
- Quick Deployment For QWEN: get_started/quick_start_qwen.md
126+
- Quick Deployment For QWEN2.5-VL: get_started/quick_start_qwen25_vl.md
125127
- Online Serving:
126128
- OpenAI-Compatible API Server: online_serving/README.md
127129
- Monitor Metrics: online_serving/metrics.md

0 commit comments

Comments
 (0)