SGLang v0.4.5.post3 benchmark #3
Description
As titled.
May you use v0.4.5.post3 to get the latest benchmark result? Thanks! @Michaelvll cc @merrymercy
ref sgl-project/sglang#5611 (comment)
50 prompts
| Input Tokens | Output Tokens | vLLM v0.8.4 | SGLang v0.4.5.post3 |
|---|---|---|---|
| 1000 | 2000 | 1042.17 | 1253.18 |
| 5000 | 1000 | 794.54 | 938.40 |
| 10000 | 500 | 436.08 | 489.97 |
| 30000 | 100 | 37.76 | 49.71 |
200 prompts
| Input Tokens | Output Tokens | vLLM v0.8.4 | SGLang v0.4.5.post3 |
|---|---|---|---|
| 1000 | 2000 | 2498.90 | 3141.22 |
| 5000 | 1000 | 930.93 | 1255.01 |
| 10000 | 500 | 341.70 | 503.81 |
| 30000 | 100 | 38.44 | 49.50 |
vLLM benchmark results is from https://github.com/Michaelvll/llm-ie-benchmarks/blob/main/README.md