115 lines
4.4 KiB
Markdown
115 lines
4.4 KiB
Markdown
# DSV4 / DeepSeek-V4-Flash 推理测速项目
|
||
|
||
> 记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。
|
||
> H200 快速上手指南见 [`docs/H200_QUICKSTART.md`](docs/H200_QUICKSTART.md)。
|
||
|
||
## 目录说明
|
||
|
||
| 目录/文件 | 说明 |
|
||
|---|---|
|
||
| `platforms/` | 芯片/加速器平台配置（`*.env`），如 P800、H200 |
|
||
| `scripts/common/` | 跨实验复用的 orchestration 组件（server 启停、health check、元数据生成、结果解析） |
|
||
| `scripts/` | 仅保留仍在使用的 legacy DSpark 脚本与 SLO 标准 |
|
||
| `experiments/` | 以实验为单位的自包含目录（脚本 + 配置 + 结果） |
|
||
| `experiments/legacy_bench_results/` | 历史归档的实验产物（只读），从原 `bench_results/` 迁移而来 |
|
||
| `BENCHMARK_WORKFLOW.md` | benchmark 目录与命名规范 |
|
||
| `docs/SLO_STANDARDS.md` | 推理服务 SLO 标准（TTFT/TPOT） |
|
||
|
||
> 注：`.gitignore` 已排除 `envs/`、`deps/`、`tmp/`、`datasets/`、`logs/`、`*.zip`、原始 `raw_outputs/*.jsonl` 等大文件/中间文件。
|
||
|
||
## 实验索引
|
||
|
||
### 新结构（experiments/）
|
||
|
||
| 实验 | 脚本 | 说明 |
|
||
|---|---|---|
|
||
| DSV4 P800 SGLang | `experiments/dsv4_p800_sglang/run_bench.sh` | Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8 |
|
||
| DSV4 H200 DSpark | `experiments/dsv4_h200_dspark/run_bench.sh` | NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark |
|
||
| DSV4 H200 vLLM baseline | `experiments/dsv4_h200_vllm/run_bench.sh` | NVIDIA H200 + vLLM + DeepSeek-V4-Flash |
|
||
| DSV4 H200 SGLang baseline | `experiments/dsv4_h200_sglang/run_bench.sh` | NVIDIA H200 + native SGLang + DeepSeek-V4-Flash |
|
||
| DSV4 H200 SGLang vs vLLM | `experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh` | NVIDIA H200 上 SGLang 与 vLLM 控制变量对比（TP=8，最长 200k 上下文） |
|
||
| DSV4 H200 vLLM DSpark vs default | `experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh` | vLLM 开启 DSpark 投机解码 vs 默认配置，验证 TTFT 差异 |
|
||
| DSV4 H200 max context length | `experiments/dsv4_h200_max_context_length/run_bench.sh` | 探索 SGLang / vLLM 在 H200 上能支持的最大输入长度 |
|
||
|
||
### 旧结构（scripts/ + bench_results/）
|
||
|
||
仅保留还在被 `experiments/dsv4_h200_dspark/` 引用的 legacy DSpark 脚本，其余旧脚本已删除。
|
||
|
||
| 实验 | 脚本 | 最终报告 | 说明 |
|
||
|---|---|---|---|
|
||
| DSpark grid benchmark | `scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh` | `experiments/legacy_bench_results/dspark_grid_20260707-132641/report.md` | P1/P2/P3 全量网格 |
|
||
| DSpark spec-tokens 对比 | `scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh` | `experiments/legacy_bench_results/dspark_st_comparison_20260707-150649/comparison_report.md` | `--spec-tokens 3` vs `5` |
|
||
|
||
## 快速复现
|
||
|
||
### P800 SGLang
|
||
|
||
```bash
|
||
bash experiments/dsv4_p800_sglang/run_bench.sh
|
||
```
|
||
|
||
### H200 DSpark
|
||
|
||
```bash
|
||
bash experiments/dsv4_h200_dspark/run_bench.sh
|
||
```
|
||
|
||
### H200 SGLang baseline
|
||
|
||
```bash
|
||
bash experiments/dsv4_h200_sglang/run_bench.sh
|
||
```
|
||
|
||
### H200 SGLang vs vLLM 对比
|
||
|
||
```bash
|
||
bash experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh
|
||
```
|
||
|
||
### H200 vLLM DSpark vs default
|
||
|
||
```bash
|
||
bash experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh
|
||
```
|
||
|
||
### H200 max context length
|
||
|
||
```bash
|
||
bash experiments/dsv4_h200_max_context_length/run_bench.sh
|
||
```
|
||
|
||
### DSpark grid / spec-tokens（旧结构）
|
||
|
||
```bash
|
||
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
||
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
|
||
```
|
||
|
||
### 解析已有结果
|
||
|
||
```bash
|
||
# 新结构
|
||
python3 experiments/dsv4_h200_dspark/parse_results.py \
|
||
experiments/dsv4_h200_dspark/results/<run_id>
|
||
|
||
# 旧结构
|
||
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
|
||
/data/user1/yy/experiments/legacy_bench_results/dspark_grid_20260707-132641
|
||
```
|
||
|
||
## SLO 参考
|
||
|
||
详见 `docs/SLO_STANDARDS.md`。
|
||
|
||
主要关注指标：
|
||
|
||
- **TTFT P95**：S2（DeepSeek-V4-Flash 所在层）目标 `< 3s`
|
||
- **TPOT**：S2 目标 `< 50ms`
|
||
|
||
## 环境要求
|
||
|
||
- **NVIDIA H200**：Python env `/data/user1/yy/envs/vllm-dspark`（服务端）、`/data/user1/yy/envs/sglang`（压测客户端）；模型 `/data/models/DeepSeek-V4-Flash`、`/data/models/DeepSeek-V4-Flash-DSpark`。
|
||
- **Kunlun P800**：Docker 镜像 `iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260511_4202`；模型 `/data1/models/DeepSeek-V4-Flash-INT8`；压测客户端在容器内运行。
|
||
|
||
平台相关常量见 `platforms/*.env`。
|