5eb605b7a2
- run_bench.sh: skip scenarios whose raw jsonl already exists and has expected completed count - parse_backend.py: replace scenarios list instead of appending, so resume+reparse stays clean - README.md: document RUN_ID-based resume
DSV4 / DeepSeek-V4-Flash 推理测速项目
记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。 H200 快速上手指南见
docs/H200_QUICKSTART.md。
目录说明
| 目录/文件 | 说明 |
|---|---|
platforms/ |
芯片/加速器平台配置（*.env），如 P800、H200 |
scripts/common/ |
跨实验复用的 orchestration 组件（server 启停、health check、元数据生成） |
scripts/analysis/ |
跨实验对比分析工具 |
scripts/ |
旧版/legacy benchmark 脚本 |
experiments/ |
以实验为单位的自包含目录（脚本 + 配置 + 结果） |
experiments/legacy_bench_results/ |
历史归档的实验产物（只读），从原 bench_results/ 迁移而来 |
BENCHMARK_WORKFLOW.md |
benchmark 目录与命名规范 |
scripts/SLO_STANDARDS.md |
推理服务 SLO 标准（TTFT/TPOT） |
注：
.gitignore已排除envs/、deps/、tmp/、datasets/、logs/、*.zip、原始raw_outputs/*.jsonl等大文件/中间文件。
实验索引
新结构（experiments/）
| 实验 | 脚本 | 说明 |
|---|---|---|
| DSV4 P800 SGLang | experiments/dsv4_p800_sglang/run_bench.sh |
Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8 |
| DSV4 H200 DSpark | experiments/dsv4_h200_dspark/run_bench.sh |
NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark |
| DSV4 H200 vLLM baseline | experiments/dsv4_h200_vllm/run_bench.sh |
NVIDIA H200 + vLLM + DeepSeek-V4-Flash |
| DSV4 H200 SGLang baseline | experiments/dsv4_h200_sglang/run_bench.sh |
NVIDIA H200 + native SGLang + DeepSeek-V4-Flash |
| DSV4 H200 SGLang vs vLLM | experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh |
NVIDIA H200 上 SGLang 与 vLLM 控制变量对比（TP=8，最长 200k 上下文） |
| DSV4 H200 vLLM DSpark vs default | experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh |
vLLM 开启 DSpark 投机解码 vs 默认配置，验证 TTFT 差异 |
| DSV4 H200 max context length | experiments/dsv4_h200_max_context_length/run_bench.sh |
探索 SGLang / vLLM 在 H200 上能支持的最大输入长度 |
旧结构（scripts/ + bench_results/）
| 实验 | 脚本 | 最终报告 | 说明 |
|---|---|---|---|
| DSpark grid benchmark | scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh |
experiments/legacy_bench_results/dspark_grid_20260707-132641/report.md |
P1/P2/P3 全量网格 |
| DSpark spec-tokens 对比 | scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh |
experiments/legacy_bench_results/dspark_st_comparison_20260707-150649/comparison_report.md |
--spec-tokens 3 vs 5 |
| SGLang EAGLE vs DSpark | scripts/start_sglang_dsv4_8card.sh + 对比解析脚本 |
experiments/legacy_bench_results/eagle_grid/dspark_vs_eagle_report.md |
EAGLE 投机解码对比 |
| SGLang vs vLLM 后端对比（0707） | scripts/benchmark_dsv4_backend_comparison.sh |
experiments/legacy_bench_results/dsv4_backend_comparison_20260707/README.md |
遗留 raw outputs 清单 |
| SGLang 8-card systematic | scripts/run_sglang_benchmark.sh |
experiments/legacy_bench_results/sglang_8card_systematic_20260704_120819/report.md |
早期 SGLang 系统扫描 |
| SGLang 8-card max throughput | scripts/run_sglang_max_throughput.sh |
experiments/legacy_bench_results/sglang_8card_max_throughput_20260705_030839/summary.json |
最大吞吐扫描 |
| vLLM-dspark Qwen3 | scripts/bench_vllm_dspark_qwen3.py |
experiments/legacy_bench_results/vllm_dspark_qwen3_20260705_121256/summary.json |
Qwen3 模型测速 |
| DSV4 后端对比（0705） | scripts/bench_dsv4_comparison.py |
experiments/legacy_bench_results/dsv4_comparison_20260705_152221/summary.json |
早期 DSpark 配置对比 |
快速复现
P800 SGLang
bash experiments/dsv4_p800_sglang/run_bench.sh
H200 DSpark
bash experiments/dsv4_h200_dspark/run_bench.sh
H200 SGLang baseline
bash experiments/dsv4_h200_sglang/run_bench.sh
H200 SGLang vs vLLM 对比
bash experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh
H200 vLLM DSpark vs default
bash experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh
H200 max context length
bash experiments/dsv4_h200_max_context_length/run_bench.sh
DSpark grid（旧结构）
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
Spec-tokens 对比
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
解析已有结果
# 新结构
python3 experiments/dsv4_p800_sglang/parse_results.py \
experiments/dsv4_p800_sglang/results/<run_id>
python3 experiments/dsv4_h200_dspark/parse_results.py \
experiments/dsv4_h200_dspark/results/<run_id>
# 旧结构
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
/data/user1/yy/experiments/legacy_bench_results/dspark_grid_20260707-132641
SLO 参考
详见 scripts/SLO_STANDARDS.md。
主要关注指标：
- TTFT P95：S2（DeepSeek-V4-Flash 所在层）目标
< 3s - TPOT：S2 目标
< 50ms
环境要求
- NVIDIA H200：Python env
/data/user1/yy/envs/vllm-dspark（服务端）、/data/user1/yy/envs/sglang（压测客户端）；模型/data/models/DeepSeek-V4-Flash、/data/models/DeepSeek-V4-Flash-DSpark。 - Kunlun P800：Docker 镜像
iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260511_4202；模型/data1/models/DeepSeek-V4-Flash-INT8；压测客户端在容器内运行。
平台相关常量见 platforms/*.env。