DSV4 / DeepSeek-V4-Flash 推理测速项目

记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。 H200 快速上手指南见 docs/H200_QUICKSTART.md。

目录说明

目录/文件	说明
platforms/	芯片/加速器平台配置（*.env），如 P800、H200、RTX 6000D
scripts/common/	跨实验复用的 orchestration 组件（server 启停、health check、元数据生成、结果解析）
scripts/	仅保留仍在使用的 legacy DSpark 脚本与 SLO 标准
experiments/	以实验为单位的自包含目录（脚本 + 配置 + 结果）
experiments/legacy_bench_results/	历史归档的实验产物（只读），从原 bench_results/ 迁移而来
BENCHMARK_WORKFLOW.md	benchmark 目录与命名规范
docs/SLO_STANDARDS.md	推理服务 SLO 标准（TTFT/TPOT）

注：.gitignore 已排除 envs/、deps/、tmp/、datasets/、logs/、*.zip、原始 raw_outputs/*.jsonl 等大文件/中间文件。

实验索引

新结构（experiments/）

实验	脚本	说明
DSV4 P800 SGLang	experiments/dsv4_p800_sglang/run_bench.sh	Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8
DSV4 H200 DSpark	experiments/dsv4_h200_dspark/run_bench.sh	NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark
DSV4 H200 vLLM baseline	experiments/dsv4_h200_vllm/run_bench.sh	NVIDIA H200 + vLLM + DeepSeek-V4-Flash
DSV4 H200 SGLang baseline	experiments/dsv4_h200_sglang/run_bench.sh	NVIDIA H200 + native SGLang + DeepSeek-V4-Flash
DSV4 H200 SGLang vs vLLM	experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh	NVIDIA H200 上 SGLang 与 vLLM 控制变量对比（TP=8，最长 200k 上下文）
DSV4 H200 vLLM DSpark vs default	experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh	vLLM 开启 DSpark 投机解码 vs 默认配置，验证 TTFT 差异
DSV4 H200 max context length	experiments/dsv4_h200_max_context_length/run_bench.sh	探索 SGLang / vLLM 在 H200 上能支持的最大输入长度
DSV4 H200 SGLang TP/DP Matrix	experiments/dsv4_h200_sglang_tp_dp_matrix/run_bench.sh	H200 上 SGLang 不同 TP×DP 配置的固定并发矩阵测试
DSV4 H200 SGLang TP/DP Adaptive	experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh	H200 上 SGLang 不同 TP×DP 配置的自适应并发饱和点搜索
DSV4 H200 vLLM TP/DP Matrix	experiments/dsv4_h200_vllm_tp_dp_matrix/run_bench.sh	H200 上 vLLM 不同 TP×DP 配置的固定并发矩阵测试
DSV4 H200 vLLM TP/DP Adaptive	experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh	H200 上 vLLM 不同 TP×DP 配置的自适应并发饱和点搜索
DSV4 H200 64k SGLang vs vLLM	experiments/dsv4_h200_64k_sglang_vs_vllm/run_bench.sh	H200 上 64k 上下文 SGLang 与 vLLM 对比
DSV4 H200 256k 4k Probe	experiments/dsv4_h200_256k_4k_probe/run_bench.sh	H200 上 256k 输入/4k 输出长上下文探测
DSV4 H200 vLLM TP2 Custom Bench	experiments/dsv4_h200_vllm_tp2_custom_bench/run_bench.sh	H200 上 vLLM TP=2 自定义压测客户端（多服务负载均衡）
DSV4 H200 vLLM TP4 Custom Bench	experiments/dsv4_h200_vllm_tp4_custom_bench/run_bench.sh	H200 上 vLLM TP=4 自定义压测客户端
DSV4 H200 vLLM TP8 Custom Bench	experiments/dsv4_h200_vllm_tp8_custom_bench/run_bench.sh	H200 上 vLLM TP=8 自定义压测客户端
DSV4 H200 vLLM MTP vs Default	experiments/dsv4_h200_vllm_mtp_vs_default/run_bench.sh	H200 上 vLLM MTP（Multi-Token Prediction）vs 默认配置对比
DSV4 H200 Long Context Matrix	experiments/dsv4_h200_long_context_matrix/run_bench.sh	H200 上长上下文矩阵测试（多后端对比）
DSV4 RTX 6000D SGLang TP/DP Adaptive	experiments/dsv4_pro6000_sglang_tp_dp_matrix/run_adaptive_concurrency.sh	RTX 6000D 上 SGLang 自适应并发饱和点搜索
DSV4 RTX 6000D vLLM TP/DP Adaptive	experiments/dsv4_pro6000_vllm_tp_dp_matrix/run_adaptive_concurrency.sh	RTX 6000D 上 vLLM 自适应并发饱和点搜索

旧结构（scripts/ + bench_results/）

仅保留还在被 experiments/dsv4_h200_dspark/ 引用的 legacy DSpark 脚本，其余旧脚本已删除。

实验	脚本	最终报告	说明
DSpark grid benchmark	scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh	experiments/legacy_bench_results/dspark_grid_20260707-132641/report.md	P1/P2/P3 全量网格
DSpark spec-tokens 对比	scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh	experiments/legacy_bench_results/dspark_st_comparison_20260707-150649/comparison_report.md	--spec-tokens 3 vs 5

快速复现

P800 SGLang

bash experiments/dsv4_p800_sglang/run_bench.sh

H200 DSpark

bash experiments/dsv4_h200_dspark/run_bench.sh

H200 SGLang baseline

bash experiments/dsv4_h200_sglang/run_bench.sh

H200 SGLang vs vLLM 对比

bash experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh

H200 vLLM DSpark vs default

bash experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh

H200 SGLang TP/DP Matrix

bash experiments/dsv4_h200_sglang_tp_dp_matrix/run_bench.sh

H200 SGLang TP/DP 自适应并发搜索

bash experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh

H200 vLLM TP/DP Matrix

bash experiments/dsv4_h200_vllm_tp_dp_matrix/run_bench.sh

H200 vLLM TP/DP 自适应并发搜索

bash experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh

H200 64k SGLang vs vLLM

bash experiments/dsv4_h200_64k_sglang_vs_vllm/run_bench.sh

H200 256k 4k Probe

bash experiments/dsv4_h200_256k_4k_probe/run_bench.sh

H200 vLLM TP2/TP4/TP8 Custom Bench

bash experiments/dsv4_h200_vllm_tp2_custom_bench/run_bench.sh
bash experiments/dsv4_h200_vllm_tp4_custom_bench/run_bench.sh
bash experiments/dsv4_h200_vllm_tp8_custom_bench/run_bench.sh

H200 vLLM MTP vs Default

bash experiments/dsv4_h200_vllm_mtp_vs_default/run_bench.sh

H200 Long Context Matrix

bash experiments/dsv4_h200_long_context_matrix/run_bench.sh

RTX 6000D SGLang TP/DP 自适应并发搜索

bash experiments/dsv4_pro6000_sglang_tp_dp_matrix/run_adaptive_concurrency.sh

RTX 6000D vLLM TP/DP 自适应并发搜索

bash experiments/dsv4_pro6000_vllm_tp_dp_matrix/run_adaptive_concurrency.sh

DSpark grid / spec-tokens（旧结构）

bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh

解析已有结果

# 新结构
python3 experiments/dsv4_h200_dspark/parse_results.py \
  experiments/dsv4_h200_dspark/results/<run_id>

# 旧结构
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
  /data/user1/yy/experiments/legacy_bench_results/dspark_grid_20260707-132641

SLO 参考

详见 docs/SLO_STANDARDS.md。

主要关注指标：

• TTFT P95：S2（DeepSeek-V4-Flash 所在层）目标 < 3s
• TPOT：S2 目标 < 50ms

环境要求

• NVIDIA H200：Python env /data/user1/yy/envs/vllm-dspark（服务端）、/data/user1/yy/envs/sglang（压测客户端）；模型 /data/models/DeepSeek-V4-Flash、/data/models/DeepSeek-V4-Flash-DSpark。
• NVIDIA RTX 6000D：Python env /root/.miniconda3/envs/sglang（压测客户端）；模型 /data/hf_models/DeepSeek-V4-Flash；Docker 镜像 lmsysorg/sglang:latest、vllm/vllm-openai:latest。
• Kunlun P800：Docker 镜像 iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260511_4202；模型 /data1/models/DeepSeek-V4-Flash-INT8；压测客户端在容器内运行。

平台相关常量见 platforms/*.env。