qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/README.md
shishi 63ab41b65a docs: consolidate project docs (dedup, relocate, expand 910C client guide) 
Project-level documentation was scattered and duplicated across README.md,
BENCHMARK_WORKFLOW.md, and docs/EXPERIMENT_GUIDE.md (directory layout +
scripts/common component table repeated 3x). Reorganize into a clear
single-source-of-truth structure.

Changes:
- README.md: drop the 6 stale changelog entries at the top (latest was
  07-21; history lives in git log). Replace the duplicated directory-
  layout + scripts/common sections with a one-line link to
  docs/EXPERIMENT_GUIDE.md. (151 -> 99 lines)
- BENCHMARK_WORKFLOW.md -> docs/BENCHMARK_WORKFLOW.md: relocate into docs/.
  Replace its duplicated Directory Layout and Quick Start/Adding sections
  with links to EXPERIMENT_GUIDE / README / NEW_PLATFORM_GUIDE; keep the
  unique parts (Rules, Naming Conventions, Final JSON Schema, Checklist).
  (394 -> 224 lines)
- docs/EXPERIMENT_GUIDE.md: now the single authority for directory layout
  + component table + experiment conventions. Add a cross-link from the
  results.json field list to BENCHMARK_WORKFLOW's full JSON Schema and
  Naming Conventions.
- docs/H200_QUICKSTART.md: deleted (outdated, repeatedly references
  removed legacy scripts; H200 usage is covered by ADAPTIVE_CONCURRENCY_USAGE
  and experiment READMEs).
- docs/DSV4_INFERENCE_COMPARISON_REPORT.md -> experiments/h200/
  dsv4_h200_vllm_mtp_vs_default/results/20260708-160349/: this is an
  experiment report, not a project doc; relocate next to its sibling
  report.md.
- envs/ASCEND_910C_ENV_SETUP.md §8: expand the vague "pip install sglang"
  note into a full sglang client image build guide -- pin sglang 0.5.2
  (not latest; >=0.5.16 deprecates bench_serving and breaks the parser),
  --no-deps minimal install loop, docker commit to a local image, with
  the exact commands used to build local/vllm-ascend:0.23-a3-dsv4-sglang.
- experiments/h200/dsv4_h200_vllm_tp2_custom_bench/README.md: fix the
  now-broken link to BENCHMARK_WORKFLOW.md (../../ -> ../../../docs/).
- .gitignore: ignore *.bak.glm52orig scratch backups.

Also includes the add16 adaptive_results produced by the dsv4 TP=4/DP=2
runs on 910c.1.
2026-07-29 11:52:42 +08:00

99 lines
6.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# sskj — 多平台大模型推理性能基准测试项目
> 历史更新见 `git log`。项目目的与工作流见下方。
> **项目目的**：当新显卡（GPU/NPU）到货时，用最短时间完成大模型在该平台上的推理性能评估与部署配置选型。
> 当前模型：DeepSeek-V4-Flash（FP8 / INT8）；后续接入 GLM5.2，**完全复用**本项目的实验与报告流程。
> 新平台接入 SOP：[`docs/NEW_PLATFORM_GUIDE.md`](docs/NEW_PLATFORM_GUIDE.md)。
## 这个项目解决什么问题
1. **新卡快速评估**：新平台到货后，按 SOP 复制实验目录、改少量配置，即可跑起与既有平台口径一致的 benchmark，横向可比。
2. **部署配置选型**：通过 TP×DP 并行矩阵 + 自适应并发搜索，回答"该平台上哪种并行配置在什么并发下满足 SLO、吞吐最高"。
3. **数据资产化**：每轮实验产出结构化结果（jsonl/csv/manifest），汇总成飞书性能报告并回填多维表格，形成可检索的历史数据库。
## 标准工作流（全流程）
```
① 跑实验 experiments/<platform>/<experiment>/run_adaptive_concurrency*.sh
└─ 公共搜索逻辑: scripts/common/adaptive_bench_lib.sh
② 落盘 adaptive_results/<run_id>/
└─ adaptive_points.jsonl / adaptive_shapes.jsonl / adaptive_summary.* / run_manifest.json
③ 分析 用 Python 从 jsonl 实算（禁止目测），SLO 口径见 docs/SLO_STANDARDS.md
④ 飞书报告 按飞书 wiki「性能报告编写指南（README）」写到「显卡性能报告/<平台>」节点下
⑤ 回填多维表格 飞书「模型推理适配 Bench 迭代跟踪」：每成功探测点一行
```
- SLO 口径（S2 层，DSV4-Flash / GLM5.2 均适用）：**TTFT P95 < 3s，TPOT P95 < 50ms**，严格小于。详见 [`docs/SLO_STANDARDS.md`](docs/SLO_STANDARDS.md)。
- 报告编写规范（章节骨架、分析口径、发布流程）以飞书 wiki「显卡性能报告 / 性能报告编写指南（README）」为准。
- 多维表格每行 = 一个成功探测点（config、Concurrency、Throughput、Input/Output TPS、TTFT/TPOT/E2E P95、SLO达标状态、开发文档链接）。
## 目录结构
仓库的目录布局、`scripts/common/` 组件职责、结果命名规范与 `results.json` schema，统一见 [`docs/EXPERIMENT_GUIDE.md`](docs/EXPERIMENT_GUIDE.md)（单一权威来源）与 [`docs/BENCHMARK_WORKFLOW.md`](docs/BENCHMARK_WORKFLOW.md)。
## 实验索引
### TP/DP matrix + 自适应并发（当前主流形态）
| 实验 | 说明 |
|---|---|
| `experiments/h20/dsv4_h20_vllm_tp_dp_matrix/` | H20 + vLLM，TP2/DP4、TP4/DP2、TP8/DP1 |
| `experiments/h20/dsv4_h20_sglang_tp_dp_matrix/` | H20 + SGLang，同上 |
| `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` | H200 + vLLM |
| `experiments/h200/dsv4_h200_sglang_tp_dp_matrix/` | H200 + SGLang |
| `experiments/p800/dsv4_p800_sglang_tp_dp_matrix/` | P800 + SGLang（INT8；TP2/DP4 启动 OOM 无数据，见 config.env 注释） |
| `experiments/pro6000/dsv4_pro6000_vllm_tp_dp_matrix/` | RTX 6000D + vLLM |
| `experiments/pro6000/dsv4_pro6000_sglang_tp_dp_matrix/` | RTX 6000D + SGLang |
每个目录内：`run_bench.sh` 跑固定并发矩阵；`run_adaptive_concurrency.sh` 从 C=1 指数倍增搜饱和点；`run_adaptive_concurrency_add16.sh` 从 C=16 线性 +16 步进、带 TTFT SLO 停止与回退（当前主力用法，见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`）。
### 其他实验（H200 老形态 & 专项）
| 实验 | 说明 |
|---|---|
| `experiments/h200/dsv4_h200_sglang/` `…/dsv4_h200_vllm/` | H200 单引擎 baseline |
| `experiments/h200/dsv4_h200_sglang_vs_vllm/` | H200 引擎控制变量对比 |
| `experiments/h200/dsv4_h200_dspark/` `…/dsv4_h200_vllm_dspark_vs_default/` | DSpark 投机解码相关 |
| `experiments/h200/dsv4_h200_vllm_mtp_vs_default/` | vLLM MTP 对比 |
| `experiments/h200/dsv4_h200_vllm_tp{2,4,8}_custom_bench/` | 自定义压测客户端（多服务负载均衡） |
| `experiments/h200/dsv4_h200_max_context_length/` `…/dsv4_h200_long_context_matrix/` `…/dsv4_h200_256k_4k_probe/` `…/dsv4_h200_64k_sglang_vs_vllm/` | 长上下文专项 |
| `experiments/p800/dsv4_p800_sglang/` `…/dsv4_p800_max_context_length/` `…/dsv4_p800_long_context_matrix/` `…/dsv4_p800_256k_4k_probe/` | P800 baseline 与长上下文专项 |
| `experiments/TEMPLATE/` | 老式固定场景实验模板 |
## 快速复现
以 H20 vLLM 自适应并发搜索为例（其他平台同理，换目录即可）：
```bash
# 1. dry-run：只打印搜索计划，不加载模型
cd experiments/h20/dsv4_h20_vllm_tp_dp_matrix
DRY_RUN=1 bash run_adaptive_concurrency_add16.sh
# 2. 单组合冒烟：只测 TP=8、1K/128，并发上限 8
RUN_ID=smoke-$(date +%Y%m%d-%H%M%S) \
TP_LIST="8" ISL_LIST="1024" OSL_LIST="128" GRID_LIMIT=1 SEARCH_MAX_CONCURRENCY=8 \
bash run_adaptive_concurrency_add16.sh
# 3. 正式跑（放 tmux）
tmux new-session -d -s h20-vllm-adaptive \
"cd $(pwd) && bash run_adaptive_concurrency_add16.sh"
```
更多用法（断点续跑 RESUME_RUN_ID、常用覆盖参数）见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`。
## 新平台 / 新模型接入
- **新显卡**：按 [`docs/NEW_PLATFORM_GUIDE.md`](docs/NEW_PLATFORM_GUIDE.md) 执行，核心动作是复制 `experiments/h20/dsv4_h20_<engine>_tp_dp_matrix`、改 `config.env` / `adaptive_config.env` / `matrix.json` 三件套，再加 `platforms/<chip>.env`。
- **新模型（GLM5.2）**：复用同一实验目录结构，新增 `experiments/<platform>/glm52_<platform>_<engine>_tp_dp_matrix/`，改模型路径、量化与引擎启动参数即可；SLO 口径、报告流程、多维表格结构不变。
## 环境要求
- 公共：bash、Python 3、`sglang.bench_serving` 压测客户端（Docker 或 venv）；绘图分析用 `envs/charts/`（matplotlib）。
- 各平台的镜像 / venv / 模型路径见对应实验目录的 `config.env` 与 `platforms/<chip>.env`；**模型与数据集路径是机器相关的**，换机器时按实际路径调整。
- 环境搭建规范见 `envs/README.md` 与 `envs/UV_ENV_SETUP.md`。
## 注意事项
- 提交内容：实验代码 + 最终产物（results.json / report.md / adaptive 的 jsonl 与 summary）；日志、raw_outputs、gpu_logs 不入库（`.gitignore` 已排除）。
- 仓库会clone到多台机器，实验脚本内引用公共组件一律用 `${SCRIPT_DIR}/../../../scripts/common`（TEMPLATE 为 `../../`），不要写绝对路径。
Reference in New Issue View Git Blame Copy Permalink