qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Docs] record DSV4-Pro long-decode results

Browse Source
This commit is contained in:
Zhiyi Hong 2026-07-31 00:15:43 +08:00
parent 06b017483c
commit ae85217225
2 changed files with 39 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -1,5 +1,9 @@
# sskj — 多平台大模型推理性能基准测试项目
> **更新（2026-07-31 00:11:25 CST）**
>
> 完成 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 的长 Decode 补测，Run `dsv4pro-phase1-long-decode-20260730-234236` 结果 2/2 成功。`1K → 4K, C=16` 生成 65,536 个目标输出 token，Output TPS 为 310.02，TPOT P95 为 50.33 ms；相比 `1K → 1K, C=16`，Output TPS 增加 4.99%，TPOT P95 仅增加 0.62%。`128K → 1K, C=1` 生成完整 1,024 token，TTFT P95 为 49.326 s，TPOT P95 为 32.24 ms；其 TTFT 与 `128K → 1` 纯 Prefill 仅差 2.03%，TPOT P95 相比 1K Context 仅增加 2.47%。Phase 1 至此共完成 11 个固定点和 3 个混合结果，14/14 成功；两节点容器和 16 张 GPU 已清理。
>
> **更新（2026-07-30 23:38:42 CST）**
>
> 为 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 Phase 1 增加两个最小长 Decode 补充场景：`1K → 4K, C=16` 用于观察持续长输出、KV 增长与稳态 Decode，`128K → 1K, C=1` 用于观察长上下文上的 Decode Attention。继续复用唯一 `run_quick_map.sh` 入口和 `CASE_IDS` 过滤，不新增启动脚本；场景验证单测由 9 点更新为 11 点，Shell、3 个 Python 单测和两点 Dry-run 已通过。

37
experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/README.md
View File

@ -38,6 +38,38 @@ The same main entry also runs a mixed-interference A/B:
This aggregate mixed test is sufficient for the first performance-map pass. Per-request time-series
attribution belongs to the later scheduler/interference phase.
## Verified Phase 1 results
The no-profiler baseline completed in two runs over the same TP16 service
configuration and the same dual-rail `NET/IB + GDRDMA` gate:
| Run | Result | Scope |
|---|---:|---|
| `dsv4pro-phase1-full-20260730-220916` | 12/12 | 9 fixed points and 3 mixed A/B outputs |
| `dsv4pro-phase1-long-decode-20260730-234236` | 2/2 | Long-output and long-context Decode |
The long-Decode supplement produced every configured output token:
| Case | Output tokens | Output TPS | TTFT P95 | TPOT P95 | E2E P95 |
|---|---:|---:|---:|---:|---:|
| `1K -> 4K, C=16` | 65,536 | 310.02 | 6.241 s | 50.33 ms | 211.364 s |
| `128K -> 1K, C=1` | 1,024 | 12.43 | 49.326 s | 32.24 ms | 82.312 s |
Compared with `1K -> 1K, C=16`, the long-output case improved aggregate
Output TPS by 4.99% while TPOT P95 changed by only +0.62%. The longer measured
Decode interval amortized fixed startup and Prefill costs; no sustained-Decode
throughput collapse was observed.
For `128K -> 1K, C=1`, TTFT was only 2.03% above the `128K -> 1` Prefill-only
case, while TPOT P95 was only 2.47% above `1K -> 1K, C=1`. In this run, the
large added latency came mainly from Prefill. Its 12.43 Output TPS includes the
49-second Prefill in benchmark duration and is not a pure Decode-rate metric.
Phase 1 therefore closes at 11/11 fixed points plus 3/3 mixed results. The next
diagnostic stage should collect aligned GPU, CPU, SGLang scheduler and dual-rail
RDMA time series for Prefill, ordinary Decode, sustained Decode, long-context
Decode and the mixed A/B.
## Files
| File | Purpose |
@ -139,8 +171,9 @@ inside ignored result directories and are not committed.
- Quick default: one repetition and one measured request wave
(`num_prompts=C`, including one request when `C=1`).
- Milestone: three repetitions.
- Short Prefill and Decode cases use one warm-up request. Expensive 32K/128K
Prefill cases do not add a same-shape warm-up.
- Short Prefill, ordinary Decode and `1K -> 4K` long-output Decode use one
warm-up request. Expensive 32K/128K Prefill and `128K -> 1K` long-context
Decode do not add a same-shape warm-up.
- Every isolated point calls SGLang benchmark with `--flush-cache`; the benchmark
performs its warm-up first and then clears the server Prefix Cache before
measured requests.