diff --git a/README.md b/README.md index 4a7a715..20f80e8 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,9 @@ # sskj — 多平台大模型推理性能基准测试项目 +> **更新(2026-07-31 00:11:25 CST)** +> +> 完成 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 的长 Decode 补测,Run `dsv4pro-phase1-long-decode-20260730-234236` 结果 2/2 成功。`1K → 4K, C=16` 生成 65,536 个目标输出 token,Output TPS 为 310.02,TPOT P95 为 50.33 ms;相比 `1K → 1K, C=16`,Output TPS 增加 4.99%,TPOT P95 仅增加 0.62%。`128K → 1K, C=1` 生成完整 1,024 token,TTFT P95 为 49.326 s,TPOT P95 为 32.24 ms;其 TTFT 与 `128K → 1` 纯 Prefill 仅差 2.03%,TPOT P95 相比 1K Context 仅增加 2.47%。Phase 1 至此共完成 11 个固定点和 3 个混合结果,14/14 成功;两节点容器和 16 张 GPU 已清理。 +> > **更新(2026-07-30 23:38:42 CST)** > > 为 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 Phase 1 增加两个最小长 Decode 补充场景:`1K → 4K, C=16` 用于观察持续长输出、KV 增长与稳态 Decode,`128K → 1K, C=1` 用于观察长上下文上的 Decode Attention。继续复用唯一 `run_quick_map.sh` 入口和 `CASE_IDS` 过滤,不新增启动脚本;场景验证单测由 9 点更新为 11 点,Shell、3 个 Python 单测和两点 Dry-run 已通过。 diff --git a/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/README.md b/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/README.md index 9e9adf8..1c8d716 100644 --- a/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/README.md +++ b/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/README.md @@ -38,6 +38,38 @@ The same main entry also runs a mixed-interference A/B: This aggregate mixed test is sufficient for the first performance-map pass. Per-request time-series attribution belongs to the later scheduler/interference phase. +## Verified Phase 1 results + +The no-profiler baseline completed in two runs over the same TP16 service +configuration and the same dual-rail `NET/IB + GDRDMA` gate: + +| Run | Result | Scope | +|---|---:|---| +| `dsv4pro-phase1-full-20260730-220916` | 12/12 | 9 fixed points and 3 mixed A/B outputs | +| `dsv4pro-phase1-long-decode-20260730-234236` | 2/2 | Long-output and long-context Decode | + +The long-Decode supplement produced every configured output token: + +| Case | Output tokens | Output TPS | TTFT P95 | TPOT P95 | E2E P95 | +|---|---:|---:|---:|---:|---:| +| `1K -> 4K, C=16` | 65,536 | 310.02 | 6.241 s | 50.33 ms | 211.364 s | +| `128K -> 1K, C=1` | 1,024 | 12.43 | 49.326 s | 32.24 ms | 82.312 s | + +Compared with `1K -> 1K, C=16`, the long-output case improved aggregate +Output TPS by 4.99% while TPOT P95 changed by only +0.62%. The longer measured +Decode interval amortized fixed startup and Prefill costs; no sustained-Decode +throughput collapse was observed. + +For `128K -> 1K, C=1`, TTFT was only 2.03% above the `128K -> 1` Prefill-only +case, while TPOT P95 was only 2.47% above `1K -> 1K, C=1`. In this run, the +large added latency came mainly from Prefill. Its 12.43 Output TPS includes the +49-second Prefill in benchmark duration and is not a pure Decode-rate metric. + +Phase 1 therefore closes at 11/11 fixed points plus 3/3 mixed results. The next +diagnostic stage should collect aligned GPU, CPU, SGLang scheduler and dual-rail +RDMA time series for Prefill, ordinary Decode, sustained Decode, long-context +Decode and the mixed A/B. + ## Files | File | Purpose | @@ -139,8 +171,9 @@ inside ignored result directories and are not committed. - Quick default: one repetition and one measured request wave (`num_prompts=C`, including one request when `C=1`). - Milestone: three repetitions. -- Short Prefill and Decode cases use one warm-up request. Expensive 32K/128K - Prefill cases do not add a same-shape warm-up. +- Short Prefill, ordinary Decode and `1K -> 4K` long-output Decode use one + warm-up request. Expensive 32K/128K Prefill and `128K -> 1K` long-context + Decode do not add a same-shape warm-up. - Every isolated point calls SGLang benchmark with `--flush-cache`; the benchmark performs its warm-up first and then clears the server Prefix Cache before measured requests.