[Feat] add DSV4-Pro long-decode cases
This commit is contained in:
parent
25418ec174
commit
06b017483c
@ -1,5 +1,9 @@
|
||||
# sskj — 多平台大模型推理性能基准测试项目
|
||||
|
||||
> **更新(2026-07-30 23:38:42 CST)**
|
||||
>
|
||||
> 为 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 Phase 1 增加两个最小长 Decode 补充场景:`1K → 4K, C=16` 用于观察持续长输出、KV 增长与稳态 Decode,`128K → 1K, C=1` 用于观察长上下文上的 Decode Attention。继续复用唯一 `run_quick_map.sh` 入口和 `CASE_IDS` 过滤,不新增启动脚本;场景验证单测由 9 点更新为 11 点,Shell、3 个 Python 单测和两点 Dry-run 已通过。
|
||||
>
|
||||
> **更新(2026-07-30 23:06:01 CST)**
|
||||
>
|
||||
> 完成 DeepSeek-V4-Pro 双机 Pro6000D SGLang TP16 Phase 1 正式 quick-map。Head 与 Worker 均通过 `mlx5_0/mlx5_3` 双 Rail `NET/IB + GDRDMA` 门禁;9 个固定点和 3 个混合 A/B 结果共 12/12 成功,总用时 28 分 36 秒。32K/128K 单请求 Prefill 输入吞吐为 2,652.76/2,710.16 token/s;在 C=32 Decode 中注入一个 128K Prefill 后,Output TPS 下降 24.08%,TPOT P95 增加 66.55%。阶段 HTML 已重写为只保留成功结果,并补充正式汇总、运行清单和 Phase 2 三个诊断负载;两节点容器和 16 张 GPU 已清理。
|
||||
|
||||
@ -6,21 +6,16 @@ DeepSeek-V4-Pro on two RTX PRO 6000 Blackwell nodes. It does not modify or call
|
||||
|
||||
## Network precondition
|
||||
|
||||
The 2026-07-30 audit found that the current service container does not expose
|
||||
`/dev/infiniband`, so NCCL falls back to `NET/Socket`. The quick-map previously
|
||||
selected a low-speed non-compute interface and produced invalid 1K/32K cold
|
||||
Prefill baselines. The Socket default is now `eth0`, one of the two deployment
|
||||
compute interfaces (`eth0/eth3`).
|
||||
|
||||
Declaring `NCCL_IB_HCA=mlx5_0,mlx5_3` does not by itself enable RDMA. The
|
||||
launcher now exposes only the three device nodes needed by the two deployment
|
||||
The launcher exposes only the three device nodes needed by the two deployment
|
||||
rails (`rdma_cm`, `uverbs0`, `uverbs3`) and fails before benchmarking unless
|
||||
both nodes' NCCL INFO logs prove `NET/IB` is using `mlx5_0` and `mlx5_3`.
|
||||
`NCCL_CROSS_NIC` has no effect while the active transport is `NET/Socket`.
|
||||
The 2026-07-30 Phase 1 run verified cross-node channels over
|
||||
`NET/IB/*/GDRDMA`. `eth0` remains the NCCL bootstrap and Socket-fallback
|
||||
interface.
|
||||
|
||||
## Scope
|
||||
|
||||
The fixed suite covers nine points:
|
||||
The fixed suite covers eleven points:
|
||||
|
||||
| Group | ISL | OSL | Concurrency |
|
||||
|---|---:|---:|---:|
|
||||
@ -28,6 +23,8 @@ The fixed suite covers nine points:
|
||||
| Prefill throughput | 32K | 1 | 16 |
|
||||
| Decode latency | 1K | 1K | 1 |
|
||||
| Decode throughput | 1K | 1K | 16 / 32 / 64 |
|
||||
| Long-output decode | 1K | 4K | 16 |
|
||||
| Long-context decode | 128K | 1K | 1 |
|
||||
| Balanced pressure | 32K | 1K | 8 |
|
||||
|
||||
The same main entry also runs a mixed-interference A/B:
|
||||
@ -47,7 +44,7 @@ attribution belongs to the later scheduler/interference phase.
|
||||
|---|---|
|
||||
| `run_quick_map.sh` | The only Shell entry: service lifecycle, fixed map, mixed A/B, cleanup |
|
||||
| `config.env` | Machine, model, SGLang and benchmark configuration |
|
||||
| `quick_map_scenarios.tsv` | Nine fixed workload points |
|
||||
| `quick_map_scenarios.tsv` | Eleven fixed workload points |
|
||||
| `quick_map_results.py` | Validate output and generate CSV/JSONL/Markdown summaries |
|
||||
| `tests/test_quick_map_results.py` | Result parser regression tests |
|
||||
|
||||
@ -77,7 +74,21 @@ CASE_IDS="short_prefill_latency_1k_c1,mid_prefill_latency_32k_c1,decode_latency_
|
||||
|
||||
Remove `DRY_RUN=1` only after the TP16 service has passed the `NET/IB` startup
|
||||
gate. An unknown `CASE_IDS` value fails before any benchmark request is sent.
|
||||
Leave `CASE_IDS` empty to run all nine fixed points.
|
||||
Leave `CASE_IDS` empty to run all eleven fixed points.
|
||||
|
||||
To run only the two long-decode supplements:
|
||||
|
||||
```bash
|
||||
export CASE_IDS="long_output_decode_1k_to_4k_c16,long_context_decode_128k_to_1k_c1"
|
||||
export RUN_ID="dsv4pro-phase1-long-decode-$(date +%Y%m%d-%H%M%S)"
|
||||
|
||||
trap 'bash run_quick_map.sh stop' EXIT INT TERM
|
||||
bash run_quick_map.sh start
|
||||
bash run_quick_map.sh fixed
|
||||
```
|
||||
|
||||
This uses the existing single entry for startup, the selected fixed cases, and
|
||||
trap-based cleanup. It does not run the mixed A/B suite.
|
||||
|
||||
## GPU run
|
||||
|
||||
|
||||
@ -71,7 +71,7 @@ RESULT_BASE="${RESULT_BASE:-${SCRIPT_DIR}/results}"
|
||||
RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}"
|
||||
|
||||
# Optional comma- or space-separated subset of quick_map_scenarios.tsv.
|
||||
# Leave empty for the complete nine-point fixed suite.
|
||||
# Leave empty for the complete eleven-point fixed suite.
|
||||
CASE_IDS="${CASE_IDS:-}"
|
||||
|
||||
DRY_RUN="${DRY_RUN:-0}"
|
||||
|
||||
@ -7,4 +7,6 @@ decode_latency_1k_to_1k_c1 decode_latency 1024 1024 1 1 1 1 Single-request decod
|
||||
decode_throughput_1k_to_1k_c16 decode_throughput 1024 1024 16 1 1 1 Decode throughput at C16
|
||||
decode_throughput_1k_to_1k_c32 decode_throughput 1024 1024 32 1 1 1 Decode throughput at C32
|
||||
decode_throughput_1k_to_1k_c64 decode_throughput 1024 1024 64 1 1 1 Decode throughput at C64
|
||||
long_output_decode_1k_to_4k_c16 long_output_decode 1024 4096 16 1 1 1 Sustained long-output decode and KV growth
|
||||
long_context_decode_128k_to_1k_c1 long_context_decode 131072 1024 1 1 1 0 Decode attention over a 128K context
|
||||
balanced_32k_to_1k_c8 balanced 32768 1024 8 1 1 0 Balanced prefill and decode pressure
|
||||
|
||||
|
@ -16,13 +16,19 @@ import quick_map_results # noqa: E402
|
||||
class QuickMapResultsTest(unittest.TestCase):
|
||||
def test_scenario_file_is_valid_and_unique(self) -> None:
|
||||
scenarios = quick_map_results.parse_scenarios(EXPERIMENT_DIR / "quick_map_scenarios.tsv")
|
||||
self.assertEqual(len(scenarios), 9)
|
||||
self.assertEqual(len({item["case_id"] for item in scenarios}), 9)
|
||||
self.assertEqual(len(scenarios), 11)
|
||||
self.assertEqual(len({item["case_id"] for item in scenarios}), 11)
|
||||
self.assertEqual(scenarios[2]["isl"], 131072)
|
||||
self.assertEqual(scenarios[2]["case_id"], "long_prefill_latency_128k_c1")
|
||||
self.assertEqual(scenarios[2]["prompt_multiplier"], 1)
|
||||
self.assertEqual(scenarios[2]["min_prompts"], 1)
|
||||
self.assertEqual(scenarios[2]["warmup_requests"], 0)
|
||||
by_id = {item["case_id"]: item for item in scenarios}
|
||||
self.assertEqual(by_id["long_output_decode_1k_to_4k_c16"]["osl"], 4096)
|
||||
self.assertEqual(by_id["long_output_decode_1k_to_4k_c16"]["concurrency"], 16)
|
||||
self.assertEqual(by_id["long_context_decode_128k_to_1k_c1"]["isl"], 131072)
|
||||
self.assertEqual(by_id["long_context_decode_128k_to_1k_c1"]["osl"], 1024)
|
||||
self.assertEqual(by_id["long_context_decode_128k_to_1k_c1"]["warmup_requests"], 0)
|
||||
|
||||
def test_latency_percentiles_fall_back_to_request_arrays(self) -> None:
|
||||
data = {
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user