# DSV4 P800 SGLang TP×DP Matrix（官方推荐参数版）

Kunlun P800 XPU + SGLang-XPU + `DeepSeek-V4-Flash-INT8` TP×DP matrix benchmark。
与 `dsv4_p800_sglang_tp_dp_matrix` 的区别仅在服务端启动参数：本目录使用
**厂商官方推荐参数**（提取自镜像内 `/workspace/ds_v4/w8a8/run_server.sh`）：

- `--ep-size`（仅 DP=1 时传入）：官方生产配置为 TP8/EP8。注意：**该镜像的
  EP 不能跨 DP 副本**——DP>1 时传 `--ep-size 8` 会在
  `data_parallel_controller.py` 触发 ZeroDivisionError（2026-07-17 实测）。
  因此 TP2/DP4 依然不可行（EP=2 时每卡 132 GiB > 96 GiB），本矩阵只跑
  TP4/DP2 与 TP8/DP1 两种配置。
- EAGLE 投机解码：`--speculative-algorithm EAGLE --speculative-num-steps 3
  --speculative-eagle-topk 1 --speculative-num-draft-tokens 4`
  （官方实测 accept length ≈ 2.5）。
- `--chunked-prefill-size 8192 --max-prefill-tokens 16384`：长 prefill 切片调度。
- `--max-running-requests 64`。

目的：量化官方参数相对此前实验参数的提升（TPOT / 长输入 TTFT / TP2 可行性）。
其余（shape 矩阵、自适应搜索规则、结果结构）与 `dsv4_p800_sglang_tp_dp_matrix`
完全一致，结果可直接对比。

Differences from the H20 version:

- Server runs in the P800 vendor image with the proven INT8 launch args
  (NSA attention backend, `w8a8_int8` quantization);
  the benchmark client runs inside the same container via `docker exec`.
- ISL is capped at 131072 because P800 INT8 sustains at most ~131k input
  tokens; concurrency ranges match the H20 matrix (up to 128 at ISL=1024).
  Scenarios beyond server capacity are recorded as failed/skipped and the
  run moves on.
- XPU memory sampling uses `xpu-smi` instead of `nvidia-smi`.

`--dp-size` support on this image was probed empirically (2026-07-16,
`results/smoke_tpdp`): TP=4/DP=2 starts and serves correctly (replicas land
on disjoint XPU groups), while TP=2/DP=4 fails during model loading with a
weight-loading OOM. Root cause: expert weights are sharded only across
the TP group (this matrix passes no `--ep-size`, so experts fall back to
TP sharding; the image also asserts `ep_size <= tp_size`, so EP can never
exceed TP anyway) and DP replicas do not share expert shards. The INT8
checkpoint holds ~264 GiB of routed-expert weights out of ~274 GiB total,
so each rank carries roughly `274/TP` GiB:
TP=8/DP=1 ≈ 34 GiB, TP=4/DP=2 ≈ 68 GiB, TP=2/DP=4 ≈ 137 GiB — more than
the 96 GiB an XPU has, hence the OOM. TP=4 is therefore the
minimum viable TP for this model on P800. Configs that cannot start are
recorded as `SKIPPED_SERVICE_START_FAILED` and the run continues with the
next config.

## Quick Start

```bash
# Full matrix (server per config -> warmup -> scenarios -> parse -> compare)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Dry run: print server args and scenario plan only
DRY_RUN=1 bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Adaptive concurrency search (find saturation point per shape)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
```

Results land in `results/<RUN_ID>/` (fixed matrix) or
`adaptive_results/<RUN_ID>/` (adaptive search).

## Files

| File | Purpose |
|---|---|
| `config.env` | Experiment-level configuration (model, port, TP×DP configs, server settings) |
| `matrix.json` | ISL/OSL matrix with Y/P/N marks and per-ISL concurrency ranges |
| `generate_scenarios.py` | Expands matrix.json into a scenario TSV |
| `start_sglang_docker.sh` | Start the P800 SGLang Docker container for a TP×DP config |
| `start_sglang_dp.sh` | Alias for `start_sglang_docker.sh` (H20 naming parity) |
| `run_bench.sh` | Orchestrator: server per config → warmup → scenarios → parse → compare |
| `compare.py` | Cross-config comparison report (`comparison.md`) |
| `adaptive_config.env` | Adaptive concurrency search settings |
| `run_adaptive_concurrency.sh` | Adaptive saturation search (shared `adaptive_bench_lib.sh`) |
| `run_adaptive_concurrency_add16.sh` | Same search starting at concurrency 16, step +16 |

## Platform

This experiment targets `platforms/kunlun_p800.env`. The container uses the
dedicated name `sglang-dsv4-flash-tpdp` so it never interferes with the
`sglang-dsv4-flash` container used by the other P800 experiments.