# DSV4 P800 SGLang TP×DP Matrix

Kunlun P800 XPU + SGLang-XPU + `DeepSeek-V4-Flash-INT8` TP×DP matrix benchmark,
modeled after `experiments/h20/dsv4_h20_sglang_tp_dp_matrix`.

## What it does

Runs each parallel configuration with its own server instance and sweeps the
ISL/OSL matrix in `matrix.json`:

- TP=2, DP=4 → 2 XPUs per replica, 4 replicas
- TP=4, DP=2 → 4 XPUs per replica, 2 replicas
- TP=8, DP=1 → 8 XPUs, no data parallelism (production configuration)

Differences from the H20 version:

- Server runs in the P800 vendor image with the proven INT8 launch args
  (NSA attention backend, `w8a8_int8` quantization);
  the benchmark client runs inside the same container via `docker exec`.
- ISL is capped at 131072 and concurrency ranges are lower, because P800
  INT8 sustains at most ~131k input tokens and far less concurrency.
- XPU memory sampling uses `xpu-smi` instead of `nvidia-smi`.

`--dp-size` support on this image was probed empirically (2026-07-16,
`results/smoke_tpdp`): TP=4/DP=2 starts and serves correctly (replicas land
on disjoint XPU groups), while TP=2/DP=4 fails during model loading with a
weight-loading OOM. Root cause: expert weights are sharded only across
the TP group (this matrix passes no `--ep-size`, so experts fall back to
TP sharding; the image also asserts `ep_size <= tp_size`, so EP can never
exceed TP anyway) and DP replicas do not share expert shards. The INT8
checkpoint holds ~264 GiB of routed-expert weights out of ~274 GiB total,
so each rank carries roughly `274/TP` GiB:
TP=8/DP=1 ≈ 34 GiB, TP=4/DP=2 ≈ 68 GiB, TP=2/DP=4 ≈ 137 GiB — more than
the 96 GiB an XPU has, hence the OOM. TP=4 is therefore the
minimum viable TP for this model on P800. Configs that cannot start are
recorded as `SKIPPED_SERVICE_START_FAILED` and the run continues with the
next config.

## Quick Start

```bash
# Full matrix (server per config -> warmup -> scenarios -> parse -> compare)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Dry run: print server args and scenario plan only
DRY_RUN=1 bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Adaptive concurrency search (find saturation point per shape)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
```

Results land in `results/<RUN_ID>/` (fixed matrix) or
`adaptive_results/<RUN_ID>/` (adaptive search).

## Files

| File | Purpose |
|---|---|
| `config.env` | Experiment-level configuration (model, port, TP×DP configs, server settings) |
| `matrix.json` | ISL/OSL matrix with Y/P/N marks and per-ISL concurrency ranges |
| `generate_scenarios.py` | Expands matrix.json into a scenario TSV |
| `start_sglang_docker.sh` | Start the P800 SGLang Docker container for a TP×DP config |
| `start_sglang_dp.sh` | Alias for `start_sglang_docker.sh` (H20 naming parity) |
| `run_bench.sh` | Orchestrator: server per config → warmup → scenarios → parse → compare |
| `compare.py` | Cross-config comparison report (`comparison.md`) |
| `adaptive_config.env` | Adaptive concurrency search settings |
| `run_adaptive_concurrency.sh` | Adaptive saturation search (shared `adaptive_bench_lib.sh`) |
| `run_adaptive_concurrency_add16.sh` | Same search starting at concurrency 16, step +16 |

## Platform

This experiment targets `platforms/kunlun_p800.env`. The container uses the
dedicated name `sglang-dsv4-flash-tpdp` so it never interferes with the
`sglang-dsv4-flash` container used by the other P800 experiments.