sskj/experiments/p800/dsv4_p800_sglang_tp_dp_official/README.md

DSV4 P800 SGLang TP×DP Matrix（官方推荐参数版）

Kunlun P800 XPU + SGLang-XPU + DeepSeek-V4-Flash-INT8 TP×DP matrix benchmark。 与 dsv4_p800_sglang_tp_dp_matrix 的区别仅在服务端启动参数：本目录使用 厂商官方推荐参数（提取自镜像内 /workspace/ds_v4/w8a8/run_server.sh）：

• --ep-size（仅 DP=1 时传入）：官方生产配置为 TP8/EP8。注意：该镜像的 EP 不能跨 DP 副本——DP>1 时传 --ep-size 8 会在 data_parallel_controller.py 触发 ZeroDivisionError（2026-07-17 实测）。 因此 TP2/DP4 依然不可行（EP=2 时每卡 132 GiB > 96 GiB），本矩阵只跑 TP4/DP2 与 TP8/DP1 两种配置。
• EAGLE 投机解码：--speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 （官方实测 accept length ≈ 2.5）。
• --chunked-prefill-size 8192 --max-prefill-tokens 16384：长 prefill 切片调度。
• --max-running-requests 64。

目的：量化官方参数相对此前实验参数的提升（TPOT / 长输入 TTFT / TP2 可行性）。 其余（shape 矩阵、自适应搜索规则、结果结构）与 dsv4_p800_sglang_tp_dp_matrix 完全一致，结果可直接对比。

Differences from the H20 version:

• Server runs in the P800 vendor image with the proven INT8 launch args (NSA attention backend, w8a8_int8 quantization); the benchmark client runs inside the same container via docker exec.
• ISL is capped at 131072 because P800 INT8 sustains at most ~131k input tokens; concurrency ranges match the H20 matrix (up to 128 at ISL=1024). Scenarios beyond server capacity are recorded as failed/skipped and the run moves on.
• XPU memory sampling uses xpu-smi instead of nvidia-smi.

--dp-size support on this image was probed empirically (2026-07-16, results/smoke_tpdp): TP=4/DP=2 starts and serves correctly (replicas land on disjoint XPU groups), while TP=2/DP=4 fails during model loading with a weight-loading OOM. Root cause: expert weights are sharded only across the TP group (this matrix passes no --ep-size, so experts fall back to TP sharding; the image also asserts ep_size <= tp_size, so EP can never exceed TP anyway) and DP replicas do not share expert shards. The INT8 checkpoint holds ~264 GiB of routed-expert weights out of ~274 GiB total, so each rank carries roughly 274/TP GiB: TP=8/DP=1 ≈ 34 GiB, TP=4/DP=2 ≈ 68 GiB, TP=2/DP=4 ≈ 137 GiB — more than the 96 GiB an XPU has, hence the OOM. TP=4 is therefore the minimum viable TP for this model on P800. Configs that cannot start are recorded as SKIPPED_SERVICE_START_FAILED and the run continues with the next config.

Quick Start

# Full matrix (server per config -> warmup -> scenarios -> parse -> compare)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Dry run: print server args and scenario plan only
DRY_RUN=1 bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_bench.sh

# Adaptive concurrency search (find saturation point per shape)
bash experiments/p800/dsv4_p800_sglang_tp_dp_matrix/run_adaptive_concurrency.sh

Results land in results/<RUN_ID>/ (fixed matrix) or adaptive_results/<RUN_ID>/ (adaptive search).

Files

File	Purpose
config.env	Experiment-level configuration (model, port, TP×DP configs, server settings)
matrix.json	ISL/OSL matrix with Y/P/N marks and per-ISL concurrency ranges
generate_scenarios.py	Expands matrix.json into a scenario TSV
start_sglang_docker.sh	Start the P800 SGLang Docker container for a TP×DP config
start_sglang_dp.sh	Alias for start_sglang_docker.sh (H20 naming parity)
run_bench.sh	Orchestrator: server per config → warmup → scenarios → parse → compare
compare.py	Cross-config comparison report (comparison.md)
adaptive_config.env	Adaptive concurrency search settings
run_adaptive_concurrency.sh	Adaptive saturation search (shared adaptive_bench_lib.sh)
run_adaptive_concurrency_add16.sh	Same search starting at concurrency 16, step +16

Platform

This experiment targets platforms/kunlun_p800.env. The container uses the dedicated name sglang-dsv4-flash-tpdp so it never interferes with the sglang-dsv4-flash container used by the other P800 experiments.