yy-fighting 0fdd3749c7 Add P800 sglang TP/DP matrix experiment

- New experiments/p800/dsv4_p800_sglang_tp_dp_matrix: TP8/DP1, TP4/DP2,
  TP2/DP4 matrix with smoke results; TP2/DP4 documents the weight-loading
  OOM root cause (274 GiB INT8 weights sharded only across TP group).
- Launch args drop --ep-size/--chunked-prefill-size/--max-prefill-tokens/
  --max-running-requests; experts fall back to TP sharding.
- Move dsv4_p800_256k_4k_probe under experiments/p800/.
- scripts/common: jq-free parsing fixes in adaptive_bench_lib.sh and
  parse_backend.py.
- .gitignore: cover raw_outputs under nested platform experiment layout.

2026-07-16 06:49:21 +00:00

1.8 KiB

Raw Blame History

DSV4 P800 256k-Input / 4k-Output Probe

Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 long-context OOM probe.

What it does

Sends 4 prompts with 262144 input tokens and requests 4096 output tokens at concurrency=2, then checks whether the server completes the scenario without OOM or context-length errors.

Quick Start

Against an already-running server (recommended on shared P800 machines)

PLATFORM=kunlun_p800 \
SKIP_MANAGE_SERVER=1 \
PORT=30000 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh

Standalone (starts its own Docker container)

PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh

This will start a fresh sglang-dsv4-flash container with the baseline w8a8_int8 launch args (no EAGLE speculative decoding), plus --context-length 270000 --max-running-requests 4, and tear it down on exit.

To use the tuned config with EAGLE speculative decoding instead:

SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh

Configuration

Edit config.env or override via environment variables:

# Use a different port or model path
PORT=30001 \
MODEL_PATH=/data1/models/DeepSeek-V4-Flash-INT8 \
PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh

Files

File	Purpose
`config.env`	Experiment-level configuration (model, port, scenario)
`start_server.sh`	Start the P800 SGLang Docker container with 256k context args
`run_bench.sh`	Orchestrator: server → warmup → probe scenario → stop server
`parse_results.py`	Parse logs and generate `results.json` + `report.md`

Platform

This experiment targets platforms/kunlun_p800.env.

1.8 KiB Raw Blame History