0fdd3749c7
- New experiments/p800/dsv4_p800_sglang_tp_dp_matrix: TP8/DP1, TP4/DP2, TP2/DP4 matrix with smoke results; TP2/DP4 documents the weight-loading OOM root cause (274 GiB INT8 weights sharded only across TP group). - Launch args drop --ep-size/--chunked-prefill-size/--max-prefill-tokens/ --max-running-requests; experts fall back to TP sharding. - Move dsv4_p800_256k_4k_probe under experiments/p800/. - scripts/common: jq-free parsing fixes in adaptive_bench_lib.sh and parse_backend.py. - .gitignore: cover raw_outputs under nested platform experiment layout.
DSV4 P800 256k-Input / 4k-Output Probe
Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 long-context OOM probe.
What it does
Sends 4 prompts with 262144 input tokens and requests 4096 output tokens at concurrency=2, then checks whether the server completes the scenario without OOM or context-length errors.
Quick Start
Against an already-running server (recommended on shared P800 machines)
PLATFORM=kunlun_p800 \
SKIP_MANAGE_SERVER=1 \
PORT=30000 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh
Standalone (starts its own Docker container)
PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh
This will start a fresh sglang-dsv4-flash container with the baseline
w8a8_int8 launch args (no EAGLE speculative decoding), plus
--context-length 270000 --max-running-requests 4, and tear it down on exit.
To use the tuned config with EAGLE speculative decoding instead:
SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh
Configuration
Edit config.env or override via environment variables:
# Use a different port or model path
PORT=30001 \
MODEL_PATH=/data1/models/DeepSeek-V4-Flash-INT8 \
PLATFORM=kunlun_p800 \
bash experiments/p800/dsv4_p800_256k_4k_probe/run_bench.sh
Files
| File | Purpose |
|---|---|
config.env |
Experiment-level configuration (model, port, scenario) |
start_server.sh |
Start the P800 SGLang Docker container with 256k context args |
run_bench.sh |
Orchestrator: server → warmup → probe scenario → stop server |
parse_results.py |
Parse logs and generate results.json + report.md |
Platform
This experiment targets platforms/kunlun_p800.env.