f6a45f1ed8
- Add experiments/dsv4_p800_long_context_matrix for 64k/128k/256k context grid on DeepSeek-V4-Flash-INT8 with Kunlun P800. - Change bench_client_docker.sh to use --concurrency instead of --max-concurrency to match the user's long-context benchmark script. - Add SGLANG_HEALTH_CHECK_TIMEOUT override to server_docker.sh for slow long-context server startups. - Tune dsv4_p800_256k_4k_probe with lower max-running-requests and mem-fraction-static to avoid OOM on P800.
DSV4 P800 Long-Context Matrix
Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 long-context matrix benchmark.
What it does
Runs a grid of input/output lengths to measure SGLang long-context serving performance on P800:
- Input lengths: 64k, 128k, 256k
- Output lengths: 256, 1k, 4k
- Concurrency: 25 for 64k; 10, 5, 2 for 128k and 256k
Each context group (64k / 128k / 256k) is started with its own server instance
using the appropriate --context-length and --max-running-requests.
Quick Start
# Run the full matrix (starts server per group, runs all scenarios, stops server)
PLATFORM=kunlun_p800 bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
Results land in experiments/dsv4_p800_long_context_matrix/results/<RUN_ID>/.
Configuration
Edit config.env or override via environment variables:
# Run with the tuned EAGLE speculative-decoding config instead of the baseline
SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
# Reduce concurrency for the 256k group if OOM occurs
CONTEXT_GROUPS=("64k 80000 25" "128k 140000 10" "256k 270000 5") \
PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
Files
| File | Purpose |
|---|---|
config.env |
Experiment-level configuration (model, scenarios, context groups) |
start_server.sh |
Start the P800 SGLang Docker container for a context group |
run_bench.sh |
Orchestrator: server per group → warmup → scenarios → parse |
Platform
This experiment targets platforms/kunlun_p800.env and uses
scripts/common/server_docker.sh / scripts/common/bench_client_docker.sh.