f6a45f1ed8
- Add experiments/dsv4_p800_long_context_matrix for 64k/128k/256k context grid on DeepSeek-V4-Flash-INT8 with Kunlun P800. - Change bench_client_docker.sh to use --concurrency instead of --max-concurrency to match the user's long-context benchmark script. - Add SGLANG_HEALTH_CHECK_TIMEOUT override to server_docker.sh for slow long-context server startups. - Tune dsv4_p800_256k_4k_probe with lower max-running-requests and mem-fraction-static to avoid OOM on P800.
53 lines
1.7 KiB
Markdown
53 lines
1.7 KiB
Markdown
# DSV4 P800 Long-Context Matrix
|
|
|
|
Kunlun P800 XPU + SGLang + `DeepSeek-V4-Flash-INT8` long-context matrix benchmark.
|
|
|
|
## What it does
|
|
|
|
Runs a grid of input/output lengths to measure SGLang long-context serving
|
|
performance on P800:
|
|
|
|
- Input lengths: 64k, 128k, 256k
|
|
- Output lengths: 256, 1k, 4k
|
|
- Concurrency: 25 for 64k; 10, 5, 2 for 128k and 256k
|
|
|
|
Each context group (64k / 128k / 256k) is started with its own server instance
|
|
using the appropriate `--context-length` and `--max-running-requests`.
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Run the full matrix (starts server per group, runs all scenarios, stops server)
|
|
PLATFORM=kunlun_p800 bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
|
|
```
|
|
|
|
Results land in `experiments/dsv4_p800_long_context_matrix/results/<RUN_ID>/`.
|
|
|
|
## Configuration
|
|
|
|
Edit `config.env` or override via environment variables:
|
|
|
|
```bash
|
|
# Run with the tuned EAGLE speculative-decoding config instead of the baseline
|
|
SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
|
|
|
|
# Reduce concurrency for the 256k group if OOM occurs
|
|
CONTEXT_GROUPS=("64k 80000 25" "128k 140000 10" "256k 270000 5") \
|
|
PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_long_context_matrix/run_bench.sh
|
|
```
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `config.env` | Experiment-level configuration (model, scenarios, context groups) |
|
|
| `start_server.sh` | Start the P800 SGLang Docker container for a context group |
|
|
| `run_bench.sh` | Orchestrator: server per group → warmup → scenarios → parse |
|
|
|
|
## Platform
|
|
|
|
This experiment targets `platforms/kunlun_p800.env` and uses
|
|
`scripts/common/server_docker.sh` / `scripts/common/bench_client_docker.sh`.
|