be94b8e7ef
- Add experiments/dsv4_p800_256k_4k_probe for testing 256k input + 4k output on DeepSeek-V4-Flash-INT8 with Kunlun P800. - Add SGLANG_EXTRA_LAUNCH_ARGS support to scripts/common/server_docker.sh so probe can inject --context-length 270000. - Add w8a8_int8_baseline server mode without EAGLE speculative decoding, matching the long-context P800 tuning. - Switch dsv4_p800_sglang and the new probe to use w8a8_int8_baseline by default; original tuned mode remains available via SERVER_MODE=w8a8_int8.
45 lines
1.3 KiB
Markdown
45 lines
1.3 KiB
Markdown
# DSV4 P800 SGLang Benchmark
|
|
|
|
Kunlun P800 XPU + SGLang + `DeepSeek-V4-Flash-INT8` benchmark experiment.
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Run the default smoke-test scenario (starts server, runs benchmark, stops server)
|
|
PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh
|
|
|
|
# Reuse an already-running server
|
|
SKIP_MANAGE_SERVER=1 PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh
|
|
```
|
|
|
|
Results land in `experiments/dsv4_p800_sglang/results/<RUN_ID>/`.
|
|
|
|
## Configuration
|
|
|
|
Edit `config.env` or override via environment variables:
|
|
|
|
```bash
|
|
# Run a full throughput grid instead of the default single smoke-test scenario
|
|
SCENARIOS=("32 512 256" "128 512 256" "256 512 256" "512 512 256") \
|
|
NUM_PROMPTS=512 \
|
|
PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_sglang/run_bench.sh
|
|
|
|
# Use the tuned config with EAGLE speculative decoding instead of the default baseline
|
|
SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_sglang/run_bench.sh
|
|
```
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `config.env` | Experiment-level configuration (model, port, scenarios) |
|
|
| `start_server.sh` | Start the P800 SGLang Docker container |
|
|
| `run_bench.sh` | Orchestrator: server → benchmark → stop server |
|
|
| `parse_results.py` | Parse logs and generate `results.json` + `report.md` |
|
|
|
|
## Platform
|
|
|
|
This experiment targets `platforms/kunlun_p800.env`.
|