be94b8e7ef
- Add experiments/dsv4_p800_256k_4k_probe for testing 256k input + 4k output on DeepSeek-V4-Flash-INT8 with Kunlun P800. - Add SGLANG_EXTRA_LAUNCH_ARGS support to scripts/common/server_docker.sh so probe can inject --context-length 270000. - Add w8a8_int8_baseline server mode without EAGLE speculative decoding, matching the long-context P800 tuning. - Switch dsv4_p800_sglang and the new probe to use w8a8_int8_baseline by default; original tuned mode remains available via SERVER_MODE=w8a8_int8.
64 lines
1.7 KiB
Markdown
64 lines
1.7 KiB
Markdown
# DSV4 P800 256k-Input / 4k-Output Probe
|
|
|
|
Kunlun P800 XPU + SGLang + `DeepSeek-V4-Flash-INT8` long-context OOM probe.
|
|
|
|
## What it does
|
|
|
|
Sends 4 prompts with **262144 input tokens** and requests **4096 output tokens**
|
|
at **concurrency=2**, then checks whether the server completes the scenario
|
|
without OOM or context-length errors.
|
|
|
|
## Quick Start
|
|
|
|
### Against an already-running server (recommended on shared P800 machines)
|
|
|
|
```bash
|
|
PLATFORM=kunlun_p800 \
|
|
SKIP_MANAGE_SERVER=1 \
|
|
PORT=30000 \
|
|
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh
|
|
```
|
|
|
|
### Standalone (starts its own Docker container)
|
|
|
|
```bash
|
|
PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh
|
|
```
|
|
|
|
This will start a fresh `sglang-dsv4-flash` container with the baseline
|
|
`w8a8_int8` launch args (no EAGLE speculative decoding), plus
|
|
`--context-length 270000 --max-running-requests 4`, and tear it down on exit.
|
|
|
|
To use the tuned config with EAGLE speculative decoding instead:
|
|
|
|
```bash
|
|
SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Edit `config.env` or override via environment variables:
|
|
|
|
```bash
|
|
# Use a different port or model path
|
|
PORT=30001 \
|
|
MODEL_PATH=/data1/models/DeepSeek-V4-Flash-INT8 \
|
|
PLATFORM=kunlun_p800 \
|
|
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh
|
|
```
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `config.env` | Experiment-level configuration (model, port, scenario) |
|
|
| `start_server.sh` | Start the P800 SGLang Docker container with 256k context args |
|
|
| `run_bench.sh` | Orchestrator: server → warmup → probe scenario → stop server |
|
|
| `parse_results.py` | Parse logs and generate `results.json` + `report.md` |
|
|
|
|
## Platform
|
|
|
|
This experiment targets `platforms/kunlun_p800.env`.
|