90eecc4ede
- New experiment: experiments/dsv4_p800_long_context_matrix/ - Single backend: sglang-xpu in Docker on Kunlun P800 - Context groups: 64k (max_context_len=80000, max_running=10) and 128k (max_context_len=140000, max_running=5) - Scenarios: concurrency sweeps x output lengths 256/1024/4096 - Parameterized start_sglang.sh enforces max_running >= 2 to avoid P800 INT8 req_to_token_pool allocation failures - Reuses proven P800 INT8 environment, NSA/EAGLE settings, and bench_serving invocations from dsv4_p800_max_context_length
34 lines
772 B
Markdown
34 lines
772 B
Markdown
# P800 Long-Context Matrix for DeepSeek-V4-Flash-INT8
|
|
|
|
Long-context throughput/latency matrix for SGLang-XPU on Kunlun P800 (8x XPU).
|
|
|
|
## Scope
|
|
|
|
- Model: `DeepSeek-V4-Flash-INT8` (W8A8 INT8)
|
|
- Backend/engine: `sglang-xpu`
|
|
- Hardware: 8x Kunlun P800 XPU
|
|
- Input lengths: 64k, 128k
|
|
- Output lengths: 256, 1024, 4096
|
|
- Concurrency: sweeps per context group (see `config.env`)
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
cd /data1/yy/sskj/experiments/dsv4_p800_long_context_matrix
|
|
bash run_bench.sh
|
|
```
|
|
|
|
Optional environment overrides:
|
|
|
|
```bash
|
|
PORT=30013 RUN_ID=myrun bash run_bench.sh
|
|
```
|
|
|
|
## Outputs
|
|
|
|
Results are saved under `results/${RUN_ID}/`:
|
|
|
|
- `raw_outputs/`: raw `sglang.bench_serving` JSONL files
|
|
- `report.md`: parsed per-scenario metrics
|
|
- `results.json`: structured metadata + scenarios
|