History

Quantong Qiu 46e05a57a4 Re-enable chunked prefill for P800 long-context matrix stability

Without chunked prefill the server OOM/crashes (exit 137) during or
immediately after concurrent 64k-context scenarios. Speculative decoding
remains disabled as requested.

2026-07-09 05:13:04 +00:00

config.env

Add P800 long-context matrix experiment (64k/128k) for DeepSeek-V4-Flash-INT8

2026-07-09 04:45:04 +00:00

README.md

Add P800 long-context matrix experiment (64k/128k) for DeepSeek-V4-Flash-INT8

2026-07-09 04:45:04 +00:00

run_bench.sh

Add P800 long-context matrix experiment (64k/128k) for DeepSeek-V4-Flash-INT8

2026-07-09 04:45:04 +00:00

start_sglang.sh

Re-enable chunked prefill for P800 long-context matrix stability

2026-07-09 05:13:04 +00:00

README.md

P800 Long-Context Matrix for DeepSeek-V4-Flash-INT8

Long-context throughput/latency matrix for SGLang-XPU on Kunlun P800 (8x XPU).

Scope

Model: DeepSeek-V4-Flash-INT8 (W8A8 INT8)
Backend/engine: sglang-xpu
Hardware: 8x Kunlun P800 XPU
Input lengths: 64k, 128k
Output lengths: 256, 1024, 4096
Concurrency: sweeps per context group (see config.env)

Usage

cd /data1/yy/sskj/experiments/dsv4_p800_long_context_matrix
bash run_bench.sh

Optional environment overrides:

PORT=30013 RUN_ID=myrun bash run_bench.sh

Outputs

Results are saved under results/${RUN_ID}/:

raw_outputs/: raw sglang.bench_serving JSONL files
report.md: parsed per-scenario metrics
results.json: structured metadata + scenarios