History

yy-fighting f6a45f1ed8 Add P800 long-context matrix and tune benchmark/health settings

- Add experiments/dsv4_p800_long_context_matrix for 64k/128k/256k context
  grid on DeepSeek-V4-Flash-INT8 with Kunlun P800.
- Change bench_client_docker.sh to use --concurrency instead of
  --max-concurrency to match the user's long-context benchmark script.
- Add SGLANG_HEALTH_CHECK_TIMEOUT override to server_docker.sh for slow
  long-context server startups.
- Tune dsv4_p800_256k_4k_probe with lower max-running-requests and
  mem-fraction-static to avoid OOM on P800.

2026-07-08 15:02:22 +00:00

config.env

Add P800 long-context matrix and tune benchmark/health settings

2026-07-08 15:02:22 +00:00

parse_results.py

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

README.md

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

run_bench.sh

Add P800 long-context matrix and tune benchmark/health settings

2026-07-08 15:02:22 +00:00

start_server.sh

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

README.md

DSV4 P800 256k-Input / 4k-Output Probe

Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 long-context OOM probe.

What it does

Sends 4 prompts with 262144 input tokens and requests 4096 output tokens at concurrency=2, then checks whether the server completes the scenario without OOM or context-length errors.

Quick Start

Against an already-running server (recommended on shared P800 machines)

PLATFORM=kunlun_p800 \
SKIP_MANAGE_SERVER=1 \
PORT=30000 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Standalone (starts its own Docker container)

PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

This will start a fresh sglang-dsv4-flash container with the baseline w8a8_int8 launch args (no EAGLE speculative decoding), plus --context-length 270000 --max-running-requests 4, and tear it down on exit.

To use the tuned config with EAGLE speculative decoding instead:

SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Configuration

Edit config.env or override via environment variables:

# Use a different port or model path
PORT=30001 \
MODEL_PATH=/data1/models/DeepSeek-V4-Flash-INT8 \
PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Files

File	Purpose
`config.env`	Experiment-level configuration (model, port, scenario)
`start_server.sh`	Start the P800 SGLang Docker container with 256k context args
`run_bench.sh`	Orchestrator: server → warmup → probe scenario → stop server
`parse_results.py`	Parse logs and generate `results.json` + `report.md`

Platform

This experiment targets platforms/kunlun_p800.env.