History

yy-fighting be94b8e7ef Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

- Add experiments/dsv4_p800_256k_4k_probe for testing 256k input + 4k output
  on DeepSeek-V4-Flash-INT8 with Kunlun P800.
- Add SGLANG_EXTRA_LAUNCH_ARGS support to scripts/common/server_docker.sh
  so probe can inject --context-length 270000.
- Add w8a8_int8_baseline server mode without EAGLE speculative decoding,
  matching the long-context P800 tuning.
- Switch dsv4_p800_sglang and the new probe to use w8a8_int8_baseline by
  default; original tuned mode remains available via SERVER_MODE=w8a8_int8.

2026-07-08 10:30:05 +00:00

config.env

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

parse_results.py

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

README.md

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

run_bench.sh

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

start_server.sh

Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode

2026-07-08 10:30:05 +00:00

README.md

DSV4 P800 256k-Input / 4k-Output Probe

Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 long-context OOM probe.

What it does

Sends 4 prompts with 262144 input tokens and requests 4096 output tokens at concurrency=2, then checks whether the server completes the scenario without OOM or context-length errors.

Quick Start

Against an already-running server (recommended on shared P800 machines)

PLATFORM=kunlun_p800 \
SKIP_MANAGE_SERVER=1 \
PORT=30000 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Standalone (starts its own Docker container)

PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

This will start a fresh sglang-dsv4-flash container with the baseline w8a8_int8 launch args (no EAGLE speculative decoding), plus --context-length 270000 --max-running-requests 4, and tear it down on exit.

To use the tuned config with EAGLE speculative decoding instead:

SERVER_MODE=w8a8_int8 PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Configuration

Edit config.env or override via environment variables:

# Use a different port or model path
PORT=30001 \
MODEL_PATH=/data1/models/DeepSeek-V4-Flash-INT8 \
PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_256k_4k_probe/run_bench.sh

Files

File	Purpose
`config.env`	Experiment-level configuration (model, port, scenario)
`start_server.sh`	Start the P800 SGLang Docker container with 256k context args
`run_bench.sh`	Orchestrator: server → warmup → probe scenario → stop server
`parse_results.py`	Parse logs and generate `results.json` + `report.md`

Platform

This experiment targets platforms/kunlun_p800.env.