Quantong Qiu e72e34c2f7 fix(p800): managed-mode server startup and smoke-test defaults

- Remove USE_FAST_BFP16_MOE=1 which caused torch parameter assignment
  crash in fresh containers for w8a8_int8 mode.
- Mount model path at its host absolute path inside the container so the
  bench client tokenizer can load it regardless of server --model-path.
- Default WARMUP to 0 and NUM_PROMPTS to 10 for a fast smoke test.
- Default to a single scenario; document how to run a full grid.
- Update experiment README with PLATFORM env var and override examples.

2026-07-08 05:49:55 +00:00

1.2 KiB

Raw Blame History

DSV4 P800 SGLang Benchmark

Kunlun P800 XPU + SGLang + DeepSeek-V4-Flash-INT8 benchmark experiment.

Quick Start

# Run the default smoke-test scenario (starts server, runs benchmark, stops server)
PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh

# Reuse an already-running server
SKIP_MANAGE_SERVER=1 PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh

Results land in experiments/dsv4_p800_sglang/results/<RUN_ID>/.

Configuration

Edit config.env or override via environment variables:

# Run a full throughput grid instead of the default single smoke-test scenario
SCENARIOS=("32 512 256" "128 512 256" "256 512 256" "512 512 256") \
NUM_PROMPTS=512 \
PLATFORM=kunlun_p800 \
bash experiments/dsv4_p800_sglang/run_bench.sh

Files

File	Purpose
`config.env`	Experiment-level configuration (model, port, scenarios)
`start_server.sh`	Start the P800 SGLang Docker container
`run_bench.sh`	Orchestrator: server → benchmark → stop server
`parse_results.py`	Parse logs and generate `results.json` + `report.md`

Platform

This experiment targets platforms/kunlun_p800.env.

1.2 KiB Raw Blame History