1482601ae5
- Document the rule in docs/EXPERIMENT_GUIDE.md section 7 - Update all config.env SCENARIOS to follow the 5x rule - Migrate dsv4_h200_sglang, dsv4_h200_vllm, dsv4_p800_sglang from global NUM_PROMPTS to per-scenario num_prompts - Keep long-context phase2 as exception (low counts documented) - Add legacy 3-field fallback in run_bench.sh loops
37 lines
1.5 KiB
Bash
37 lines
1.5 KiB
Bash
# Experiment-level configuration for dsv4_p800_sglang
|
|
# All values can be overridden via environment variables.
|
|
|
|
EXPERIMENT="${EXPERIMENT:-dsv4_p800_sglang}"
|
|
MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-INT8}"
|
|
MODEL_PATH="${MODEL_PATH:-/data1/models/DeepSeek-V4-Flash-INT8}"
|
|
SERVED_MODEL_NAME="${SERVED_MODEL_NAME:-deepseek-v4-flash-int8}"
|
|
PORT="${PORT:-30000}"
|
|
BACKEND="${BACKEND:-sglang}"
|
|
ENGINE="${ENGINE:-sglang-xpu}"
|
|
DATASET="${DATASET:-random}"
|
|
# Path to a local ShareGPT-style dataset inside the container.
|
|
# Required when HF Hub is offline. The repo ships a dummy dataset under
|
|
# platforms/patches/kunlun_p800/dummy_sharegpt.json; for managed mode it is
|
|
# mounted to /workspace/dummy_sharegpt.json.
|
|
DATASET_PATH="${DATASET_PATH:-/workspace/dummy_sharegpt.json}"
|
|
WARMUP="${WARMUP:-0}"
|
|
# Default to a small number of prompts for a quick smoke test. Increase to 512
|
|
# or more for a real throughput measurement.
|
|
NUM_PROMPTS="${NUM_PROMPTS:-10}"
|
|
|
|
# Server launch mode for P800 SGLang.
|
|
# - "fp8" : non-INT8 DeepSeek-V4-Flash checkpoint
|
|
# - "w8a8_int8": DeepSeek-V4-Flash-INT8 checkpoint
|
|
SERVER_MODE="${SERVER_MODE:-w8a8_int8}"
|
|
|
|
# Scenario list: "concurrency input_len output_len num_prompts" (one per line).
|
|
# Following docs/EXPERIMENT_GUIDE.md: num_prompts = concurrency * 5.
|
|
# Default to a single scenario for fast validation. Override to run a full grid, e.g.:
|
|
# SCENARIOS=("32 512 256 160" "128 512 256 640")
|
|
if [[ -z "${SCENARIOS:-}" ]]; then
|
|
SCENARIOS=(
|
|
"32 512 256 160"
|
|
)
|
|
fi
|
|
|