sskj/docs/H200_QUICKSTART.md

H200 Quick Start Guide

This guide covers how to use this benchmark repository on an NVIDIA H200 machine.

Current state

The older H200 experiments (e.g. experiments/h200/dsv4_h200_dspark/) are based on native host virtual environments (not Docker):

• Server engine: vllm-dspark
• Server env: envs/vllm-dspark
• Benchmark client env: envs/sglang (uses sglang.bench_serving --backend vllm)
• Default model: /data/models/DeepSeek-V4-Flash-DSpark
• Default port: 30004

Note: the current mainstream H200 experiments — the TP/DP matrix adaptive-concurrency suites experiments/h200/dsv4_h200_vllm_tp_dp_matrix/ and experiments/h200/dsv4_h200_sglang_tp_dp_matrix/ — are Docker-based (USE_DOCKER=1 for the server, benchmark client via DOCKER_CLIENT_IMAGE); see experiments/ADAPTIVE_CONCURRENCY_USAGE.md.

The legacy benchmark suite scripts/benchmark_dspark_0707/ has been removed from this repo. The migration wrapper experiment experiments/h200/dsv4_h200_dspark/ still exists and produces the experiments/h200/<name>/results/<RUN_ID>/ layout, but its default LEGACY_GRID_SCRIPT / SERVER_START_SCRIPT point at the removed scripts and must be overridden to run.

1. Pull and verify

git clone <repo-url> sskj   # clone path: adjust to your machine
cd sskj

# Platform should auto-detect as nvidia_h200
source scripts/common/platform.sh

If auto-detection fails, set it explicitly:

PLATFORM=nvidia_h200 source scripts/common/platform.sh

2. Legacy DSpark benchmark grid (removed)

The legacy suite scripts/benchmark_dspark_0707/ has been removed from this repo, along with its bench_results/dspark_grid_<RUN_ID>/ output location (historical results were archived elsewhere). Use the wrapper experiment below, or the current tp_dp matrix experiments (experiments/h200/dsv4_h200_{vllm,sglang}_tp_dp_matrix/), instead.

3. Run the migration wrapper experiment

This produces results in the experiment-centric layout:

bash experiments/h200/dsv4_h200_dspark/run_bench.sh

Results land in experiments/h200/dsv4_h200_dspark/results/<RUN_ID>/.

Caveat: the wrapper's default LEGACY_GRID_SCRIPT and SERVER_START_SCRIPT refer to the removed scripts/benchmark_dspark_0707/ suite and scripts/start_dsv4_dspark_8card.sh; set both explicitly (or restore equivalent scripts) before running.

4. Create a new H200 experiment

To add a new H200 benchmark (for example a different model or engine), create:

experiments/h200/<your_name>/
├── README.md          # What this experiment measures
├── config.env         # Model, port, scenarios, venv paths
├── start_server.sh    # (optional) native server launch
├── run_bench.sh       # Orchestrator: server → benchmark → stop
└── parse_results.py   # Generate results.json + report.md

Minimum config.env:

EXPERIMENT="${EXPERIMENT:-<your_name>}"
MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}"
MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}"
PORT="${PORT:-30004}"
BACKEND="${BACKEND:-vllm}"
ENGINE="${ENGINE:-vllm-dspark}"

# Native venvs (paths are per-machine; adjust to yours)
VENV_SERVER="${VENV_SERVER:-/path/to/envs/vllm-dspark}"
VENV_CLIENT="${VENV_CLIENT:-/path/to/envs/sglang}"

SCENARIOS=(
  "32 512 256"
)

Then source the platform loader and shared helpers at the top of run_bench.sh:

source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"

For a concrete example, see experiments/h200/dsv4_h200_dspark/. For the current mainstream form (TP/DP matrix + adaptive concurrency search, Docker-based), copy experiments/h200/dsv4_h200_vllm_tp_dp_matrix/ instead; usage in experiments/ADAPTIVE_CONCURRENCY_USAGE.md.

5. Cross-platform comparison

跨平台对比脚本目前未统一提供。可分别读取各实验 results/<run_id>/results.json 中的结构化数据，按 scenario 聚合后生成对比表。

Notes

• platforms/nvidia_h200.env still carries venv defaults under /data/user1/yy (paths from another machine). Override VENV_VLLM_DSPARK, VENV_SGLANG, MODEL_ROOT, or SERVER_START_SCRIPT to match your machine.
• Native server management helpers are not as mature as the Docker helpers in scripts/common/server_docker.sh. The wrapper's default SERVER_START_SCRIPT (scripts/start_dsv4_dspark_8card.sh) no longer exists in this repo and must be overridden.