docs(h200): H200 quick-start guide, porting workflow, and example experiment

- Update platforms/nvidia_h200.env with ENGINE and configurable venv/paths. - Update platforms/README.md with H200 notes and platform-addition guide. - Add docs/H200_QUICKSTART.md with concrete H200 usage steps. - Add experiments/dsv4_h200_dspark/ as a migration wrapper around the existing scripts/benchmark_dspark_0707/ grid benchmark, writing results into the new experiments/<name>/results/<RUN_ID>/ layout. - Update BENCHMARK_WORKFLOW.md with a 'Adding a New Platform or Experiment' section and mention the H200 wrapper in Quick Start. - Update main README.md to index the H200 experiment and link to the guide.
2026-07-08 05:55:11 +00:00 · 2026-07-08 05:55:11 +00:00 · a5d69dfd9a
commit a5d69dfd9a
parent f0d4c4753b
9 changed files with 374 additions and 7 deletions
--- a/BENCHMARK_WORKFLOW.md
+++ b/BENCHMARK_WORKFLOW.md
@ -218,12 +218,18 @@ The JSON file inside each `bench_results/<run>/` directory should follow a stabl

 ## Quick Start

-### Run P800 SGLang benchmark
+### Run P800 SGLang benchmark (Kunlun P800, Docker)

 ```bash
 bash experiments/dsv4_p800_sglang/run_bench.sh
 ```

+### Run H200 DSpark benchmark (NVIDIA H200, native venv)
+
+```bash
+bash experiments/dsv4_h200_dspark/run_bench.sh
+```
+
 ### Run legacy DSpark grid benchmark

 ```bash
@ -254,6 +260,63 @@ python3 experiments/dsv4_p800_sglang/parse_results.py \
 python3 scripts/analysis/compare_experiments.py
 ```

+## Adding a New Platform or Experiment
+
+### 1. Add or update a platform config
+
+Create `platforms/<chip>.env` with identity and platform-wide paths:
+
+```bash
+CHIP="my_chip"
+ACCELERATOR="My Accelerator"
+HARDWARE="8x My Accelerator"
+ENGINE="vllm-myengine"
+DEFAULT_PORT="30000"
+MODEL_ROOT="/data/models"
+```
+
+For Docker-based platforms, also set `DOCKER_IMAGE`, `CONTAINER_NAME`,
+`CONTAINER_PYTHON`, and `PATCH_ROOT` (see `platforms/kunlun_p800.env`).
+For host-native platforms, set the relevant venv paths (see
+`platforms/nvidia_h200.env`).
+
+### 2. Create an experiment directory
+
+```
+experiments/<name>/
+├── README.md          # Purpose and usage
+├── config.env         # Model, port, scenarios, engine overrides
+├── start_server.sh    # (optional) platform-specific server launch
+├── run_bench.sh       # Orchestrator
+└── parse_results.py   # Convert raw outputs to results.json + report.md
+```
+
+At minimum, `run_bench.sh` should:
+
+1. Source `scripts/common/lib.sh` and `scripts/common/platform.sh`.
+2. Read `config.env`.
+3. Create `experiments/<name>/results/<RUN_ID>/`.
+4. Call `write_metadata_json` to create `results.json`.
+5. Run the benchmark scenarios.
+6. Call `parse_results.py` to generate `report.md`.
+
+### 3. Reuse shared helpers
+
+- `scripts/common/lib.sh`: logging, health checks, metadata JSON.
+- `scripts/common/platform.sh`: platform auto-detection and env loading.
+- `scripts/common/server_docker.sh`: Docker-based server lifecycle.
+- `scripts/common/bench_client_docker.sh`: run `sglang.bench_serving` inside a container.
+
+Native (non-Docker) platforms can delegate to existing start scripts or add
+new shared helpers under `scripts/common/`.
+
+### 4. Example experiments
+
+- Docker / XPU: `experiments/dsv4_p800_sglang/`
+- Native / H200: `experiments/dsv4_h200_dspark/`
+
+See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough.
+
 ## Checklist Before Committing / Archiving

 - [ ] No `.jsonl`, `.json`, `.log`, or `.md` files left in the project root.
--- a/README.md
+++ b/README.md
@ -1,6 +1,7 @@
 # DSV4 / DeepSeek-V4-Flash 推理测速项目

 > 记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。
+> H200 快速上手指南见 [`docs/H200_QUICKSTART.md`](docs/H200_QUICKSTART.md)。

 ## 目录说明

@ -24,6 +25,7 @@
 | 实验 | 脚本 | 说明 |
 |---|---|---|
 | DSV4 P800 SGLang | `experiments/dsv4_p800_sglang/run_bench.sh` | Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8 |
+| DSV4 H200 DSpark | `experiments/dsv4_h200_dspark/run_bench.sh` | NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark |

 ### 旧结构（scripts/ + bench_results/）

@ -46,7 +48,13 @@
 bash experiments/dsv4_p800_sglang/run_bench.sh
 ```

-### DSpark grid
+### H200 DSpark
+
+```bash
+bash experiments/dsv4_h200_dspark/run_bench.sh
+```
+
+### DSpark grid（旧结构）

 ```bash
 bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
@ -65,6 +73,9 @@ bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
 python3 experiments/dsv4_p800_sglang/parse_results.py \
  experiments/dsv4_p800_sglang/results/<run_id>

+python3 experiments/dsv4_h200_dspark/parse_results.py \
+  experiments/dsv4_h200_dspark/results/<run_id>
+
 # 旧结构
 /data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
  /data/user1/yy/bench_results/dspark_grid_20260707-132641
--- a/docs/H200_QUICKSTART.md
+++ b/docs/H200_QUICKSTART.md
@ -0,0 +1,111 @@
+# H200 Quick Start Guide
+
+This guide covers how to use this benchmark repository on an NVIDIA H200 machine.
+
+## Current state
+
+The H200 path is currently based on **native host virtual environments** (not Docker):
+
+- Server engine: `vllm-dspark`
+- Server env: `envs/vllm-dspark`
+- Benchmark client env: `envs/sglang` (uses `sglang.bench_serving --backend vllm`)
+- Default model: `/data/models/DeepSeek-V4-Flash-DSpark`
+- Default port: `30004`
+
+The legacy benchmark scripts live under `scripts/benchmark_dspark_0707/`. A migration wrapper experiment `experiments/dsv4_h200_dspark/` is also provided so the results follow the new `experiments/<name>/results/<RUN_ID>/` layout.
+
+## 1. Pull and verify
+
+```bash
+git clone <repo-url> /data/user1/yy
+cd /data/user1/yy
+
+# Platform should auto-detect as nvidia_h200
+source scripts/common/platform.sh
+```
+
+If auto-detection fails, set it explicitly:
+
+```bash
+PLATFORM=nvidia_h200 source scripts/common/platform.sh
+```
+
+## 2. Run the existing DSpark benchmark grid
+
+This uses the legacy scripts and writes results to `bench_results/dspark_grid_<RUN_ID>/`.
+
+```bash
+bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
+```
+
+To reuse an already-running server:
+
+```bash
+SKIP_MANAGE_SERVER=1 bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
+```
+
+## 3. Run the migration wrapper experiment
+
+This produces results in the new experiment-centric layout:
+
+```bash
+bash experiments/dsv4_h200_dspark/run_bench.sh
+```
+
+Results land in `experiments/dsv4_h200_dspark/results/<RUN_ID>/`.
+
+## 4. Create a new H200 experiment
+
+To add a new H200 benchmark (for example a different model or engine), create:
+
+```
+experiments/<your_name>/
+├── README.md          # What this experiment measures
+├── config.env         # Model, port, scenarios, venv paths
+├── start_server.sh    # (optional) native server launch
+├── run_bench.sh       # Orchestrator: server → benchmark → stop
+└── parse_results.py   # Generate results.json + report.md
+```
+
+Minimum `config.env`:
+
+```bash
+EXPERIMENT="${EXPERIMENT:-<your_name>}"
+MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}"
+MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}"
+PORT="${PORT:-30004}"
+BACKEND="${BACKEND:-vllm}"
+ENGINE="${ENGINE:-vllm-dspark}"
+
+# Native venvs
+VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}"
+VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}"
+
+SCENARIOS=(
+  "32 512 256"
+)
+```
+
+Then source the platform loader and shared helpers at the top of `run_bench.sh`:
+
+```bash
+source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
+source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
+```
+
+For a concrete example, see `experiments/dsv4_h200_dspark/`.
+
+## 5. Cross-platform comparison
+
+After running experiments on both P800 and H200, compare them with:
+
+```bash
+python3 scripts/analysis/compare_experiments.py \
+  experiments/dsv4_p800_sglang/results/<run_id> \
+  experiments/dsv4_h200_dspark/results/<run_id>
+```
+
+## Notes
+
+- `platforms/nvidia_h200.env` assumes the legacy directory layout under `/data/user1/yy`. If your H200 machine uses different paths, override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT`.
+- Native server management helpers are not yet as mature as the Docker helpers in `scripts/common/server_docker.sh`. The H200 wrapper currently delegates to the existing `scripts/start_dsv4_dspark_8card.sh`.
--- a/experiments/dsv4_h200_dspark/README.md
+++ b/experiments/dsv4_h200_dspark/README.md
@ -0,0 +1,41 @@
+# DSV4 H200 DSpark Benchmark
+
+NVIDIA H200 + native `vllm-dspark` + `DeepSeek-V4-Flash-DSpark` benchmark experiment.
+
+This is a migration wrapper: it reuses the proven legacy scripts under
+`scripts/benchmark_dspark_0707/` and `scripts/start_dsv4_dspark_8card.sh`,
+but stores results in the new `experiments/<name>/results/<RUN_ID>/` layout.
+
+## Quick Start
+
+```bash
+# Run the full DSpark P1/P2/P3 grid
+bash experiments/dsv4_h200_dspark/run_bench.sh
+
+# Reuse an already-running server
+SKIP_MANAGE_SERVER=1 bash experiments/dsv4_h200_dspark/run_bench.sh
+```
+
+Results land in `experiments/dsv4_h200_dspark/results/<RUN_ID>/`.
+
+## Configuration
+
+Edit `config.env` or override via environment variables:
+
+```bash
+MODEL_PATH=/data/models/DeepSeek-V4-Flash-DSpark \
+PORT=30004 \
+bash experiments/dsv4_h200_dspark/run_bench.sh
+```
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `config.env` | Experiment-level configuration (model, port, venv paths) |
+| `run_bench.sh` | Orchestrator: metadata → legacy grid benchmark → parse |
+| `parse_results.py` | Wraps the legacy parser to generate `report.md` |
+
+## Platform
+
+This experiment targets `platforms/nvidia_h200.env`.
--- a/experiments/dsv4_h200_dspark/config.env
+++ b/experiments/dsv4_h200_dspark/config.env
@ -0,0 +1,22 @@
+# Experiment-level configuration for dsv4_h200_dspark
+# All values can be overridden via environment variables.
+
+EXPERIMENT="${EXPERIMENT:-dsv4_h200_dspark}"
+MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}"
+MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}"
+SERVED_MODEL_NAME="${SERVED_MODEL_NAME:-deepseek-v4-flash-dspark}"
+PORT="${PORT:-30004}"
+BACKEND="${BACKEND:-vllm}"
+ENGINE="${ENGINE:-vllm-dspark}"
+
+# Native virtual environments on the host.
+# Override if your H200 machine uses different paths.
+VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}"
+VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}"
+
+# Legacy server start script.
+SERVER_START_SCRIPT="${SERVER_START_SCRIPT:-${ROOT_DIR}/scripts/start_dsv4_dspark_8card.sh}"
+
+# Legacy benchmark orchestrator.
+LEGACY_GRID_SCRIPT="${LEGACY_GRID_SCRIPT:-${ROOT_DIR}/scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh}"
+LEGACY_PARSE_SCRIPT="${LEGACY_PARSE_SCRIPT:-${ROOT_DIR}/scripts/benchmark_dspark_0707/parse_results.py}"
--- a/experiments/dsv4_h200_dspark/parse_results.py
+++ b/experiments/dsv4_h200_dspark/parse_results.py
@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+"""Parse H200 DSpark benchmark outputs.
+
+This is a thin wrapper around the legacy parser in
+scripts/benchmark_dspark_0707/parse_results.py. It exists so that the
+experiment keeps the same entry point as other experiments.
+
+Usage:
+    python3 parse_results.py <result_root>
+"""
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+
+def main() -> None:
+    result_root = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("results")
+    repo_root = Path(__file__).resolve().parents[2]
+    legacy_parser = repo_root / "scripts" / "benchmark_dspark_0707" / "parse_results.py"
+
+    if not legacy_parser.exists():
+        raise SystemExit(f"legacy parser not found: {legacy_parser}")
+
+    venv_client = os.environ.get("VENV_CLIENT", "/data/user1/yy/envs/sglang")
+    python = Path(venv_client) / "bin" / "python"
+    if not python.exists():
+        python = Path(sys.executable)
+
+    subprocess.run([str(python), str(legacy_parser), str(result_root)], check=True)
+    print(f"Parsed {result_root}")
+
+
+if __name__ == "__main__":
+    main()
--- a/experiments/dsv4_h200_dspark/run_bench.sh
+++ b/experiments/dsv4_h200_dspark/run_bench.sh
@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# H200 native vllm-dspark benchmark orchestrator.
+# This wraps the legacy scripts/benchmark_dspark_0707/ grid benchmark so that
+# results live under experiments/dsv4_h200_dspark/results/<RUN_ID>/.
+
+set -Eeuo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
+
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+
+RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')}"
+RESULT_ROOT="${RESULT_ROOT:-${SCRIPT_DIR}/results/${RUN_ID}}"
+LOG_DIR="${RESULT_ROOT}/logs"
+
+ensure_result_root "$RESULT_ROOT"
+log_init "${LOG_DIR}/orchestrator.log"
+
+log "experiment=${EXPERIMENT_NAME}"
+log "run_id=${RUN_ID}"
+log "result_root=${RESULT_ROOT}"
+log "platform=${PLATFORM}"
+log "chip=${CHIP}"
+log "accelerator=${ACCELERATOR}"
+log "engine=${ENGINE}"
+log "hardware=${HARDWARE}"
+log "model=${MODEL_PATH}"
+log "server_start_script=${SERVER_START_SCRIPT}"
+
+# Write initial metadata for this run.
+METADATA_JSON="${RESULT_ROOT}/results.json"
+write_metadata_json \
+  "$METADATA_JSON" \
+  "$EXPERIMENT_NAME" \
+  "$RUN_ID" \
+  "$MODEL_PATH" \
+  "$BACKEND" \
+  "$ENGINE" \
+  "$HARDWARE" \
+  "$ACCELERATOR" \
+  "$CHIP" \
+  "experiments/${EXPERIMENT_NAME}/run_bench.sh" \
+  "$VENV_SERVER" \
+  "H200 native vllm-dspark benchmark for DeepSeek-V4-Flash-DSpark"
+
+log "starting legacy DSpark grid benchmark"
+RUN_ID="$RUN_ID" \
+RESULT_ROOT="$RESULT_ROOT" \
+PORT="$PORT" \
+MODEL_PATH="$MODEL_PATH" \
+SERVER_START_SCRIPT="$SERVER_START_SCRIPT" \
+SKIP_MANAGE_SERVER="${SKIP_MANAGE_SERVER:-}" \
+  bash "$LEGACY_GRID_SCRIPT" >> "${LOG_DIR}/legacy_grid.log" 2>&1
+
+log "parsing results"
+"$VENV_CLIENT/bin/python" "$LEGACY_PARSE_SCRIPT" "$RESULT_ROOT" >> "${LOG_DIR}/parse.log" 2>&1 || {
+  log "WARNING: legacy parser failed; see ${LOG_DIR}/parse.log"
+}
+
+log "all results saved to ${RESULT_ROOT}"
--- a/platforms/README.md
+++ b/platforms/README.md
@ -19,14 +19,15 @@ PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh
 | File | Chip/Accelerator | Engine | Notes |
 |---|---|---|---|
 | `kunlun_p800.env` | Kunlun P800 XPU | `sglang-xpu` | Docker-based SGLang serving image |
-| `nvidia_h200.env` | NVIDIA H200 | native vllm/sglang | Host virtual environments |
+| `nvidia_h200.env` | NVIDIA H200 | `vllm-dspark` | Native host virtual environments |

 ## What belongs here

- Chip/accelerator identity (`CHIP`, `ACCELERATOR`, `HARDWARE`).
+- Chip/accelerator identity (`CHIP`, `ACCELERATOR`, `HARDWARE`, `ENGINE`).
 - Device selection environment variables.
 - Platform-wide paths that rarely change (model root, default port).
 - Container image / interpreter paths for Docker-based platforms.
+- Native interpreter / venv paths for host-based platforms.

 ## What does NOT belong here

@ -34,3 +35,14 @@ PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh
  `experiments/<name>/config.env`.
 - Engine-specific launch flags — those go in the experiment's
  `start_server.sh` or `run_bench.sh`.
+
+## Adding a new platform
+
+1. Create `platforms/<chip>.env` with at least `CHIP`, `ACCELERATOR`,
+   `HARDWARE`, `ENGINE`, `DEFAULT_PORT`, `MODEL_ROOT`.
+2. If the platform runs inside Docker, set `DOCKER_IMAGE`, `CONTAINER_NAME`,
+   `CONTAINER_PYTHON`, and `PATCH_ROOT` (see `kunlun_p800.env`).
+3. If the platform runs natively on the host, set the relevant venv paths
+   (see `nvidia_h200.env`).
+4. Add a row to the table above and write a quick-start experiment under
+   `experiments/<name>/`.
--- a/platforms/nvidia_h200.env
+++ b/platforms/nvidia_h200.env
@ -4,6 +4,7 @@
 CHIP="nvidia_h200"
 ACCELERATOR="NVIDIA H200"
 HARDWARE="8x NVIDIA H200 143GB"
+ENGINE="vllm-dspark"

 # Device selection
 DEVICE_SELECT_ENV="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
@ -13,6 +14,10 @@ CUDA_VISIBLE_DEVICES="0,1,2,3,4,5,6,7"
 DEFAULT_PORT="30004"
 MODEL_ROOT="/data/models"

-# Virtual environments on the host (used by native H200 scripts)
-VENV_VLLM_DSPARK="/data/user1/yy/envs/vllm-dspark"
-VENV_SGLANG="/data/user1/yy/envs/sglang"
+# Virtual environments on the host (used by native H200 scripts).
+# Override these if your H200 machine uses different paths.
+VENV_VLLM_DSPARK="${VENV_VLLM_DSPARK:-/data/user1/yy/envs/vllm-dspark}"
+VENV_SGLANG="${VENV_SGLANG:-/data/user1/yy/envs/sglang}"
+
+# Default server start script for the legacy DSpark benchmark grid.
+SERVER_START_SCRIPT="${SERVER_START_SCRIPT:-${ROOT_DIR}/scripts/start_dsv4_dspark_8card.sh}"