diff --git a/BENCHMARK_WORKFLOW.md b/BENCHMARK_WORKFLOW.md index b0bd601..b5ffe05 100644 --- a/BENCHMARK_WORKFLOW.md +++ b/BENCHMARK_WORKFLOW.md @@ -218,12 +218,18 @@ The JSON file inside each `bench_results//` directory should follow a stabl ## Quick Start -### Run P800 SGLang benchmark +### Run P800 SGLang benchmark (Kunlun P800, Docker) ```bash bash experiments/dsv4_p800_sglang/run_bench.sh ``` +### Run H200 DSpark benchmark (NVIDIA H200, native venv) + +```bash +bash experiments/dsv4_h200_dspark/run_bench.sh +``` + ### Run legacy DSpark grid benchmark ```bash @@ -254,6 +260,63 @@ python3 experiments/dsv4_p800_sglang/parse_results.py \ python3 scripts/analysis/compare_experiments.py ``` +## Adding a New Platform or Experiment + +### 1. Add or update a platform config + +Create `platforms/.env` with identity and platform-wide paths: + +```bash +CHIP="my_chip" +ACCELERATOR="My Accelerator" +HARDWARE="8x My Accelerator" +ENGINE="vllm-myengine" +DEFAULT_PORT="30000" +MODEL_ROOT="/data/models" +``` + +For Docker-based platforms, also set `DOCKER_IMAGE`, `CONTAINER_NAME`, +`CONTAINER_PYTHON`, and `PATCH_ROOT` (see `platforms/kunlun_p800.env`). +For host-native platforms, set the relevant venv paths (see +`platforms/nvidia_h200.env`). + +### 2. Create an experiment directory + +``` +experiments// +├── README.md # Purpose and usage +├── config.env # Model, port, scenarios, engine overrides +├── start_server.sh # (optional) platform-specific server launch +├── run_bench.sh # Orchestrator +└── parse_results.py # Convert raw outputs to results.json + report.md +``` + +At minimum, `run_bench.sh` should: + +1. Source `scripts/common/lib.sh` and `scripts/common/platform.sh`. +2. Read `config.env`. +3. Create `experiments//results//`. +4. Call `write_metadata_json` to create `results.json`. +5. Run the benchmark scenarios. +6. Call `parse_results.py` to generate `report.md`. + +### 3. Reuse shared helpers + +- `scripts/common/lib.sh`: logging, health checks, metadata JSON. +- `scripts/common/platform.sh`: platform auto-detection and env loading. +- `scripts/common/server_docker.sh`: Docker-based server lifecycle. +- `scripts/common/bench_client_docker.sh`: run `sglang.bench_serving` inside a container. + +Native (non-Docker) platforms can delegate to existing start scripts or add +new shared helpers under `scripts/common/`. + +### 4. Example experiments + +- Docker / XPU: `experiments/dsv4_p800_sglang/` +- Native / H200: `experiments/dsv4_h200_dspark/` + +See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough. + ## Checklist Before Committing / Archiving - [ ] No `.jsonl`, `.json`, `.log`, or `.md` files left in the project root. diff --git a/README.md b/README.md index e039add..7b1c872 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # DSV4 / DeepSeek-V4-Flash 推理测速项目 > 记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。 +> H200 快速上手指南见 [`docs/H200_QUICKSTART.md`](docs/H200_QUICKSTART.md)。 ## 目录说明 @@ -24,6 +25,7 @@ | 实验 | 脚本 | 说明 | |---|---|---| | DSV4 P800 SGLang | `experiments/dsv4_p800_sglang/run_bench.sh` | Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8 | +| DSV4 H200 DSpark | `experiments/dsv4_h200_dspark/run_bench.sh` | NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark | ### 旧结构(scripts/ + bench_results/) @@ -46,7 +48,13 @@ bash experiments/dsv4_p800_sglang/run_bench.sh ``` -### DSpark grid +### H200 DSpark + +```bash +bash experiments/dsv4_h200_dspark/run_bench.sh +``` + +### DSpark grid(旧结构) ```bash bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh @@ -65,6 +73,9 @@ bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh python3 experiments/dsv4_p800_sglang/parse_results.py \ experiments/dsv4_p800_sglang/results/ +python3 experiments/dsv4_h200_dspark/parse_results.py \ + experiments/dsv4_h200_dspark/results/ + # 旧结构 /data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \ /data/user1/yy/bench_results/dspark_grid_20260707-132641 diff --git a/docs/H200_QUICKSTART.md b/docs/H200_QUICKSTART.md new file mode 100644 index 0000000..664dee2 --- /dev/null +++ b/docs/H200_QUICKSTART.md @@ -0,0 +1,111 @@ +# H200 Quick Start Guide + +This guide covers how to use this benchmark repository on an NVIDIA H200 machine. + +## Current state + +The H200 path is currently based on **native host virtual environments** (not Docker): + +- Server engine: `vllm-dspark` +- Server env: `envs/vllm-dspark` +- Benchmark client env: `envs/sglang` (uses `sglang.bench_serving --backend vllm`) +- Default model: `/data/models/DeepSeek-V4-Flash-DSpark` +- Default port: `30004` + +The legacy benchmark scripts live under `scripts/benchmark_dspark_0707/`. A migration wrapper experiment `experiments/dsv4_h200_dspark/` is also provided so the results follow the new `experiments//results//` layout. + +## 1. Pull and verify + +```bash +git clone /data/user1/yy +cd /data/user1/yy + +# Platform should auto-detect as nvidia_h200 +source scripts/common/platform.sh +``` + +If auto-detection fails, set it explicitly: + +```bash +PLATFORM=nvidia_h200 source scripts/common/platform.sh +``` + +## 2. Run the existing DSpark benchmark grid + +This uses the legacy scripts and writes results to `bench_results/dspark_grid_/`. + +```bash +bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh +``` + +To reuse an already-running server: + +```bash +SKIP_MANAGE_SERVER=1 bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh +``` + +## 3. Run the migration wrapper experiment + +This produces results in the new experiment-centric layout: + +```bash +bash experiments/dsv4_h200_dspark/run_bench.sh +``` + +Results land in `experiments/dsv4_h200_dspark/results//`. + +## 4. Create a new H200 experiment + +To add a new H200 benchmark (for example a different model or engine), create: + +``` +experiments// +├── README.md # What this experiment measures +├── config.env # Model, port, scenarios, venv paths +├── start_server.sh # (optional) native server launch +├── run_bench.sh # Orchestrator: server → benchmark → stop +└── parse_results.py # Generate results.json + report.md +``` + +Minimum `config.env`: + +```bash +EXPERIMENT="${EXPERIMENT:-}" +MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}" +MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}" +PORT="${PORT:-30004}" +BACKEND="${BACKEND:-vllm}" +ENGINE="${ENGINE:-vllm-dspark}" + +# Native venvs +VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}" +VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}" + +SCENARIOS=( + "32 512 256" +) +``` + +Then source the platform loader and shared helpers at the top of `run_bench.sh`: + +```bash +source "${SCRIPT_DIR}/../../scripts/common/lib.sh" +source "${SCRIPT_DIR}/../../scripts/common/platform.sh" +``` + +For a concrete example, see `experiments/dsv4_h200_dspark/`. + +## 5. Cross-platform comparison + +After running experiments on both P800 and H200, compare them with: + +```bash +python3 scripts/analysis/compare_experiments.py \ + experiments/dsv4_p800_sglang/results/ \ + experiments/dsv4_h200_dspark/results/ +``` + +## Notes + +- `platforms/nvidia_h200.env` assumes the legacy directory layout under `/data/user1/yy`. If your H200 machine uses different paths, override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT`. +- Native server management helpers are not yet as mature as the Docker helpers in `scripts/common/server_docker.sh`. The H200 wrapper currently delegates to the existing `scripts/start_dsv4_dspark_8card.sh`. diff --git a/experiments/dsv4_h200_dspark/README.md b/experiments/dsv4_h200_dspark/README.md new file mode 100644 index 0000000..e7e3d10 --- /dev/null +++ b/experiments/dsv4_h200_dspark/README.md @@ -0,0 +1,41 @@ +# DSV4 H200 DSpark Benchmark + +NVIDIA H200 + native `vllm-dspark` + `DeepSeek-V4-Flash-DSpark` benchmark experiment. + +This is a migration wrapper: it reuses the proven legacy scripts under +`scripts/benchmark_dspark_0707/` and `scripts/start_dsv4_dspark_8card.sh`, +but stores results in the new `experiments//results//` layout. + +## Quick Start + +```bash +# Run the full DSpark P1/P2/P3 grid +bash experiments/dsv4_h200_dspark/run_bench.sh + +# Reuse an already-running server +SKIP_MANAGE_SERVER=1 bash experiments/dsv4_h200_dspark/run_bench.sh +``` + +Results land in `experiments/dsv4_h200_dspark/results//`. + +## Configuration + +Edit `config.env` or override via environment variables: + +```bash +MODEL_PATH=/data/models/DeepSeek-V4-Flash-DSpark \ +PORT=30004 \ +bash experiments/dsv4_h200_dspark/run_bench.sh +``` + +## Files + +| File | Purpose | +|---|---| +| `config.env` | Experiment-level configuration (model, port, venv paths) | +| `run_bench.sh` | Orchestrator: metadata → legacy grid benchmark → parse | +| `parse_results.py` | Wraps the legacy parser to generate `report.md` | + +## Platform + +This experiment targets `platforms/nvidia_h200.env`. diff --git a/experiments/dsv4_h200_dspark/config.env b/experiments/dsv4_h200_dspark/config.env new file mode 100644 index 0000000..e06db6d --- /dev/null +++ b/experiments/dsv4_h200_dspark/config.env @@ -0,0 +1,22 @@ +# Experiment-level configuration for dsv4_h200_dspark +# All values can be overridden via environment variables. + +EXPERIMENT="${EXPERIMENT:-dsv4_h200_dspark}" +MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}" +MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}" +SERVED_MODEL_NAME="${SERVED_MODEL_NAME:-deepseek-v4-flash-dspark}" +PORT="${PORT:-30004}" +BACKEND="${BACKEND:-vllm}" +ENGINE="${ENGINE:-vllm-dspark}" + +# Native virtual environments on the host. +# Override if your H200 machine uses different paths. +VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}" +VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}" + +# Legacy server start script. +SERVER_START_SCRIPT="${SERVER_START_SCRIPT:-${ROOT_DIR}/scripts/start_dsv4_dspark_8card.sh}" + +# Legacy benchmark orchestrator. +LEGACY_GRID_SCRIPT="${LEGACY_GRID_SCRIPT:-${ROOT_DIR}/scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh}" +LEGACY_PARSE_SCRIPT="${LEGACY_PARSE_SCRIPT:-${ROOT_DIR}/scripts/benchmark_dspark_0707/parse_results.py}" diff --git a/experiments/dsv4_h200_dspark/parse_results.py b/experiments/dsv4_h200_dspark/parse_results.py new file mode 100755 index 0000000..c0450fa --- /dev/null +++ b/experiments/dsv4_h200_dspark/parse_results.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Parse H200 DSpark benchmark outputs. + +This is a thin wrapper around the legacy parser in +scripts/benchmark_dspark_0707/parse_results.py. It exists so that the +experiment keeps the same entry point as other experiments. + +Usage: + python3 parse_results.py +""" + +import os +import subprocess +import sys +from pathlib import Path + + +def main() -> None: + result_root = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("results") + repo_root = Path(__file__).resolve().parents[2] + legacy_parser = repo_root / "scripts" / "benchmark_dspark_0707" / "parse_results.py" + + if not legacy_parser.exists(): + raise SystemExit(f"legacy parser not found: {legacy_parser}") + + venv_client = os.environ.get("VENV_CLIENT", "/data/user1/yy/envs/sglang") + python = Path(venv_client) / "bin" / "python" + if not python.exists(): + python = Path(sys.executable) + + subprocess.run([str(python), str(legacy_parser), str(result_root)], check=True) + print(f"Parsed {result_root}") + + +if __name__ == "__main__": + main() diff --git a/experiments/dsv4_h200_dspark/run_bench.sh b/experiments/dsv4_h200_dspark/run_bench.sh new file mode 100755 index 0000000..c4b677a --- /dev/null +++ b/experiments/dsv4_h200_dspark/run_bench.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# H200 native vllm-dspark benchmark orchestrator. +# This wraps the legacy scripts/benchmark_dspark_0707/ grid benchmark so that +# results live under experiments/dsv4_h200_dspark/results//. + +set -Eeuo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")" + +# shellcheck source=/dev/null +source "${SCRIPT_DIR}/../../scripts/common/lib.sh" +# shellcheck source=/dev/null +source "${SCRIPT_DIR}/../../scripts/common/platform.sh" +# shellcheck source=/dev/null +source "${SCRIPT_DIR}/config.env" + +RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')}" +RESULT_ROOT="${RESULT_ROOT:-${SCRIPT_DIR}/results/${RUN_ID}}" +LOG_DIR="${RESULT_ROOT}/logs" + +ensure_result_root "$RESULT_ROOT" +log_init "${LOG_DIR}/orchestrator.log" + +log "experiment=${EXPERIMENT_NAME}" +log "run_id=${RUN_ID}" +log "result_root=${RESULT_ROOT}" +log "platform=${PLATFORM}" +log "chip=${CHIP}" +log "accelerator=${ACCELERATOR}" +log "engine=${ENGINE}" +log "hardware=${HARDWARE}" +log "model=${MODEL_PATH}" +log "server_start_script=${SERVER_START_SCRIPT}" + +# Write initial metadata for this run. +METADATA_JSON="${RESULT_ROOT}/results.json" +write_metadata_json \ + "$METADATA_JSON" \ + "$EXPERIMENT_NAME" \ + "$RUN_ID" \ + "$MODEL_PATH" \ + "$BACKEND" \ + "$ENGINE" \ + "$HARDWARE" \ + "$ACCELERATOR" \ + "$CHIP" \ + "experiments/${EXPERIMENT_NAME}/run_bench.sh" \ + "$VENV_SERVER" \ + "H200 native vllm-dspark benchmark for DeepSeek-V4-Flash-DSpark" + +log "starting legacy DSpark grid benchmark" +RUN_ID="$RUN_ID" \ +RESULT_ROOT="$RESULT_ROOT" \ +PORT="$PORT" \ +MODEL_PATH="$MODEL_PATH" \ +SERVER_START_SCRIPT="$SERVER_START_SCRIPT" \ +SKIP_MANAGE_SERVER="${SKIP_MANAGE_SERVER:-}" \ + bash "$LEGACY_GRID_SCRIPT" >> "${LOG_DIR}/legacy_grid.log" 2>&1 + +log "parsing results" +"$VENV_CLIENT/bin/python" "$LEGACY_PARSE_SCRIPT" "$RESULT_ROOT" >> "${LOG_DIR}/parse.log" 2>&1 || { + log "WARNING: legacy parser failed; see ${LOG_DIR}/parse.log" +} + +log "all results saved to ${RESULT_ROOT}" diff --git a/platforms/README.md b/platforms/README.md index a03c9f2..671fa70 100644 --- a/platforms/README.md +++ b/platforms/README.md @@ -19,14 +19,15 @@ PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh | File | Chip/Accelerator | Engine | Notes | |---|---|---|---| | `kunlun_p800.env` | Kunlun P800 XPU | `sglang-xpu` | Docker-based SGLang serving image | -| `nvidia_h200.env` | NVIDIA H200 | native vllm/sglang | Host virtual environments | +| `nvidia_h200.env` | NVIDIA H200 | `vllm-dspark` | Native host virtual environments | ## What belongs here -- Chip/accelerator identity (`CHIP`, `ACCELERATOR`, `HARDWARE`). +- Chip/accelerator identity (`CHIP`, `ACCELERATOR`, `HARDWARE`, `ENGINE`). - Device selection environment variables. - Platform-wide paths that rarely change (model root, default port). - Container image / interpreter paths for Docker-based platforms. +- Native interpreter / venv paths for host-based platforms. ## What does NOT belong here @@ -34,3 +35,14 @@ PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh `experiments//config.env`. - Engine-specific launch flags — those go in the experiment's `start_server.sh` or `run_bench.sh`. + +## Adding a new platform + +1. Create `platforms/.env` with at least `CHIP`, `ACCELERATOR`, + `HARDWARE`, `ENGINE`, `DEFAULT_PORT`, `MODEL_ROOT`. +2. If the platform runs inside Docker, set `DOCKER_IMAGE`, `CONTAINER_NAME`, + `CONTAINER_PYTHON`, and `PATCH_ROOT` (see `kunlun_p800.env`). +3. If the platform runs natively on the host, set the relevant venv paths + (see `nvidia_h200.env`). +4. Add a row to the table above and write a quick-start experiment under + `experiments//`. diff --git a/platforms/nvidia_h200.env b/platforms/nvidia_h200.env index a7a0e8e..7dae8de 100644 --- a/platforms/nvidia_h200.env +++ b/platforms/nvidia_h200.env @@ -4,6 +4,7 @@ CHIP="nvidia_h200" ACCELERATOR="NVIDIA H200" HARDWARE="8x NVIDIA H200 143GB" +ENGINE="vllm-dspark" # Device selection DEVICE_SELECT_ENV="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7" @@ -13,6 +14,10 @@ CUDA_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" DEFAULT_PORT="30004" MODEL_ROOT="/data/models" -# Virtual environments on the host (used by native H200 scripts) -VENV_VLLM_DSPARK="/data/user1/yy/envs/vllm-dspark" -VENV_SGLANG="/data/user1/yy/envs/sglang" +# Virtual environments on the host (used by native H200 scripts). +# Override these if your H200 machine uses different paths. +VENV_VLLM_DSPARK="${VENV_VLLM_DSPARK:-/data/user1/yy/envs/vllm-dspark}" +VENV_SGLANG="${VENV_SGLANG:-/data/user1/yy/envs/sglang}" + +# Default server start script for the legacy DSpark benchmark grid. +SERVER_START_SCRIPT="${SERVER_START_SCRIPT:-${ROOT_DIR}/scripts/start_dsv4_dspark_8card.sh}"