106 lines
3.3 KiB
Markdown
106 lines
3.3 KiB
Markdown
# H200 Quick Start Guide
|
|
|
|
This guide covers how to use this benchmark repository on an NVIDIA H200 machine.
|
|
|
|
## Current state
|
|
|
|
The H200 path is currently based on **native host virtual environments** (not Docker):
|
|
|
|
- Server engine: `vllm-dspark`
|
|
- Server env: `envs/vllm-dspark`
|
|
- Benchmark client env: `envs/sglang` (uses `sglang.bench_serving --backend vllm`)
|
|
- Default model: `/data/models/DeepSeek-V4-Flash-DSpark`
|
|
- Default port: `30004`
|
|
|
|
The legacy benchmark scripts live under `scripts/benchmark_dspark_0707/`. A migration wrapper experiment `experiments/dsv4_h200_dspark/` is also provided so the results follow the new `experiments/<name>/results/<RUN_ID>/` layout.
|
|
|
|
## 1. Pull and verify
|
|
|
|
```bash
|
|
git clone <repo-url> /data/user1/yy
|
|
cd /data/user1/yy
|
|
|
|
# Platform should auto-detect as nvidia_h200
|
|
source scripts/common/platform.sh
|
|
```
|
|
|
|
If auto-detection fails, set it explicitly:
|
|
|
|
```bash
|
|
PLATFORM=nvidia_h200 source scripts/common/platform.sh
|
|
```
|
|
|
|
## 2. Run the existing DSpark benchmark grid
|
|
|
|
This uses the legacy scripts and writes results to `bench_results/dspark_grid_<RUN_ID>/`.
|
|
|
|
```bash
|
|
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
```
|
|
|
|
To reuse an already-running server:
|
|
|
|
```bash
|
|
SKIP_MANAGE_SERVER=1 bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
```
|
|
|
|
## 3. Run the migration wrapper experiment
|
|
|
|
This produces results in the new experiment-centric layout:
|
|
|
|
```bash
|
|
bash experiments/dsv4_h200_dspark/run_bench.sh
|
|
```
|
|
|
|
Results land in `experiments/dsv4_h200_dspark/results/<RUN_ID>/`.
|
|
|
|
## 4. Create a new H200 experiment
|
|
|
|
To add a new H200 benchmark (for example a different model or engine), create:
|
|
|
|
```
|
|
experiments/<your_name>/
|
|
├── README.md # What this experiment measures
|
|
├── config.env # Model, port, scenarios, venv paths
|
|
├── start_server.sh # (optional) native server launch
|
|
├── run_bench.sh # Orchestrator: server → benchmark → stop
|
|
└── parse_results.py # Generate results.json + report.md
|
|
```
|
|
|
|
Minimum `config.env`:
|
|
|
|
```bash
|
|
EXPERIMENT="${EXPERIMENT:-<your_name>}"
|
|
MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}"
|
|
MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}"
|
|
PORT="${PORT:-30004}"
|
|
BACKEND="${BACKEND:-vllm}"
|
|
ENGINE="${ENGINE:-vllm-dspark}"
|
|
|
|
# Native venvs
|
|
VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}"
|
|
VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}"
|
|
|
|
SCENARIOS=(
|
|
"32 512 256"
|
|
)
|
|
```
|
|
|
|
Then source the platform loader and shared helpers at the top of `run_bench.sh`:
|
|
|
|
```bash
|
|
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
|
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
|
```
|
|
|
|
For a concrete example, see `experiments/dsv4_h200_dspark/`.
|
|
|
|
## 5. Cross-platform comparison
|
|
|
|
跨平台对比脚本目前未统一提供。可分别读取各实验 `results/<run_id>/results.json` 中的结构化数据，按 scenario 聚合后生成对比表。
|
|
|
|
## Notes
|
|
|
|
- `platforms/nvidia_h200.env` assumes the legacy directory layout under `/data/user1/yy`. If your H200 machine uses different paths, override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT`.
|
|
- Native server management helpers are not yet as mature as the Docker helpers in `scripts/common/server_docker.sh`. The H200 wrapper currently delegates to the existing `scripts/start_dsv4_dspark_8card.sh`.
|