8652a685e6
- rewrite README with project purpose, standard workflow, corrected index - add docs/NEW_PLATFORM_GUIDE.md (new GPU onboarding SOP, GLM5.2 reuse) - fix ../../scripts/common -> ../../../scripts/common in 42 experiment scripts - refresh stale docs (EXPERIMENT_GUIDE, H200_QUICKSTART, ADAPTIVE_CONCURRENCY_USAGE, BENCHMARK_WORKFLOW) - remove dead code (dp_proxy.py) and .bak leftovers - add p800 adaptive results (tp4_dp2/tp8_dp1 metrics + summary) - gitignore envs/charts and .tmp_charts
100 lines
4.4 KiB
Markdown
100 lines
4.4 KiB
Markdown
# H200 Quick Start Guide
|
|
|
|
This guide covers how to use this benchmark repository on an NVIDIA H200 machine.
|
|
|
|
## Current state
|
|
|
|
The older H200 experiments (e.g. `experiments/h200/dsv4_h200_dspark/`) are based on **native host virtual environments** (not Docker):
|
|
|
|
- Server engine: `vllm-dspark`
|
|
- Server env: `envs/vllm-dspark`
|
|
- Benchmark client env: `envs/sglang` (uses `sglang.bench_serving --backend vllm`)
|
|
- Default model: `/data/models/DeepSeek-V4-Flash-DSpark`
|
|
- Default port: `30004`
|
|
|
|
Note: the current mainstream H200 experiments — the TP/DP matrix adaptive-concurrency suites `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` and `experiments/h200/dsv4_h200_sglang_tp_dp_matrix/` — are Docker-based (`USE_DOCKER=1` for the server, benchmark client via `DOCKER_CLIENT_IMAGE`); see `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`.
|
|
|
|
The legacy benchmark suite `scripts/benchmark_dspark_0707/` has been removed from this repo. The migration wrapper experiment `experiments/h200/dsv4_h200_dspark/` still exists and produces the `experiments/h200/<name>/results/<RUN_ID>/` layout, but its default `LEGACY_GRID_SCRIPT` / `SERVER_START_SCRIPT` point at the removed scripts and must be overridden to run.
|
|
|
|
## 1. Pull and verify
|
|
|
|
```bash
|
|
git clone <repo-url> sskj # clone path: adjust to your machine
|
|
cd sskj
|
|
|
|
# Platform should auto-detect as nvidia_h200
|
|
source scripts/common/platform.sh
|
|
```
|
|
|
|
If auto-detection fails, set it explicitly:
|
|
|
|
```bash
|
|
PLATFORM=nvidia_h200 source scripts/common/platform.sh
|
|
```
|
|
|
|
## 2. Legacy DSpark benchmark grid (removed)
|
|
|
|
The legacy suite `scripts/benchmark_dspark_0707/` has been removed from this repo, along with its `bench_results/dspark_grid_<RUN_ID>/` output location (historical results were archived elsewhere). Use the wrapper experiment below, or the current tp_dp matrix experiments (`experiments/h200/dsv4_h200_{vllm,sglang}_tp_dp_matrix/`), instead.
|
|
|
|
## 3. Run the migration wrapper experiment
|
|
|
|
This produces results in the experiment-centric layout:
|
|
|
|
```bash
|
|
bash experiments/h200/dsv4_h200_dspark/run_bench.sh
|
|
```
|
|
|
|
Results land in `experiments/h200/dsv4_h200_dspark/results/<RUN_ID>/`.
|
|
|
|
Caveat: the wrapper's default `LEGACY_GRID_SCRIPT` and `SERVER_START_SCRIPT` refer to the removed `scripts/benchmark_dspark_0707/` suite and `scripts/start_dsv4_dspark_8card.sh`; set both explicitly (or restore equivalent scripts) before running.
|
|
|
|
## 4. Create a new H200 experiment
|
|
|
|
To add a new H200 benchmark (for example a different model or engine), create:
|
|
|
|
```
|
|
experiments/h200/<your_name>/
|
|
├── README.md # What this experiment measures
|
|
├── config.env # Model, port, scenarios, venv paths
|
|
├── start_server.sh # (optional) native server launch
|
|
├── run_bench.sh # Orchestrator: server → benchmark → stop
|
|
└── parse_results.py # Generate results.json + report.md
|
|
```
|
|
|
|
Minimum `config.env`:
|
|
|
|
```bash
|
|
EXPERIMENT="${EXPERIMENT:-<your_name>}"
|
|
MODEL_NAME="${MODEL_NAME:-DeepSeek-V4-Flash-DSpark}"
|
|
MODEL_PATH="${MODEL_PATH:-/data/models/DeepSeek-V4-Flash-DSpark}"
|
|
PORT="${PORT:-30004}"
|
|
BACKEND="${BACKEND:-vllm}"
|
|
ENGINE="${ENGINE:-vllm-dspark}"
|
|
|
|
# Native venvs (paths are per-machine; adjust to yours)
|
|
VENV_SERVER="${VENV_SERVER:-/path/to/envs/vllm-dspark}"
|
|
VENV_CLIENT="${VENV_CLIENT:-/path/to/envs/sglang}"
|
|
|
|
SCENARIOS=(
|
|
"32 512 256"
|
|
)
|
|
```
|
|
|
|
Then source the platform loader and shared helpers at the top of `run_bench.sh`:
|
|
|
|
```bash
|
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
|
```
|
|
|
|
For a concrete example, see `experiments/h200/dsv4_h200_dspark/`. For the current mainstream form (TP/DP matrix + adaptive concurrency search, Docker-based), copy `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` instead; usage in `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`.
|
|
|
|
## 5. Cross-platform comparison
|
|
|
|
跨平台对比脚本目前未统一提供。可分别读取各实验 `results/<run_id>/results.json` 中的结构化数据，按 scenario 聚合后生成对比表。
|
|
|
|
## Notes
|
|
|
|
- `platforms/nvidia_h200.env` still carries venv defaults under `/data/user1/yy` (paths from another machine). Override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT` to match your machine.
|
|
- Native server management helpers are not as mature as the Docker helpers in `scripts/common/server_docker.sh`. The wrapper's default `SERVER_START_SCRIPT` (`scripts/start_dsv4_dspark_8card.sh`) no longer exists in this repo and must be overridden.
|