61 lines
2.6 KiB
Markdown
61 lines
2.6 KiB
Markdown
# DSpark Benchmark Scripts (2026-07-07)
|
|
|
|
## Location
|
|
|
|
`/data/user1/yy/scripts/benchmark_dspark_0707`
|
|
|
|
## Scripts
|
|
|
|
| Script | Purpose | Output Location |
|
|
|---|---|---|
|
|
| `bench_dspark_p1.sh` | Quick benchmark: chat_standard, generation_standard, summarization | `bench_results/dspark_grid_${RUN_ID}/p1_quick/...` |
|
|
| `bench_dspark_p2.sh` | Core benchmark: chat_short, rag_medium, long_context_probe | `bench_results/dspark_grid_${RUN_ID}/p2_core/...` |
|
|
| `bench_dspark_p3.sh` | Extension benchmark: stress_standard, decode_heavy | `bench_results/dspark_grid_${RUN_ID}/p3_extension/...` |
|
|
| `bench_dspark_focused.sh` | Focused sweep used for spec-tokens comparison | `bench_results/dspark_grid_${RUN_ID}/focused/...` or `bench_results/dspark_st_comparison_${RUN_ID}/focused/...` |
|
|
| `run_dspark_benchmark_grid.sh` | Orchestrator: starts vllm-dspark server, runs P1/P2/P3, stops server | `bench_results/dspark_grid_${RUN_ID}/` |
|
|
| `run_dspark_st_comparison.sh` | Orchestrator: compares `--spec-tokens 3` vs `5` | `bench_results/dspark_st_comparison_${RUN_ID}/` |
|
|
| `parse_results.py` | Parses grid logs and generates `report.md` | Writes to `bench_results/<result_root>/report.md` |
|
|
| `parse_st_comparison.py` | Parses spec-tokens comparison logs | Writes `comparison_report.md` under result root |
|
|
| `parse_eagle_vs_dspark.py` | Merges DSpark st comparison with SGLang EAGLE results | Writes `dspark_vs_eagle_report.md` under EAGLE result root |
|
|
|
|
## Common Environment
|
|
|
|
- Python env: `/data/user1/yy/envs/vllm-dspark` for server
|
|
- Benchmark client: `/data/user1/yy/envs/sglang/bin/python -m sglang.bench_serving --backend vllm`
|
|
- Model: `/data/models/DeepSeek-V4-Flash-DSpark`
|
|
- Default port: `30004`
|
|
- Default warmup: `100` requests
|
|
|
|
## Usage
|
|
|
|
### Full grid benchmark
|
|
|
|
```bash
|
|
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
```
|
|
|
|
Results land in `bench_results/dspark_grid_YYYYMMDD-HHMMSS/`.
|
|
|
|
### Spec-tokens comparison
|
|
|
|
```bash
|
|
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
|
|
```
|
|
|
|
Results land in `bench_results/dspark_st_comparison_YYYYMMDD-HHMMSS/`.
|
|
|
|
### Parse existing results
|
|
|
|
```bash
|
|
# Grid
|
|
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py /data/user1/yy/bench_results/dspark_grid_<run_id>
|
|
|
|
# Spec-tokens comparison
|
|
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_st_comparison.py /data/user1/yy/bench_results/dspark_st_comparison_<run_id>
|
|
```
|
|
|
|
## Notes
|
|
|
|
- These scripts were originally located in `/data/user1/yy/benchmark_dspark_0707/` and moved here on 2026-07-08.
|
|
- `run_dspark_benchmark_grid.sh` and `run_dspark_st_comparison.sh` assume server start scripts are in `scripts/` (one directory up).
|