# DSpark Benchmark Scripts (2026-07-07)

## Location

`/data/user1/yy/scripts/benchmark_dspark_0707`

## Scripts

| Script | Purpose | Output Location |
|---|---|---|
| `bench_dspark_p1.sh` | Quick benchmark: chat_standard, generation_standard, summarization | `bench_results/dspark_grid_${RUN_ID}/p1_quick/...` |
| `bench_dspark_p2.sh` | Core benchmark: chat_short, rag_medium, long_context_probe | `bench_results/dspark_grid_${RUN_ID}/p2_core/...` |
| `bench_dspark_p3.sh` | Extension benchmark: stress_standard, decode_heavy | `bench_results/dspark_grid_${RUN_ID}/p3_extension/...` |
| `bench_dspark_focused.sh` | Focused sweep used for spec-tokens comparison | `bench_results/dspark_grid_${RUN_ID}/focused/...` or `bench_results/dspark_st_comparison_${RUN_ID}/focused/...` |
| `run_dspark_benchmark_grid.sh` | Orchestrator: starts vllm-dspark server, runs P1/P2/P3, stops server | `bench_results/dspark_grid_${RUN_ID}/` |
| `run_dspark_st_comparison.sh` | Orchestrator: compares `--spec-tokens 3` vs `5` | `bench_results/dspark_st_comparison_${RUN_ID}/` |
| `parse_results.py` | Parses grid logs and generates `report.md` | Writes to `bench_results/<result_root>/report.md` |
| `parse_st_comparison.py` | Parses spec-tokens comparison logs | Writes `comparison_report.md` under result root |
| `parse_eagle_vs_dspark.py` | Merges DSpark st comparison with SGLang EAGLE results | Writes `dspark_vs_eagle_report.md` under EAGLE result root |

## Common Environment

- Python env: `/data/user1/yy/envs/vllm-dspark` for server
- Benchmark client: `/data/user1/yy/envs/sglang/bin/python -m sglang.bench_serving --backend vllm`
- Model: `/data/models/DeepSeek-V4-Flash-DSpark`
- Default port: `30004`
- Default warmup: `100` requests

## Usage

### Full grid benchmark

```bash
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
```

Results land in `bench_results/dspark_grid_YYYYMMDD-HHMMSS/`.

### Spec-tokens comparison

```bash
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
```

Results land in `bench_results/dspark_st_comparison_YYYYMMDD-HHMMSS/`.

### Parse existing results

```bash
# Grid
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py /data/user1/yy/bench_results/dspark_grid_<run_id>

# Spec-tokens comparison
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_st_comparison.py /data/user1/yy/bench_results/dspark_st_comparison_<run_id>
```

## Notes

- These scripts were originally located in `/data/user1/yy/benchmark_dspark_0707/` and moved here on 2026-07-08.
- `run_dspark_benchmark_grid.sh` and `run_dspark_st_comparison.sh` assume server start scripts are in `scripts/` (one directory up).