a4e38b9e33
Move all experiments under hardware-specific folders: - experiments/h200/ : H200 GPU experiments (15 dirs) - experiments/h20/ : H20 GPU experiments (2 dirs) - experiments/p800/ : Kunlun P800 experiments (3 dirs) - experiments/pro6000/ : RTX 6000D experiments (2 dirs) This improves discoverability and keeps hardware-specific configs isolated from each other.
1.1 KiB
1.1 KiB
H200 vLLM Baseline Benchmark Report
- Result root:
experiments/dsv4_h200_vllm/results/20260708-062348 - Model:
/data/models/DeepSeek-V4-Flash - Backend: vLLM (TP=4, FP8 KV cache, no speculative decoding)
- Benchmark client:
sglang.bench_serving --backend vllm
Results
| Scenario | Concurrency | Input | Output | Duration(s) | Success | Req/s | In tok/s | Out tok/s | Total tok/s | Mean TTFT(ms) | P95 TTFT(ms) | P99 TTFT(ms) | Mean TPOT(ms) | P95 TPOT(ms) | P99 TPOT(ms) | Mean E2E(ms) | P95 E2E(ms) | P99 E2E(ms) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| c128_i512_o256 | 128 | 512 | 256 | 5.98 | 128 | 21.40 | 5892.52 | 2825.23 | 8717.75 | 634.93 | 873.62 | 879.97 | 28.09 | 40.67 | 91.17 | 3786.22 | 5857.90 | 5919.94 |
| c32_i4000_o512 | 32 | 4000 | 512 | 63.97 | 128 | 2.00 | 4229.98 | 505.98 | 4735.95 | 917.05 | 2853.31 | 6638.21 | 62.67 | 165.57 | 219.72 | 15397.36 | 42952.86 | 46968.37 |
| c32_i512_o256 | 32 | 512 | 256 | 38.26 | 128 | 3.35 | 921.37 | 441.76 | 1363.13 | 1228.47 | 11441.48 | 11446.53 | 79.50 | 246.70 | 674.49 | 9197.96 | 27486.69 | 28020.43 |