qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vllm tp_dp_matrix: add final results (report, results.json, comparison, scenarios)

Browse Source 
Source code was already committed; this adds only the structured final
artifacts from run 20260709-152419_matrix. Raw jsonl/logs/gpu_logs remain
ignored per .gitignore.
This commit is contained in:
yy-fighting 2026-07-10 02:20:36 +00:00
parent 06cfab20d2
commit 666c583844
11 changed files with 11112 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

230
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/comparison.md Normal file
View File

@ -0,0 +1,230 @@
# vLLM TP×DP matrix comparison (8x NVIDIA H200 143GB)
## Summary
- Model: `/data/models/DeepSeek-V4-Flash`
- Hardware: 8x NVIDIA H200 143GB
- Backend: vLLM
- Benchmark client: `sglang.bench_serving`
- SLO reference: TTFT P95 < 3000.0ms, TPOT mean < 50.0ms
### Configurations
| Config | TP | DP | GPUs/replica | Notes |
|---|---:|---:|---:|---|
| tp2_dp4 | 2 | 4 | 2 | server args recorded per ISL in results.json |
| tp4_dp2 | 4 | 2 | 4 | server args recorded per ISL in results.json |
| tp8_dp1 | 8 | 1 | 8 | server args recorded per ISL in results.json |
## Side-by-side results
| Scenario | ISL | DSL | Config | Conc | Req/s | OutTok/s | TTFT P95(ms) | TTFT P99(ms) | TPOT Mean(ms) | TPOT P95(ms) | TPOT P99(ms) | E2E P99(ms) | Peak GPU mem | SLO |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| c1_i1024_o128 | 1024 | 128 | tp2_dp4 | 1 | 1.05 | 79.65 | 319.08 | 355.57 | 10.56 | 10.99 | 11.00 | 1406.84 | 138264/143771 (96.2%) | ✅ |
| c1_i1024_o128 | 1024 | 128 | tp4_dp2 | 1 | 1.34 | 101.85 | 302.85 | 345.83 | 8.17 | 8.32 | 8.35 | 1055.00 | 138396/143771 (96.3%) | ✅ |
| c1_i1024_o128 | 1024 | 128 | tp8_dp1 | 1 | 1.61 | 121.80 | 274.43 | 314.97 | 6.74 | 6.78 | 6.79 | 871.45 | 131900/143771 (91.7%) | ✅ |
| c1_i1024_o256 | 1024 | 256 | tp2_dp4 | 1 | 0.84 | 84.85 | 107.93 | 108.38 | 11.01 | 11.57 | 11.61 | 2082.56 | 139322/143771 (96.9%) | ✅ |
| c1_i1024_o256 | 1024 | 256 | tp4_dp2 | 1 | 1.12 | 113.80 | 88.52 | 90.59 | 8.12 | 8.21 | 8.23 | 1572.36 | 139454/143771 (97.0%) | ✅ |
| c1_i1024_o256 | 1024 | 256 | tp8_dp1 | 1 | 1.40 | 141.77 | 44.63 | 45.58 | 6.71 | 6.72 | 6.72 | 1281.54 | 132930/143771 (92.5%) | ✅ |
| c1_i1024_o512 | 1024 | 512 | tp2_dp4 | 1 | 0.79 | 79.92 | 113.71 | 115.02 | 11.50 | 11.85 | 11.86 | 2303.60 | 139322/143771 (96.9%) | ✅ |
| c1_i1024_o512 | 1024 | 512 | tp4_dp2 | 1 | 1.09 | 110.67 | 187.00 | 208.64 | 8.12 | 8.17 | 8.18 | 1572.17 | 139454/143771 (97.0%) | ✅ |
| c1_i1024_o512 | 1024 | 512 | tp8_dp1 | 1 | 1.36 | 137.55 | 65.04 | 65.69 | 6.74 | 6.78 | 6.79 | 1305.74 | 132930/143771 (92.5%) | ✅ |
| c1_i1024_o1024 | 1024 | 1024 | tp2_dp4 | 1 | 0.27 | 83.58 | 100.14 | 100.52 | 11.71 | 11.90 | 11.93 | 8200.76 | 139322/143771 (96.9%) | ✅ |
| c1_i1024_o1024 | 1024 | 1024 | tp4_dp2 | 1 | 0.39 | 120.49 | 90.47 | 90.97 | 8.11 | 8.19 | 8.21 | 5705.34 | 139456/143771 (97.0%) | ✅ |
| c1_i1024_o1024 | 1024 | 1024 | tp8_dp1 | 1 | 0.48 | 146.08 | 60.67 | 60.68 | 6.71 | 6.73 | 6.73 | 4727.62 | 132932/143771 (92.5%) | ✅ |
| c1_i1024_o2048 | 1024 | 2048 | tp2_dp4 | 1 | 0.08 | 84.74 | 105.11 | 105.45 | 11.70 | 11.81 | 11.81 | 20192.89 | 139322/143771 (96.9%) | ✅ |
| c1_i1024_o2048 | 1024 | 2048 | tp4_dp2 | 1 | 0.11 | 122.91 | 84.75 | 84.77 | 8.08 | 8.09 | 8.09 | 13978.61 | 139456/143771 (97.0%) | ✅ |
| c1_i1024_o2048 | 1024 | 2048 | tp8_dp1 | 1 | 0.13 | 148.32 | 60.19 | 62.81 | 6.71 | 6.72 | 6.72 | 11586.89 | 132932/143771 (92.5%) | ✅ |
| c1_i1024_o4096 | 1024 | 4096 | tp2_dp4 | 1 | 0.04 | 86.04 | 114.24 | 116.37 | 11.59 | 11.84 | 11.86 | 43455.08 | 139322/143771 (96.9%) | ✅ |
| c1_i1024_o4096 | 1024 | 4096 | tp4_dp2 | 1 | 0.05 | 122.97 | 82.10 | 82.14 | 8.09 | 8.12 | 8.13 | 30482.21 | 139456/143771 (97.0%) | ✅ |
| c1_i1024_o4096 | 1024 | 4096 | tp8_dp1 | 1 | 0.06 | 148.14 | 46.60 | 47.48 | 6.73 | 6.76 | 6.76 | 25332.98 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o128 | 4096 | 128 | tp2_dp4 | 1 | 0.92 | 69.56 | 353.45 | 371.20 | 11.63 | 11.95 | 12.00 | 1608.68 | 139322/143771 (96.9%) | ✅ |
| c1_i4096_o128 | 4096 | 128 | tp4_dp2 | 1 | 1.44 | 109.20 | 96.00 | 96.62 | 8.16 | 8.23 | 8.24 | 1054.23 | 139456/143771 (97.0%) | ✅ |
| c1_i4096_o128 | 4096 | 128 | tp8_dp1 | 1 | 1.62 | 122.84 | 148.51 | 155.20 | 6.72 | 6.74 | 6.74 | 918.13 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o256 | 4096 | 256 | tp2_dp4 | 1 | 0.77 | 78.01 | 208.90 | 226.63 | 11.61 | 11.87 | 11.89 | 2296.91 | 139324/143771 (96.9%) | ✅ |
| c1_i4096_o256 | 4096 | 256 | tp4_dp2 | 1 | 1.11 | 112.37 | 91.50 | 91.90 | 8.16 | 8.23 | 8.24 | 1604.71 | 139456/143771 (97.0%) | ✅ |
| c1_i4096_o256 | 4096 | 256 | tp8_dp1 | 1 | 1.37 | 138.57 | 67.18 | 67.54 | 6.73 | 6.77 | 6.78 | 1320.30 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o512 | 4096 | 512 | tp2_dp4 | 1 | 0.77 | 78.30 | 210.04 | 232.13 | 11.63 | 11.95 | 11.98 | 2378.47 | 139324/143771 (96.9%) | ✅ |
| c1_i4096_o512 | 4096 | 512 | tp4_dp2 | 1 | 1.10 | 111.43 | 104.51 | 107.11 | 8.16 | 8.24 | 8.25 | 1621.06 | 139456/143771 (97.0%) | ✅ |
| c1_i4096_o512 | 4096 | 512 | tp8_dp1 | 1 | 1.37 | 138.55 | 64.63 | 65.10 | 6.73 | 6.78 | 6.78 | 1320.67 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o1024 | 4096 | 1024 | tp2_dp4 | 1 | 0.27 | 83.60 | 113.91 | 115.64 | 11.55 | 11.86 | 11.87 | 8360.74 | 139324/143771 (96.9%) | ✅ |
| c1_i4096_o1024 | 4096 | 1024 | tp4_dp2 | 1 | 0.39 | 119.41 | 92.40 | 93.46 | 8.15 | 8.23 | 8.24 | 5771.30 | 139456/143771 (97.0%) | ✅ |
| c1_i4096_o1024 | 4096 | 1024 | tp8_dp1 | 1 | 0.47 | 144.76 | 66.58 | 66.81 | 6.73 | 6.78 | 6.79 | 4791.74 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o2048 | 4096 | 2048 | tp2_dp4 | 1 | 0.08 | 86.30 | 122.31 | 124.03 | 11.54 | 11.75 | 11.78 | 19528.19 | 139324/143771 (96.9%) | ✅ |
| c1_i4096_o2048 | 4096 | 2048 | tp4_dp2 | 1 | 0.11 | 121.84 | 105.61 | 108.34 | 8.13 | 8.16 | 8.16 | 14152.06 | 139824/143771 (97.3%) | ✅ |
| c1_i4096_o2048 | 4096 | 2048 | tp8_dp1 | 1 | 0.13 | 146.91 | 67.22 | 67.42 | 6.75 | 6.79 | 6.80 | 11755.04 | 132932/143771 (92.5%) | ✅ |
| c1_i4096_o4096 | 4096 | 4096 | tp2_dp4 | 1 | 0.04 | 85.60 | 130.43 | 132.68 | 11.66 | 11.87 | 11.89 | 43588.26 | 139324/143771 (96.9%) | ✅ |
| c1_i4096_o4096 | 4096 | 4096 | tp4_dp2 | 1 | 0.05 | 122.26 | 110.67 | 114.17 | 8.14 | 8.17 | 8.17 | 30710.72 | 139824/143771 (97.3%) | ✅ |
| c1_i4096_o4096 | 4096 | 4096 | tp8_dp1 | 1 | 0.06 | 147.18 | 66.70 | 66.85 | 6.77 | 6.80 | 6.80 | 25525.53 | 132932/143771 (92.5%) | ✅ |
| c1_i16384_o128 | 16384 | 128 | tp2_dp4 | 1 | 0.88 | 66.35 | 599.10 | 618.22 | 11.00 | 11.48 | 11.54 | 1948.72 | 139324/143771 (96.9%) | ✅ |
| c1_i16384_o128 | 16384 | 128 | tp4_dp2 | 1 | 1.14 | 86.24 | 512.76 | 518.98 | 8.11 | 8.23 | 8.25 | 1441.62 | 139824/143771 (97.3%) | ✅ |
| c1_i16384_o128 | 16384 | 128 | tp8_dp1 | 1 | 1.34 | 101.29 | 433.34 | 440.01 | 6.72 | 6.80 | 6.80 | 1242.36 | 132932/143771 (92.5%) | ✅ |
| c1_i16384_o256 | 16384 | 256 | tp2_dp4 | 1 | 0.71 | 71.69 | 448.96 | 486.54 | 11.52 | 11.66 | 11.67 | 2621.34 | 139324/143771 (96.9%) | ✅ |
| c1_i16384_o256 | 16384 | 256 | tp4_dp2 | 1 | 0.98 | 99.42 | 407.06 | 440.10 | 8.14 | 8.23 | 8.24 | 1937.46 | 139824/143771 (97.3%) | ✅ |
| c1_i16384_o256 | 16384 | 256 | tp8_dp1 | 1 | 1.31 | 132.55 | 122.35 | 122.83 | 6.73 | 6.76 | 6.76 | 1372.37 | 132932/143771 (92.5%) | ✅ |
| c1_i16384_o512 | 16384 | 512 | tp2_dp4 | 1 | 0.77 | 77.70 | 188.54 | 190.96 | 11.55 | 11.73 | 11.74 | 2356.50 | 139684/143771 (97.2%) | ✅ |
| c1_i16384_o512 | 16384 | 512 | tp4_dp2 | 1 | 1.07 | 108.69 | 155.92 | 157.11 | 8.16 | 8.23 | 8.25 | 1661.52 | 140338/143771 (97.6%) | ✅ |
| c1_i16384_o512 | 16384 | 512 | tp8_dp1 | 1 | 1.31 | 132.98 | 118.65 | 119.23 | 6.73 | 6.76 | 6.76 | 1365.21 | 132932/143771 (92.5%) | ✅ |
| c1_i16384_o1024 | 16384 | 1024 | tp2_dp4 | 1 | 0.27 | 83.88 | 322.92 | 349.35 | 11.31 | 11.64 | 11.64 | 7915.88 | 139684/143771 (97.2%) | ✅ |
| c1_i16384_o1024 | 16384 | 1024 | tp4_dp2 | 1 | 0.38 | 117.41 | 158.31 | 161.09 | 8.16 | 8.23 | 8.24 | 5837.98 | 140854/143771 (98.0%) | ✅ |
| c1_i16384_o1024 | 16384 | 1024 | tp8_dp1 | 1 | 0.46 | 141.13 | 132.08 | 134.06 | 6.75 | 6.80 | 6.81 | 4873.19 | 132932/143771 (92.5%) | ✅ |
| c1_i16384_o2048 | 16384 | 2048 | tp2_dp4 | 1 | 0.08 | 85.62 | 249.46 | 261.08 | 11.54 | 11.64 | 11.65 | 20210.50 | 139684/143771 (97.2%) | ✅ |
| c1_i16384_o2048 | 16384 | 2048 | tp4_dp2 | 1 | 0.11 | 121.16 | 155.57 | 156.44 | 8.14 | 8.18 | 8.19 | 14233.84 | 140854/143771 (98.0%) | ✅ |
| c1_i16384_o2048 | 16384 | 2048 | tp8_dp1 | 1 | 0.13 | 145.70 | 132.65 | 135.00 | 6.77 | 6.82 | 6.82 | 11867.20 | 132932/143771 (92.5%) | ✅ |
| c1_i65536_o128 | 65536 | 128 | tp2_dp4 | 1 | 0.44 | 33.25 | 2811.65 | 2892.88 | 11.47 | 12.08 | 12.21 | 4233.91 | 139706/143771 (97.2%) | ✅ |
| c1_i65536_o128 | 65536 | 128 | tp4_dp2 | 1 | 0.56 | 42.54 | 2211.56 | 2223.60 | 7.89 | 8.16 | 8.19 | 3092.42 | 140854/143771 (98.0%) | ✅ |
| c1_i65536_o128 | 65536 | 128 | tp8_dp1 | 1 | 0.84 | 63.38 | 1783.81 | 1903.90 | 6.67 | 6.75 | 6.75 | 2686.50 | 133260/143771 (92.7%) | ✅ |
| c1_i65536_o256 | 65536 | 256 | tp2_dp4 | 1 | 0.48 | 48.96 | 2181.49 | 2314.91 | 11.55 | 11.95 | 11.96 | 3709.96 | 139708/143771 (97.2%) | ✅ |
| c1_i65536_o256 | 65536 | 256 | tp4_dp2 | 1 | 1.01 | 102.18 | 274.65 | 283.52 | 7.97 | 8.23 | 8.25 | 1647.86 | 140854/143771 (98.0%) | ✅ |
| c1_i65536_o256 | 65536 | 256 | tp8_dp1 | 1 | 1.19 | 120.56 | 254.47 | 256.20 | 6.74 | 6.79 | 6.80 | 1352.80 | 133780/143771 (93.1%) | ✅ |
| c1_i65536_o512 | 65536 | 512 | tp2_dp4 | 1 | 0.62 | 63.19 | 1276.37 | 1453.96 | 11.06 | 11.68 | 11.78 | 2624.42 | 139708/143771 (97.2%) | ✅ |
| c1_i65536_o512 | 65536 | 512 | tp4_dp2 | 1 | 0.98 | 99.45 | 326.62 | 330.25 | 7.91 | 8.17 | 8.18 | 1655.09 | 140854/143771 (98.0%) | ✅ |
| c1_i65536_o512 | 65536 | 512 | tp8_dp1 | 1 | 1.18 | 119.95 | 288.97 | 290.20 | 6.54 | 6.74 | 6.75 | 1356.99 | 133780/143771 (93.1%) | ✅ |
| c1_i65536_o1024 | 65536 | 1024 | tp2_dp4 | 1 | 0.27 | 83.13 | 375.58 | 383.90 | 11.12 | 11.57 | 11.58 | 8213.89 | 139708/143771 (97.2%) | ✅ |
| c1_i65536_o1024 | 65536 | 1024 | tp4_dp2 | 1 | 0.37 | 112.31 | 358.50 | 364.11 | 8.07 | 8.26 | 8.26 | 5842.34 | 140854/143771 (98.0%) | ✅ |
| c1_i65536_o1024 | 65536 | 1024 | tp8_dp1 | 1 | 0.45 | 136.44 | 254.42 | 254.84 | 6.76 | 6.85 | 6.86 | 4841.93 | 133780/143771 (93.1%) | ✅ |
| c1_i131072_o128 | 131072 | 128 | tp2_dp4 | 1 | 0.32 | 24.40 | 3831.71 | 3847.18 | 11.06 | 11.65 | 11.68 | 5071.55 | 140228/143771 (97.5%) | ⚠️ |
| c1_i131072_o128 | 131072 | 128 | tp4_dp2 | 1 | 0.38 | 28.62 | 3498.43 | 3510.98 | 7.74 | 8.18 | 8.22 | 4353.06 | 141374/143771 (98.3%) | ⚠️ |
| c1_i131072_o128 | 131072 | 128 | tp8_dp1 | 1 | 0.55 | 42.03 | 3042.40 | 3072.01 | 6.34 | 6.70 | 6.72 | 3832.09 | 133780/143771 (93.1%) | ⚠️ |
| c1_i131072_o256 | 131072 | 256 | tp2_dp4 | 1 | 0.46 | 46.84 | 3143.30 | 3630.53 | 11.11 | 11.90 | 11.97 | 4950.01 | 140228/143771 (97.5%) | ⚠️ |
| c1_i131072_o256 | 131072 | 256 | tp4_dp2 | 1 | 0.61 | 61.94 | 2612.01 | 3033.01 | 7.81 | 8.23 | 8.24 | 3829.62 | 141374/143771 (98.3%) | ✅ |
| c1_i131072_o256 | 131072 | 256 | tp8_dp1 | 1 | 1.03 | 104.48 | 526.13 | 531.79 | 6.38 | 6.74 | 6.75 | 1365.14 | 134302/143771 (93.4%) | ✅ |
| c1_i131072_o512 | 131072 | 512 | tp2_dp4 | 1 | 0.67 | 68.11 | 574.50 | 577.45 | 10.89 | 11.54 | 11.54 | 2318.95 | 140228/143771 (97.5%) | ✅ |
| c1_i131072_o512 | 131072 | 512 | tp4_dp2 | 1 | 0.89 | 90.55 | 555.04 | 573.87 | 7.81 | 8.24 | 8.27 | 1649.69 | 141374/143771 (98.3%) | ✅ |
| c1_i131072_o512 | 131072 | 512 | tp8_dp1 | 1 | 1.03 | 104.09 | 528.58 | 529.52 | 6.38 | 6.74 | 6.75 | 1365.66 | 134302/143771 (93.4%) | ✅ |
| c1_i262144_o128 | 262144 | 128 | tp2_dp4 | 1 | 0.15 | 11.35 | 13444.27 | 14788.12 | 9.88 | 11.09 | 11.15 | 16032.30 | 140744/143771 (97.9%) | ⚠️ |
| c1_i262144_o128 | 262144 | 128 | tp4_dp2 | 1 | 0.20 | 15.35 | 8221.07 | 8613.48 | 6.70 | 7.64 | 7.66 | 9509.11 | 141374/143771 (98.3%) | ⚠️ |
| c1_i262144_o128 | 262144 | 128 | tp8_dp1 | 1 | 0.23 | 17.26 | 7547.16 | 7933.13 | 5.31 | 6.25 | 6.26 | 8667.71 | 134304/143771 (93.4%) | ⚠️ |
| c1_i262144_o256 | 262144 | 256 | tp2_dp4 | 1 | 0.18 | 18.19 | 8943.43 | 9385.72 | 10.07 | 11.43 | 11.50 | 10671.30 | 141258/143771 (98.3%) | ⚠️ |
| c1_i262144_o256 | 262144 | 256 | tp4_dp2 | 1 | 0.71 | 72.25 | 962.12 | 1036.06 | 6.92 | 7.88 | 7.93 | 2080.08 | 141374/143771 (98.3%) | ✅ |
| c1_i262144_o256 | 262144 | 256 | tp8_dp1 | 1 | 0.91 | 92.31 | 741.06 | 802.19 | 5.45 | 6.50 | 6.55 | 1664.47 | 134304/143771 (93.4%) | ✅ |
| c1_i524288_o128 | 524288 | 128 | tp2_dp4 | 1 | 0.13 | 9.76 | 17139.22 | 19091.80 | 9.70 | 10.39 | 10.40 | 20099.12 | 141258/143771 (98.3%) | ⚠️ |
| c1_i524288_o128 | 524288 | 128 | tp4_dp2 | 1 | 0.21 | 16.00 | 14806.51 | 17551.38 | 6.63 | 7.42 | 7.46 | 18281.61 | 141374/143771 (98.3%) | ⚠️ |
| c1_i524288_o128 | 524288 | 128 | tp8_dp1 | 1 | 0.23 | 17.65 | 13982.40 | 16622.77 | 5.18 | 6.10 | 6.17 | 17210.26 | 134304/143771 (93.4%) | ⚠️ |
| c1_i1048576_o128 | 1048576 | 128 | tp2_dp4 | 1 | 0.06 | 4.85 | 46081.66 | 51394.85 | 8.95 | 10.50 | 10.51 | 51849.45 | 141258/143771 (98.3%) | ⚠️ |
| c1_i1048576_o128 | 1048576 | 128 | tp4_dp2 | 1 | 0.06 | 4.88 | 43228.08 | 48170.56 | 6.14 | 7.51 | 7.56 | 48471.96 | 141374/143771 (98.3%) | ⚠️ |
| c1_i1048576_o128 | 1048576 | 128 | tp8_dp1 | 1 | 0.10 | 7.26 | 38265.96 | 45702.81 | 4.67 | 6.04 | 6.08 | 45912.47 | 134310/143771 (93.4%) | ⚠️ |
| c2_i262144_o128 | 262144 | 128 | tp2_dp4 | 2 | 0.27 | 20.65 | 9292.65 | 9433.23 | 24.30 | 74.40 | 75.57 | 10323.72 | 140744/143771 (97.9%) | ⚠️ |
| c2_i262144_o128 | 262144 | 128 | tp4_dp2 | 2 | 0.33 | 24.58 | 7994.94 | 8377.33 | 25.23 | 70.28 | 72.19 | 9106.67 | 141374/143771 (98.3%) | ⚠️ |
| c2_i262144_o128 | 262144 | 128 | tp8_dp1 | 2 | 0.58 | 43.70 | 6405.29 | 6864.92 | 12.04 | 40.84 | 63.02 | 7624.83 | 134304/143771 (93.4%) | ⚠️ |
| c2_i262144_o256 | 262144 | 256 | tp2_dp4 | 2 | 0.30 | 42.07 | 9153.02 | 10642.68 | 24.92 | 66.10 | 74.81 | 12104.02 | 141258/143771 (98.3%) | ⚠️ |
| c2_i262144_o256 | 262144 | 256 | tp4_dp2 | 2 | 0.48 | 66.52 | 6399.54 | 6424.61 | 10.75 | 22.96 | 32.38 | 8063.51 | 141374/143771 (98.3%) | ⚠️ |
| c2_i262144_o256 | 262144 | 256 | tp8_dp1 | 2 | 1.23 | 171.18 | 872.87 | 960.16 | 6.69 | 7.16 | 7.25 | 2191.39 | 134304/143771 (93.4%) | ✅ |
| c2_i524288_o128 | 524288 | 128 | tp2_dp4 | 2 | 0.69 | 52.24 | 4695.87 | 6463.46 | 15.77 | 44.14 | 65.40 | 8060.58 | 141258/143771 (98.3%) | ⚠️ |
| c2_i524288_o128 | 524288 | 128 | tp4_dp2 | 2 | 0.64 | 48.43 | 6241.67 | 9313.98 | 44.54 | 213.86 | 348.24 | 10806.98 | 141374/143771 (98.3%) | ⚠️ |
| c2_i524288_o128 | 524288 | 128 | tp8_dp1 | 2 | 1.76 | 133.35 | 1167.12 | 1232.75 | 5.64 | 6.90 | 6.94 | 1708.86 | 134306/143771 (93.4%) | ✅ |
| c2_i1048576_o128 | 1048576 | 128 | tp2_dp4 | 2 | 0.06 | 4.16 | 61414.70 | 65340.99 | 117.07 | 500.43 | 531.18 | 66294.91 | 141780/143771 (98.6%) | ❌ |
| c2_i1048576_o128 | 1048576 | 128 | tp4_dp2 | 2 | 0.06 | 4.56 | 51577.89 | 52987.83 | 277.94 | 766.41 | 858.05 | 56546.05 | 141374/143771 (98.3%) | ❌ |
| c2_i1048576_o128 | 1048576 | 128 | tp8_dp1 | 2 | 0.08 | 6.16 | 59936.57 | 60383.98 | 15.73 | 56.69 | 68.50 | 66073.78 | 134312/143771 (93.4%) | ⚠️ |
| c4_i131072_o128 | 131072 | 128 | tp2_dp4 | 4 | 0.74 | 41.97 | 5381.79 | 6485.26 | 63.46 | 234.23 | 254.36 | 9800.84 | 140228/143771 (97.5%) | ❌ |
| c4_i131072_o128 | 131072 | 128 | tp4_dp2 | 4 | 0.61 | 34.68 | 7688.92 | 7746.95 | 55.15 | 151.40 | 157.05 | 13546.32 | 141374/143771 (98.3%) | ❌ |
| c4_i131072_o128 | 131072 | 128 | tp8_dp1 | 4 | 0.82 | 46.76 | 5665.45 | 7563.45 | 46.92 | 130.55 | 135.39 | 9263.16 | 134302/143771 (93.4%) | ⚠️ |
| c4_i131072_o256 | 131072 | 256 | tp2_dp4 | 4 | 0.51 | 64.78 | 4431.12 | 6485.73 | 38.02 | 82.59 | 93.31 | 21326.10 | 140228/143771 (97.5%) | ⚠️ |
| c4_i131072_o256 | 131072 | 256 | tp4_dp2 | 4 | 0.79 | 101.20 | 6019.33 | 6092.09 | 20.62 | 43.90 | 46.90 | 10875.06 | 141374/143771 (98.3%) | ⚠️ |
| c4_i131072_o256 | 131072 | 256 | tp8_dp1 | 4 | 2.40 | 306.01 | 826.11 | 873.84 | 7.11 | 8.42 | 8.69 | 2602.99 | 134302/143771 (93.4%) | ✅ |
| c4_i131072_o512 | 131072 | 512 | tp2_dp4 | 4 | 0.46 | 136.39 | 5162.58 | 5751.13 | 24.26 | 43.94 | 51.73 | 14391.10 | 140228/143771 (97.5%) | ⚠️ |
| c4_i131072_o512 | 131072 | 512 | tp4_dp2 | 4 | 1.03 | 302.02 | 724.28 | 1360.67 | 9.94 | 13.97 | 16.16 | 5861.60 | 141374/143771 (98.3%) | ✅ |
| c4_i131072_o512 | 131072 | 512 | tp8_dp1 | 4 | 1.30 | 382.40 | 581.21 | 686.73 | 7.67 | 8.27 | 8.30 | 4365.16 | 134302/143771 (93.4%) | ✅ |
| c8_i65536_o128 | 65536 | 128 | tp2_dp4 | 8 | 0.86 | 55.23 | 5855.87 | 7061.54 | 115.39 | 327.74 | 399.92 | 18685.86 | 139708/143771 (97.2%) | ❌ |
| c8_i65536_o128 | 65536 | 128 | tp4_dp2 | 8 | 0.94 | 60.49 | 5620.08 | 5766.46 | 118.91 | 301.63 | 309.58 | 16549.56 | 140854/143771 (98.0%) | ❌ |
| c8_i65536_o128 | 65536 | 128 | tp8_dp1 | 8 | 1.01 | 64.77 | 7445.34 | 8645.53 | 88.42 | 248.75 | 263.80 | 16484.17 | 133780/143771 (93.1%) | ❌ |
| c8_i65536_o256 | 65536 | 256 | tp2_dp4 | 8 | 0.90 | 117.73 | 4029.43 | 4806.74 | 54.45 | 101.30 | 122.71 | 18052.15 | 139708/143771 (97.2%) | ❌ |
| c8_i65536_o256 | 65536 | 256 | tp4_dp2 | 8 | 1.31 | 171.79 | 3353.05 | 4860.71 | 44.63 | 113.59 | 141.58 | 15303.64 | 140854/143771 (98.0%) | ⚠️ |
| c8_i65536_o256 | 65536 | 256 | tp8_dp1 | 8 | 4.63 | 608.09 | 749.11 | 864.77 | 9.14 | 10.21 | 10.30 | 2998.37 | 133780/143771 (93.1%) | ✅ |
| c8_i65536_o512 | 65536 | 512 | tp2_dp4 | 8 | 0.98 | 272.84 | 3168.19 | 5671.91 | 24.56 | 51.52 | 85.56 | 18250.24 | 139708/143771 (97.2%) | ⚠️ |
| c8_i65536_o512 | 65536 | 512 | tp4_dp2 | 8 | 1.47 | 410.24 | 1809.22 | 2638.39 | 17.90 | 23.23 | 79.99 | 9676.38 | 140854/143771 (98.0%) | ✅ |
| c8_i65536_o512 | 65536 | 512 | tp8_dp1 | 8 | 2.53 | 705.69 | 661.39 | 679.12 | 9.10 | 9.63 | 10.39 | 5200.71 | 133780/143771 (93.1%) | ✅ |
| c8_i65536_o1024 | 65536 | 1024 | tp2_dp4 | 8 | 0.64 | 351.24 | 2850.09 | 2967.29 | 19.43 | 29.44 | 36.68 | 21507.37 | 139708/143771 (97.2%) | ✅ |
| c8_i65536_o1024 | 65536 | 1024 | tp4_dp2 | 8 | 0.97 | 528.38 | 2177.55 | 2783.15 | 13.13 | 20.42 | 32.65 | 13878.42 | 140854/143771 (98.0%) | ✅ |
| c8_i65536_o1024 | 65536 | 1024 | tp8_dp1 | 8 | 1.34 | 734.78 | 671.08 | 707.77 | 8.91 | 9.20 | 10.49 | 9476.99 | 133780/143771 (93.1%) | ✅ |
| c32_i16384_o128 | 16384 | 128 | tp2_dp4 | 32 | 4.31 | 284.72 | 3298.27 | 4051.80 | 107.21 | 186.59 | 376.91 | 15245.93 | 139324/143771 (96.9%) | ❌ |
| c32_i16384_o128 | 16384 | 128 | tp4_dp2 | 32 | 5.16 | 340.76 | 3479.51 | 3760.82 | 85.23 | 160.17 | 235.33 | 12999.09 | 139824/143771 (97.3%) | ❌ |
| c32_i16384_o128 | 16384 | 128 | tp8_dp1 | 32 | 5.39 | 356.17 | 3789.36 | 4532.55 | 78.86 | 150.73 | 226.61 | 11830.35 | 132932/143771 (92.5%) | ❌ |
| c32_i16384_o256 | 16384 | 256 | tp2_dp4 | 32 | 4.49 | 623.03 | 1913.91 | 2384.97 | 47.34 | 71.67 | 147.97 | 13936.33 | 139684/143771 (97.2%) | ✅ |
| c32_i16384_o256 | 16384 | 256 | tp4_dp2 | 32 | 5.41 | 751.48 | 1701.48 | 2572.97 | 38.88 | 55.35 | 126.86 | 11645.26 | 140338/143771 (97.6%) | ✅ |
| c32_i16384_o256 | 16384 | 256 | tp8_dp1 | 32 | 12.84 | 1782.54 | 636.48 | 644.96 | 14.84 | 16.94 | 20.78 | 4262.99 | 132932/143771 (92.5%) | ✅ |
| c32_i16384_o512 | 16384 | 512 | tp2_dp4 | 32 | 3.77 | 952.48 | 1520.07 | 2259.12 | 30.21 | 45.07 | 57.04 | 18769.12 | 139684/143771 (97.2%) | ✅ |
| c32_i16384_o512 | 16384 | 512 | tp4_dp2 | 32 | 5.44 | 1373.00 | 965.45 | 1010.01 | 20.31 | 25.61 | 28.12 | 12002.66 | 140854/143771 (98.0%) | ✅ |
| c32_i16384_o512 | 16384 | 512 | tp8_dp1 | 32 | 7.83 | 1977.19 | 640.86 | 739.06 | 14.04 | 15.70 | 18.53 | 7504.82 | 132932/143771 (92.5%) | ✅ |
| c32_i16384_o1024 | 16384 | 1024 | tp2_dp4 | 32 | 2.52 | 1337.54 | 1871.97 | 2404.37 | 22.95 | 27.86 | 72.28 | 22977.81 | 139684/143771 (97.2%) | ✅ |
| c32_i16384_o1024 | 16384 | 1024 | tp4_dp2 | 32 | 3.21 | 1704.78 | 1131.20 | 1660.57 | 17.14 | 21.79 | 37.19 | 18824.49 | 140854/143771 (98.0%) | ✅ |
| c32_i16384_o1024 | 16384 | 1024 | tp8_dp1 | 32 | 4.00 | 2122.35 | 789.94 | 950.40 | 13.38 | 14.23 | 22.02 | 13797.93 | 132932/143771 (92.5%) | ✅ |
| c32_i16384_o2048 | 16384 | 2048 | tp2_dp4 | 32 | 1.56 | 1785.47 | 1089.13 | 1111.26 | 15.83 | 17.47 | 19.70 | 33430.94 | 139684/143771 (97.2%) | ✅ |
| c32_i16384_o2048 | 16384 | 2048 | tp4_dp2 | 32 | 1.73 | 1982.72 | 1143.92 | 2172.49 | 14.60 | 16.68 | 25.49 | 30322.94 | 140854/143771 (98.0%) | ✅ |
| c32_i16384_o2048 | 16384 | 2048 | tp8_dp1 | 32 | 1.94 | 2226.25 | 628.02 | 806.24 | 12.88 | 13.39 | 14.85 | 26578.49 | 132932/143771 (92.5%) | ✅ |
| c64_i4096_o128 | 4096 | 128 | tp2_dp4 | 64 | 11.02 | 721.10 | 1790.01 | 2034.19 | 83.97 | 111.62 | 188.61 | 12237.14 | 139324/143771 (96.9%) | ⚠️ |
| c64_i4096_o128 | 4096 | 128 | tp4_dp2 | 64 | 10.43 | 682.69 | 1904.20 | 2262.19 | 88.50 | 123.52 | 173.68 | 12463.74 | 139456/143771 (97.0%) | ⚠️ |
| c64_i4096_o128 | 4096 | 128 | tp8_dp1 | 64 | 11.81 | 772.92 | 2978.27 | 3725.28 | 76.32 | 108.07 | 233.29 | 11989.51 | 132932/143771 (92.5%) | ⚠️ |
| c64_i4096_o256 | 4096 | 256 | tp2_dp4 | 64 | 8.50 | 1101.01 | 1290.52 | 1667.29 | 55.00 | 72.09 | 89.93 | 15631.13 | 139324/143771 (96.9%) | ⚠️ |
| c64_i4096_o256 | 4096 | 256 | tp4_dp2 | 64 | 10.23 | 1323.77 | 1198.05 | 1234.72 | 45.94 | 59.77 | 82.58 | 13085.09 | 139456/143771 (97.0%) | ✅ |
| c64_i4096_o256 | 4096 | 256 | tp8_dp1 | 64 | 19.45 | 2518.43 | 804.28 | 815.33 | 22.25 | 25.88 | 28.53 | 6644.63 | 132932/143771 (92.5%) | ✅ |
| c64_i4096_o512 | 4096 | 512 | tp2_dp4 | 64 | 6.47 | 1712.46 | 1099.58 | 1280.63 | 32.92 | 40.95 | 51.15 | 18838.52 | 139324/143771 (96.9%) | ✅ |
| c64_i4096_o512 | 4096 | 512 | tp4_dp2 | 64 | 8.10 | 2143.08 | 946.13 | 974.08 | 26.25 | 32.55 | 35.60 | 15173.06 | 139456/143771 (97.0%) | ✅ |
| c64_i4096_o512 | 4096 | 512 | tp8_dp1 | 64 | 11.71 | 3100.03 | 584.91 | 601.95 | 17.83 | 18.78 | 19.95 | 9668.74 | 132932/143771 (92.5%) | ✅ |
| c64_i4096_o1024 | 4096 | 1024 | tp2_dp4 | 64 | 4.39 | 2351.38 | 1083.22 | 1200.04 | 24.41 | 28.13 | 32.24 | 26088.66 | 139324/143771 (96.9%) | ✅ |
| c64_i4096_o1024 | 4096 | 1024 | tp4_dp2 | 64 | 5.45 | 2915.21 | 1023.51 | 1212.54 | 19.18 | 21.78 | 24.49 | 21920.34 | 139824/143771 (97.3%) | ✅ |
| c64_i4096_o1024 | 4096 | 1024 | tp8_dp1 | 64 | 6.14 | 3283.93 | 789.79 | 800.08 | 17.06 | 17.95 | 19.33 | 17906.82 | 132932/143771 (92.5%) | ✅ |
| c64_i4096_o2048 | 4096 | 2048 | tp2_dp4 | 64 | 2.73 | 2846.07 | 1149.71 | 1194.86 | 19.80 | 22.43 | 24.96 | 41227.42 | 139324/143771 (96.9%) | ✅ |
| c64_i4096_o2048 | 4096 | 2048 | tp4_dp2 | 64 | 3.04 | 3165.95 | 856.07 | 869.36 | 17.64 | 18.77 | 20.69 | 37188.62 | 139824/143771 (97.3%) | ✅ |
| c64_i4096_o2048 | 4096 | 2048 | tp8_dp1 | 64 | 3.28 | 3408.88 | 570.75 | 582.03 | 16.56 | 17.06 | 18.07 | 34336.54 | 132932/143771 (92.5%) | ✅ |
| c64_i4096_o4096 | 4096 | 4096 | tp2_dp4 | 64 | 1.49 | 3050.95 | 900.93 | 999.74 | 17.93 | 19.28 | 20.04 | 75036.72 | 139324/143771 (96.9%) | ✅ |
| c64_i4096_o4096 | 4096 | 4096 | tp4_dp2 | 64 | 1.62 | 3323.06 | 727.11 | 753.66 | 16.53 | 17.22 | 17.64 | 68039.71 | 139824/143771 (97.3%) | ✅ |
| c64_i4096_o4096 | 4096 | 4096 | tp8_dp1 | 64 | 1.67 | 3408.94 | 576.41 | 599.00 | 16.44 | 16.79 | 16.98 | 68183.67 | 132932/143771 (92.5%) | ✅ |
| c128_i1024_o128 | 1024 | 128 | tp2_dp4 | 128 | 14.09 | 919.53 | 1787.94 | 1815.68 | 131.99 | 159.76 | 191.60 | 19385.13 | 139322/143771 (96.9%) | ⚠️ |
| c128_i1024_o128 | 1024 | 128 | tp4_dp2 | 128 | 15.24 | 994.43 | 1659.18 | 1917.69 | 122.83 | 156.63 | 189.20 | 18386.47 | 139454/143771 (97.0%) | ⚠️ |
| c128_i1024_o128 | 1024 | 128 | tp8_dp1 | 128 | 25.21 | 1644.87 | 1647.93 | 2127.36 | 71.51 | 108.98 | 146.42 | 11987.25 | 132930/143771 (92.5%) | ⚠️ |
| c128_i1024_o256 | 1024 | 256 | tp2_dp4 | 128 | 12.51 | 1654.84 | 1302.99 | 1324.67 | 72.57 | 92.16 | 105.85 | 20487.34 | 139322/143771 (96.9%) | ⚠️ |
| c128_i1024_o256 | 1024 | 256 | tp4_dp2 | 128 | 15.10 | 1996.42 | 938.43 | 1171.74 | 60.05 | 74.97 | 96.47 | 16952.04 | 139454/143771 (97.0%) | ⚠️ |
| c128_i1024_o256 | 1024 | 256 | tp8_dp1 | 128 | 23.31 | 3082.56 | 725.46 | 741.75 | 37.61 | 47.17 | 56.15 | 10662.58 | 132930/143771 (92.5%) | ✅ |
| c128_i1024_o512 | 1024 | 512 | tp2_dp4 | 128 | 9.80 | 2483.34 | 1236.35 | 1486.37 | 47.72 | 59.20 | 70.21 | 26017.94 | 139322/143771 (96.9%) | ✅ |
| c128_i1024_o512 | 1024 | 512 | tp4_dp2 | 128 | 11.78 | 2986.46 | 996.03 | 1011.18 | 39.13 | 48.34 | 56.02 | 22283.80 | 139456/143771 (97.0%) | ✅ |
| c128_i1024_o512 | 1024 | 512 | tp8_dp1 | 128 | 17.09 | 4330.93 | 760.28 | 777.02 | 26.29 | 29.37 | 34.23 | 14073.81 | 132932/143771 (92.5%) | ✅ |
| c128_i1024_o1024 | 1024 | 1024 | tp2_dp4 | 128 | 7.08 | 3578.74 | 1147.11 | 1259.64 | 32.79 | 39.48 | 50.72 | 34693.91 | 139322/143771 (96.9%) | ✅ |
| c128_i1024_o1024 | 1024 | 1024 | tp4_dp2 | 128 | 8.62 | 4359.31 | 719.08 | 837.92 | 26.31 | 30.84 | 33.28 | 27367.26 | 139456/143771 (97.0%) | ✅ |
| c128_i1024_o1024 | 1024 | 1024 | tp8_dp1 | 128 | 9.35 | 4727.69 | 724.06 | 738.02 | 24.27 | 25.99 | 28.67 | 25355.91 | 132932/143771 (92.5%) | ✅ |
| c128_i1024_o2048 | 1024 | 2048 | tp2_dp4 | 128 | 4.40 | 4467.90 | 990.93 | 1038.67 | 25.24 | 28.25 | 31.35 | 52638.14 | 139322/143771 (96.9%) | ✅ |
| c128_i1024_o2048 | 1024 | 2048 | tp4_dp2 | 128 | 4.83 | 4909.69 | 979.55 | 1002.42 | 22.76 | 24.25 | 25.28 | 47303.18 | 139456/143771 (97.0%) | ✅ |
| c128_i1024_o2048 | 1024 | 2048 | tp8_dp1 | 128 | 4.80 | 4871.12 | 746.05 | 761.60 | 23.37 | 24.26 | 24.89 | 48201.59 | 132932/143771 (92.5%) | ✅ |
| c128_i1024_o4096 | 1024 | 4096 | tp2_dp4 | 128 | 2.42 | 4961.16 | 901.88 | 946.01 | 22.33 | 23.78 | 25.83 | 91949.11 | 139322/143771 (96.9%) | ✅ |
| c128_i1024_o4096 | 1024 | 4096 | tp4_dp2 | 128 | 2.49 | 5110.60 | 737.03 | 754.71 | 21.93 | 23.01 | 23.26 | 90427.39 | 139456/143771 (97.0%) | ✅ |
| c128_i1024_o4096 | 1024 | 4096 | tp8_dp1 | 128 | 2.33 | 4775.68 | 736.09 | 756.63 | 23.99 | 25.92 | 26.62 | 97316.79 | 132932/143771 (92.5%) | ✅ |
## Best throughput per (ISL, DSL)
| ISL | DSL | Best Config | Concurrency | OutTok/s | TTFT P95(ms) | TPOT Mean(ms) | SLO |
|---:|---:|---|---:|---:|---:|---:|---:|
| 1024 | 128 | tp8_dp1 | 128 | 1644.87 | 1647.93 | 71.51 | ⚠️ |
| 1024 | 256 | tp8_dp1 | 128 | 3082.56 | 725.46 | 37.61 | ✅ |
| 1024 | 512 | tp8_dp1 | 128 | 4330.93 | 760.28 | 26.29 | ✅ |
| 1024 | 1024 | tp8_dp1 | 128 | 4727.69 | 724.06 | 24.27 | ✅ |
| 1024 | 2048 | tp4_dp2 | 128 | 4909.69 | 979.55 | 22.76 | ✅ |
| 1024 | 4096 | tp4_dp2 | 128 | 5110.60 | 737.03 | 21.93 | ✅ |
| 4096 | 128 | tp8_dp1 | 64 | 772.92 | 2978.27 | 76.32 | ⚠️ |
| 4096 | 256 | tp8_dp1 | 64 | 2518.43 | 804.28 | 22.25 | ✅ |
| 4096 | 512 | tp8_dp1 | 64 | 3100.03 | 584.91 | 17.83 | ✅ |
| 4096 | 1024 | tp8_dp1 | 64 | 3283.93 | 789.79 | 17.06 | ✅ |
| 4096 | 2048 | tp8_dp1 | 64 | 3408.88 | 570.75 | 16.56 | ✅ |
| 4096 | 4096 | tp8_dp1 | 64 | 3408.94 | 576.41 | 16.44 | ✅ |
| 16384 | 128 | tp8_dp1 | 32 | 356.17 | 3789.36 | 78.86 | ❌ |
| 16384 | 256 | tp8_dp1 | 32 | 1782.54 | 636.48 | 14.84 | ✅ |
| 16384 | 512 | tp8_dp1 | 32 | 1977.19 | 640.86 | 14.04 | ✅ |
| 16384 | 1024 | tp8_dp1 | 32 | 2122.35 | 789.94 | 13.38 | ✅ |
| 16384 | 2048 | tp8_dp1 | 32 | 2226.25 | 628.02 | 12.88 | ✅ |
| 65536 | 128 | tp8_dp1 | 8 | 64.77 | 7445.34 | 88.42 | ❌ |
| 65536 | 256 | tp8_dp1 | 8 | 608.09 | 749.11 | 9.14 | ✅ |
| 65536 | 512 | tp8_dp1 | 8 | 705.69 | 661.39 | 9.10 | ✅ |
| 65536 | 1024 | tp8_dp1 | 8 | 734.78 | 671.08 | 8.91 | ✅ |
| 131072 | 128 | tp8_dp1 | 4 | 46.76 | 5665.45 | 46.92 | ⚠️ |
| 131072 | 256 | tp8_dp1 | 4 | 306.01 | 826.11 | 7.11 | ✅ |
| 131072 | 512 | tp8_dp1 | 4 | 382.40 | 581.21 | 7.67 | ✅ |
| 262144 | 128 | tp8_dp1 | 2 | 43.70 | 6405.29 | 12.04 | ⚠️ |
| 262144 | 256 | tp8_dp1 | 2 | 171.18 | 872.87 | 6.69 | ✅ |
| 524288 | 128 | tp8_dp1 | 2 | 133.35 | 1167.12 | 5.64 | ✅ |
| 1048576 | 128 | tp8_dp1 | 1 | 7.26 | 38265.96 | 4.67 | ⚠️ |
## Notes
- SLO check uses TTFT P95 and TPOT mean.
- A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.
- `Peak GPU mem` shows peak used / total MB and utilization percentage.
- Optional (P) combinations that failed are marked as skipped/OOM and do not break the run.

1
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/skipped_after_oom.csv Normal file
View File

@ -0,0 +1 @@
engine,tp,dp,mark,isl,dsl,concurrency,status,reason,detail_log
1 engine tp dp mark isl dsl concurrency status reason detail_log

70
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp2_dp4/report.md Normal file
View File

@ -0,0 +1,70 @@
# 8x NVIDIA H200 143GB VLLM Benchmark Report
- Result root: `/data/user1/yy/experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp2_dp4`
- Model: `/data/models/DeepSeek-V4-Flash`
- Backend: VLLM
- Benchmark client: `sglang.bench_serving --backend vllm`
## Results
| Scenario | Phase | Concurrency | Input | Output | Duration(s) | Success | Req/s | In tok/s | Out tok/s | Total tok/s | Mean TTFT(ms) | P95 TTFT(ms) | P99 TTFT(ms) | Mean TPOT(ms) | P95 TPOT(ms) | P99 TPOT(ms) | Mean E2E(ms) | P95 E2E(ms) | P99 E2E(ms) | Peak GPU mem | SLO |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| c1_i1024_o128 | main | 1 | 1024 | 128 | 4.76 | 5 | 1.05 | 373.68 | 79.65 | 453.33 | 159.97 | 319.08 | 355.57 | 10.56 | 10.99 | 11.00 | 950.14 | 1366.62 | 1406.84 | 138264/143771 MiB (96.2%) | ✅ |
| c128_i1024_o128 | main | 128 | 1024 | 128 | 45.42 | 640 | 14.09 | 7372.62 | 919.53 | 8292.15 | 682.86 | 1787.94 | 1815.68 | 131.99 | 159.76 | 191.60 | 8857.59 | 17800.52 | 19385.13 | 139322/143771 MiB (96.9%) | ⚠️ |
| c1_i1024_o256 | main | 1 | 1024 | 256 | 5.98 | 5 | 0.84 | 297.55 | 84.85 | 382.39 | 96.69 | 107.93 | 108.38 | 11.01 | 11.57 | 11.61 | 1193.49 | 1982.06 | 2082.56 | 139322/143771 MiB (96.9%) | ✅ |
| c128_i1024_o256 | main | 128 | 1024 | 256 | 51.15 | 640 | 12.51 | 6546.63 | 1654.84 | 8201.47 | 455.67 | 1302.99 | 1324.67 | 72.57 | 92.16 | 105.85 | 9769.30 | 18928.98 | 20487.34 | 139322/143771 MiB (96.9%) | ⚠️ |
| c1_i1024_o512 | main | 1 | 1024 | 512 | 6.34 | 5 | 0.79 | 280.27 | 79.92 | 360.18 | 95.33 | 113.71 | 115.02 | 11.50 | 11.85 | 11.86 | 1267.01 | 2173.11 | 2303.60 | 139322/143771 MiB (96.9%) | ✅ |
| c128_i1024_o512 | main | 128 | 1024 | 512 | 65.32 | 640 | 9.80 | 5126.38 | 2483.34 | 7609.72 | 392.20 | 1236.35 | 1486.37 | 47.72 | 59.20 | 70.21 | 12165.11 | 24348.08 | 26017.94 | 139322/143771 MiB (96.9%) | ✅ |
| c1_i1024_o1024 | main | 1 | 1024 | 1024 | 18.32 | 5 | 0.27 | 97.06 | 83.58 | 180.64 | 98.32 | 100.14 | 100.52 | 11.71 | 11.90 | 11.93 | 3661.85 | 8044.41 | 8200.76 | 139322/143771 MiB (96.9%) | ✅ |
| c128_i1024_o1024 | main | 128 | 1024 | 1024 | 90.39 | 640 | 7.08 | 3704.45 | 3578.74 | 7283.20 | 340.83 | 1147.11 | 1259.64 | 32.79 | 39.48 | 50.72 | 16393.11 | 32588.31 | 34693.91 | 139322/143771 MiB (96.9%) | ✅ |
| c1_i1024_o2048 | main | 1 | 1024 | 2048 | 66.40 | 5 | 0.08 | 26.78 | 84.74 | 111.51 | 95.99 | 105.11 | 105.45 | 11.70 | 11.81 | 11.81 | 13278.26 | 20062.55 | 20192.89 | 139322/143771 MiB (96.9%) | ✅ |
| c128_i1024_o2048 | main | 128 | 1024 | 2048 | 145.52 | 640 | 4.40 | 2301.16 | 4467.90 | 6769.06 | 282.02 | 990.93 | 1038.67 | 25.24 | 28.25 | 31.35 | 25564.98 | 48968.45 | 52638.14 | 139322/143771 MiB (96.9%) | ✅ |
| c1_i1024_o4096 | main | 1 | 1024 | 4096 | 136.81 | 5 | 0.04 | 13.00 | 86.04 | 99.04 | 98.28 | 114.24 | 116.37 | 11.59 | 11.84 | 11.86 | 27359.51 | 42290.37 | 43455.08 | 139322/143771 MiB (96.9%) | ✅ |
| c128_i1024_o4096 | main | 128 | 1024 | 4096 | 264.80 | 640 | 2.42 | 1264.57 | 4961.16 | 6225.74 | 265.10 | 901.88 | 946.01 | 22.33 | 23.78 | 25.83 | 45632.76 | 86442.33 | 91949.11 | 139322/143771 MiB (96.9%) | ✅ |
| c1_i4096_o128 | main | 1 | 4096 | 128 | 5.45 | 5 | 0.92 | 1829.89 | 69.56 | 1899.45 | 221.18 | 353.45 | 371.20 | 11.63 | 11.95 | 12.00 | 1088.26 | 1560.46 | 1608.68 | 139322/143771 MiB (96.9%) | ✅ |
| c64_i4096_o128 | main | 64 | 4096 | 128 | 29.05 | 320 | 11.02 | 23259.85 | 721.10 | 23980.95 | 566.58 | 1790.01 | 2034.19 | 83.97 | 111.62 | 188.61 | 5607.28 | 11212.93 | 12237.14 | 139324/143771 MiB (96.9%) | ⚠️ |
| c1_i4096_o256 | main | 1 | 4096 | 256 | 6.50 | 5 | 0.77 | 1534.00 | 78.01 | 1612.01 | 128.46 | 208.90 | 226.63 | 11.61 | 11.87 | 11.89 | 1298.45 | 2189.81 | 2296.91 | 139324/143771 MiB (96.9%) | ✅ |
| c64_i4096_o256 | main | 64 | 4096 | 256 | 37.63 | 320 | 8.50 | 17956.97 | 1101.01 | 19057.98 | 447.33 | 1290.52 | 1667.29 | 55.00 | 72.09 | 89.93 | 7132.99 | 14073.09 | 15631.13 | 139324/143771 MiB (96.9%) | ⚠️ |
| c1_i4096_o512 | main | 1 | 4096 | 512 | 6.48 | 5 | 0.77 | 1539.75 | 78.30 | 1618.05 | 124.97 | 210.04 | 232.13 | 11.63 | 11.95 | 11.98 | 1293.49 | 2240.30 | 2378.47 | 139324/143771 MiB (96.9%) | ✅ |
| c64_i4096_o512 | main | 64 | 4096 | 512 | 49.46 | 320 | 6.47 | 13661.57 | 1712.46 | 15374.03 | 363.21 | 1099.58 | 1280.63 | 32.92 | 40.95 | 51.15 | 9074.10 | 17834.26 | 18838.52 | 139324/143771 MiB (96.9%) | ✅ |
| c1_i4096_o1024 | main | 1 | 4096 | 1024 | 18.31 | 5 | 0.27 | 544.41 | 83.60 | 628.01 | 103.66 | 113.91 | 115.64 | 11.55 | 11.86 | 11.87 | 3661.10 | 8164.68 | 8360.74 | 139324/143771 MiB (96.9%) | ✅ |
| c64_i4096_o1024 | main | 64 | 4096 | 1024 | 72.82 | 320 | 4.39 | 9278.68 | 2351.38 | 11630.06 | 336.61 | 1083.22 | 1200.04 | 24.41 | 28.13 | 32.24 | 13050.59 | 25157.60 | 26088.66 | 139324/143771 MiB (96.9%) | ✅ |
| c1_i4096_o2048 | main | 1 | 4096 | 2048 | 65.21 | 5 | 0.08 | 152.90 | 86.30 | 239.20 | 107.97 | 122.31 | 124.03 | 11.54 | 11.75 | 11.78 | 13039.42 | 19438.63 | 19528.19 | 139324/143771 MiB (96.9%) | ✅ |
| c64_i4096_o2048 | main | 64 | 4096 | 2048 | 117.01 | 320 | 2.73 | 5774.33 | 2846.07 | 8620.40 | 297.64 | 1149.71 | 1194.86 | 19.80 | 22.43 | 24.96 | 20535.82 | 39745.31 | 41227.42 | 139324/143771 MiB (96.9%) | ✅ |
| c1_i4096_o4096 | main | 1 | 4096 | 4096 | 137.51 | 5 | 0.04 | 72.50 | 85.60 | 158.10 | 114.72 | 130.43 | 132.68 | 11.66 | 11.87 | 11.89 | 27500.53 | 42270.91 | 43588.26 | 139324/143771 MiB (96.9%) | ✅ |
| c64_i4096_o4096 | main | 64 | 4096 | 4096 | 214.54 | 320 | 1.49 | 3149.26 | 3050.95 | 6200.21 | 274.73 | 900.93 | 999.74 | 17.93 | 19.28 | 20.04 | 36687.19 | 72324.60 | 75036.72 | 139324/143771 MiB (96.9%) | ✅ |
| c1_i16384_o128 | main | 1 | 16384 | 128 | 5.71 | 5 | 0.88 | 7481.41 | 66.35 | 7547.76 | 320.96 | 599.10 | 618.22 | 11.00 | 11.48 | 11.54 | 1140.76 | 1856.53 | 1948.72 | 139324/143771 MiB (96.9%) | ✅ |
| c32_i16384_o128 | main | 32 | 16384 | 128 | 37.11 | 160 | 4.31 | 34791.25 | 284.72 | 35075.97 | 956.28 | 3298.27 | 4051.80 | 107.21 | 186.59 | 376.91 | 7222.82 | 13755.15 | 15245.93 | 139324/143771 MiB (96.9%) | ❌ |
| c1_i16384_o256 | main | 1 | 16384 | 256 | 7.07 | 5 | 0.71 | 6043.42 | 71.69 | 6115.12 | 256.48 | 448.96 | 486.54 | 11.52 | 11.66 | 11.67 | 1412.57 | 2447.12 | 2621.34 | 139324/143771 MiB (96.9%) | ✅ |
| c32_i16384_o256 | main | 32 | 16384 | 256 | 35.65 | 160 | 4.49 | 36211.60 | 623.03 | 36834.63 | 680.08 | 1913.91 | 2384.97 | 47.34 | 71.67 | 147.97 | 6749.38 | 12780.12 | 13936.33 | 139684/143771 MiB (97.2%) | ✅ |
| c1_i16384_o512 | main | 1 | 16384 | 512 | 6.52 | 5 | 0.77 | 6550.19 | 77.70 | 6627.89 | 139.05 | 188.54 | 190.96 | 11.55 | 11.73 | 11.74 | 1303.13 | 2227.30 | 2356.50 | 139684/143771 MiB (97.2%) | ✅ |
| c32_i16384_o512 | main | 32 | 16384 | 512 | 42.41 | 160 | 3.77 | 30447.27 | 952.48 | 31399.75 | 538.18 | 1520.07 | 2259.12 | 30.21 | 45.07 | 57.04 | 7937.47 | 16413.95 | 18769.12 | 139684/143771 MiB (97.2%) | ✅ |
| c1_i16384_o1024 | main | 1 | 16384 | 1024 | 18.25 | 5 | 0.27 | 2341.54 | 83.88 | 2425.42 | 188.47 | 322.92 | 349.35 | 11.31 | 11.64 | 11.64 | 3648.93 | 7809.61 | 7915.88 | 139684/143771 MiB (97.2%) | ✅ |
| c32_i16384_o1024 | main | 32 | 16384 | 1024 | 63.50 | 160 | 2.52 | 20332.49 | 1337.54 | 21670.03 | 485.63 | 1871.97 | 2404.37 | 22.95 | 27.86 | 72.28 | 11657.91 | 21919.16 | 22977.81 | 139684/143771 MiB (97.2%) | ✅ |
| c1_i16384_o2048 | main | 1 | 16384 | 2048 | 65.72 | 5 | 0.08 | 650.30 | 85.62 | 735.91 | 175.36 | 249.46 | 261.08 | 11.54 | 11.64 | 11.65 | 13142.27 | 19985.51 | 20210.50 | 139684/143771 MiB (97.2%) | ✅ |
| c32_i16384_o2048 | main | 32 | 16384 | 2048 | 102.63 | 160 | 1.56 | 12580.64 | 1785.47 | 14366.11 | 336.45 | 1089.13 | 1111.26 | 15.83 | 17.47 | 19.70 | 18450.05 | 31359.53 | 33430.94 | 139684/143771 MiB (97.2%) | ✅ |
| c1_i65536_o128 | main | 1 | 65536 | 128 | 11.40 | 5 | 0.44 | 15249.82 | 33.25 | 15283.07 | 1429.50 | 2811.65 | 2892.88 | 11.47 | 12.08 | 12.21 | 2278.14 | 4059.40 | 4233.91 | 139706/143771 MiB (97.2%) | ✅ |
| c8_i65536_o128 | main | 8 | 65536 | 128 | 46.50 | 40 | 0.86 | 31649.77 | 55.23 | 31705.00 | 2642.31 | 5855.87 | 7061.54 | 115.39 | 327.74 | 399.92 | 9193.45 | 17992.56 | 18685.86 | 139708/143771 MiB (97.2%) | ❌ |
| c1_i65536_o256 | main | 1 | 65536 | 256 | 10.35 | 5 | 0.48 | 16785.56 | 48.96 | 16834.52 | 902.47 | 2181.49 | 2314.91 | 11.55 | 11.95 | 11.96 | 2069.19 | 3540.82 | 3709.96 | 139708/143771 MiB (97.2%) | ✅ |
| c8_i65536_o256 | main | 8 | 65536 | 256 | 44.64 | 40 | 0.90 | 32962.95 | 117.73 | 33080.69 | 1800.38 | 4029.43 | 4806.74 | 54.45 | 101.30 | 122.71 | 8621.68 | 16550.72 | 18052.15 | 139708/143771 MiB (97.2%) | ❌ |
| c1_i65536_o512 | main | 1 | 65536 | 512 | 8.02 | 5 | 0.62 | 21662.85 | 63.19 | 21726.04 | 501.88 | 1276.37 | 1453.96 | 11.06 | 11.68 | 11.78 | 1602.98 | 2564.65 | 2624.42 | 139708/143771 MiB (97.2%) | ✅ |
| c8_i65536_o512 | main | 8 | 65536 | 512 | 40.84 | 40 | 0.98 | 36028.75 | 272.84 | 36301.59 | 1260.91 | 3168.19 | 5671.91 | 24.56 | 51.52 | 85.56 | 7629.47 | 14481.52 | 18250.24 | 139708/143771 MiB (97.2%) | ⚠️ |
| c1_i65536_o1024 | main | 1 | 65536 | 1024 | 18.42 | 5 | 0.27 | 9437.05 | 83.13 | 9520.17 | 255.57 | 375.58 | 383.90 | 11.12 | 11.57 | 11.58 | 3681.79 | 8015.79 | 8213.89 | 139708/143771 MiB (97.2%) | ✅ |
| c8_i65536_o1024 | main | 8 | 65536 | 1024 | 62.34 | 40 | 0.64 | 23606.18 | 351.24 | 23957.42 | 962.35 | 2850.09 | 2967.29 | 19.43 | 29.44 | 36.68 | 11157.95 | 21105.82 | 21507.37 | 139708/143771 MiB (97.2%) | ✅ |
| c1_i131072_o128 | main | 1 | 131072 | 128 | 15.53 | 5 | 0.32 | 23847.78 | 24.40 | 23872.18 | 2286.42 | 3831.71 | 3847.18 | 11.06 | 11.65 | 11.68 | 3104.64 | 4990.35 | 5071.55 | 140228/143771 MiB (97.5%) | ⚠️ |
| c4_i131072_o128 | main | 4 | 131072 | 128 | 27.14 | 20 | 0.74 | 54328.24 | 41.97 | 54370.21 | 2648.47 | 5381.79 | 6485.26 | 63.46 | 234.23 | 254.36 | 5273.33 | 9530.16 | 9800.84 | 140228/143771 MiB (97.5%) | ❌ |
| c1_i131072_o256 | main | 1 | 131072 | 256 | 10.82 | 5 | 0.46 | 34222.74 | 46.84 | 34269.59 | 1053.51 | 3143.30 | 3630.53 | 11.11 | 11.90 | 11.97 | 2162.71 | 4514.33 | 4950.01 | 140228/143771 MiB (97.5%) | ⚠️ |
| c4_i131072_o256 | main | 4 | 131072 | 256 | 39.32 | 20 | 0.51 | 37498.25 | 64.78 | 37563.03 | 2168.98 | 4431.12 | 6485.73 | 38.02 | 82.59 | 93.31 | 7621.59 | 19293.47 | 21326.10 | 140228/143771 MiB (97.5%) | ⚠️ |
| c1_i131072_o512 | main | 1 | 131072 | 512 | 7.44 | 5 | 0.67 | 49763.22 | 68.11 | 49831.33 | 391.27 | 574.50 | 577.45 | 10.89 | 11.54 | 11.54 | 1487.11 | 2243.60 | 2318.95 | 140228/143771 MiB (97.5%) | ✅ |
| c4_i131072_o512 | main | 4 | 131072 | 512 | 43.08 | 20 | 0.46 | 34227.86 | 136.39 | 34364.24 | 1588.49 | 5162.58 | 5751.13 | 24.26 | 43.94 | 51.73 | 8028.29 | 13969.64 | 14391.10 | 140228/143771 MiB (97.5%) | ⚠️ |
| c1_i262144_o128 | main | 1 | 262144 | 128 | 33.40 | 5 | 0.15 | 22865.42 | 11.35 | 22876.77 | 5896.15 | 13444.27 | 14788.12 | 9.88 | 11.09 | 11.15 | 6677.66 | 14582.28 | 16032.30 | 140744/143771 MiB (97.9%) | ⚠️ |
| c2_i262144_o128 | main | 2 | 262144 | 128 | 36.61 | 10 | 0.27 | 38069.90 | 20.65 | 38090.55 | 4480.80 | 9292.65 | 9433.23 | 24.30 | 74.40 | 75.57 | 6665.65 | 10320.34 | 10323.72 | 140744/143771 MiB (97.9%) | ⚠️ |
| c1_i262144_o256 | main | 1 | 262144 | 256 | 27.87 | 5 | 0.18 | 27404.52 | 18.19 | 27422.71 | 4494.81 | 8943.43 | 9385.72 | 10.07 | 11.43 | 11.50 | 5571.33 | 10356.69 | 10671.30 | 141258/143771 MiB (98.3%) | ⚠️ |
| c2_i262144_o256 | main | 2 | 262144 | 256 | 33.18 | 10 | 0.30 | 42010.29 | 42.07 | 42052.36 | 2785.85 | 9153.02 | 10642.68 | 24.92 | 66.10 | 74.81 | 6437.85 | 12049.75 | 12104.02 | 141258/143771 MiB (98.3%) | ⚠️ |
| c1_i524288_o128 | main | 1 | 524288 | 128 | 38.85 | 5 | 0.13 | 26404.42 | 9.76 | 26414.18 | 7011.27 | 17139.22 | 19091.80 | 9.70 | 10.39 | 10.40 | 7768.26 | 18082.00 | 20099.12 | 141258/143771 MiB (98.3%) | ⚠️ |
| c2_i524288_o128 | main | 2 | 524288 | 128 | 14.47 | 10 | 0.69 | 114432.83 | 52.24 | 114485.07 | 1445.03 | 4695.87 | 6463.46 | 15.77 | 44.14 | 65.40 | 2876.67 | 7604.53 | 8060.58 | 141258/143771 MiB (98.3%) | ⚠️ |
| c1_i1048576_o128 | main | 1 | 1048576 | 128 | 78.14 | 5 | 0.06 | 19836.89 | 4.85 | 19841.74 | 14906.36 | 46081.66 | 51394.85 | 8.95 | 10.50 | 10.51 | 15626.56 | 46632.30 | 51849.45 | 141258/143771 MiB (98.3%) | ⚠️ |
| c2_i1048576_o128 | main | 2 | 1048576 | 128 | 181.75 | 10 | 0.06 | 17765.33 | 4.16 | 17769.49 | 24258.31 | 61414.70 | 65340.99 | 117.07 | 500.43 | 531.18 | 30965.88 | 62664.75 | 66294.91 | 141780/143771 MiB (98.6%) | ❌ |
SLO: S2 tier — TTFT P95 < 3000ms, TPOT mean < 50ms. ✅ pass, ⚠️ partial, ❌ fail.

3500
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp2_dp4/results.json Normal file
View File

File diff suppressed because it is too large Load Diff

57
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp2_dp4/scenarios.tsv Normal file
View File

@ -0,0 +1,57 @@
mark input_len output_len concurrency num_prompts
Y 1024 128 1 5
Y 1024 128 128 640
Y 1024 256 1 5
Y 1024 256 128 640
Y 1024 512 1 5
Y 1024 512 128 640
Y 1024 1024 1 5
Y 1024 1024 128 640
Y 1024 2048 1 5
Y 1024 2048 128 640
Y 1024 4096 1 5
Y 1024 4096 128 640
Y 4096 128 1 5
Y 4096 128 64 320
Y 4096 256 1 5
Y 4096 256 64 320
Y 4096 512 1 5
Y 4096 512 64 320
Y 4096 1024 1 5
Y 4096 1024 64 320
Y 4096 2048 1 5
Y 4096 2048 64 320
Y 4096 4096 1 5
Y 4096 4096 64 320
Y 16384 128 1 5
Y 16384 128 32 160
Y 16384 256 1 5
Y 16384 256 32 160
Y 16384 512 1 5
Y 16384 512 32 160
Y 16384 1024 1 5
Y 16384 1024 32 160
Y 16384 2048 1 5
Y 16384 2048 32 160
Y 65536 128 1 5
Y 65536 128 8 40
Y 65536 256 1 5
Y 65536 256 8 40
Y 65536 512 1 5
Y 65536 512 8 40
Y 65536 1024 1 5
Y 65536 1024 8 40
Y 131072 128 1 5
Y 131072 128 4 20
Y 131072 256 1 5
Y 131072 256 4 20
Y 131072 512 1 5
Y 131072 512 4 20
Y 262144 128 1 5
Y 262144 128 2 10
Y 262144 256 1 5
Y 262144 256 2 10
Y 524288 128 1 5
Y 524288 128 2 10
Y 1048576 128 1 5
Y 1048576 128 2 10
1 mark input_len output_len concurrency num_prompts
2 Y 1024 128 1 5
3 Y 1024 128 128 640
4 Y 1024 256 1 5
5 Y 1024 256 128 640
6 Y 1024 512 1 5
7 Y 1024 512 128 640
8 Y 1024 1024 1 5
9 Y 1024 1024 128 640
10 Y 1024 2048 1 5
11 Y 1024 2048 128 640
12 Y 1024 4096 1 5
13 Y 1024 4096 128 640
14 Y 4096 128 1 5
15 Y 4096 128 64 320
16 Y 4096 256 1 5
17 Y 4096 256 64 320
18 Y 4096 512 1 5
19 Y 4096 512 64 320
20 Y 4096 1024 1 5
21 Y 4096 1024 64 320
22 Y 4096 2048 1 5
23 Y 4096 2048 64 320
24 Y 4096 4096 1 5
25 Y 4096 4096 64 320
26 Y 16384 128 1 5
27 Y 16384 128 32 160
28 Y 16384 256 1 5
29 Y 16384 256 32 160
30 Y 16384 512 1 5
31 Y 16384 512 32 160
32 Y 16384 1024 1 5
33 Y 16384 1024 32 160
34 Y 16384 2048 1 5
35 Y 16384 2048 32 160
36 Y 65536 128 1 5
37 Y 65536 128 8 40
38 Y 65536 256 1 5
39 Y 65536 256 8 40
40 Y 65536 512 1 5
41 Y 65536 512 8 40
42 Y 65536 1024 1 5
43 Y 65536 1024 8 40
44 Y 131072 128 1 5
45 Y 131072 128 4 20
46 Y 131072 256 1 5
47 Y 131072 256 4 20
48 Y 131072 512 1 5
49 Y 131072 512 4 20
50 Y 262144 128 1 5
51 Y 262144 128 2 10
52 Y 262144 256 1 5
53 Y 262144 256 2 10
54 Y 524288 128 1 5
55 Y 524288 128 2 10
56 Y 1048576 128 1 5
57 Y 1048576 128 2 10

70
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp4_dp2/report.md Normal file
View File

@ -0,0 +1,70 @@
# 8x NVIDIA H200 143GB VLLM Benchmark Report
- Result root: `/data/user1/yy/experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp4_dp2`
- Model: `/data/models/DeepSeek-V4-Flash`
- Backend: VLLM
- Benchmark client: `sglang.bench_serving --backend vllm`
## Results
| Scenario | Phase | Concurrency | Input | Output | Duration(s) | Success | Req/s | In tok/s | Out tok/s | Total tok/s | Mean TTFT(ms) | P95 TTFT(ms) | P99 TTFT(ms) | Mean TPOT(ms) | P95 TPOT(ms) | P99 TPOT(ms) | Mean E2E(ms) | P95 E2E(ms) | P99 E2E(ms) | Peak GPU mem | SLO |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| c1_i1024_o128 | main | 1 | 1024 | 128 | 3.72 | 5 | 1.34 | 477.80 | 101.85 | 579.65 | 134.63 | 302.85 | 345.83 | 8.17 | 8.32 | 8.35 | 742.89 | 1030.20 | 1055.00 | 138396/143771 MiB (96.3%) | ✅ |
| c128_i1024_o128 | main | 128 | 1024 | 128 | 42.00 | 640 | 15.24 | 7973.13 | 994.43 | 8967.56 | 615.83 | 1659.18 | 1917.69 | 122.83 | 156.63 | 189.20 | 8188.50 | 16737.02 | 18386.47 | 139454/143771 MiB (97.0%) | ⚠️ |
| c1_i1024_o256 | main | 1 | 1024 | 256 | 4.46 | 5 | 1.12 | 399.09 | 113.80 | 512.89 | 76.65 | 88.52 | 90.59 | 8.12 | 8.21 | 8.23 | 889.64 | 1486.72 | 1572.36 | 139454/143771 MiB (97.0%) | ✅ |
| c128_i1024_o256 | main | 128 | 1024 | 256 | 42.40 | 640 | 15.10 | 7897.93 | 1996.42 | 9894.36 | 381.79 | 938.43 | 1171.74 | 60.05 | 74.97 | 96.47 | 8045.59 | 15999.15 | 16952.04 | 139454/143771 MiB (97.0%) | ⚠️ |
| c1_i1024_o512 | main | 1 | 1024 | 512 | 4.58 | 5 | 1.09 | 388.12 | 110.67 | 498.80 | 101.05 | 187.00 | 208.64 | 8.12 | 8.17 | 8.18 | 914.85 | 1486.90 | 1572.17 | 139454/143771 MiB (97.0%) | ✅ |
| c128_i1024_o512 | main | 128 | 1024 | 512 | 54.32 | 640 | 11.78 | 6164.96 | 2986.46 | 9151.42 | 316.84 | 996.03 | 1011.18 | 39.13 | 48.34 | 56.02 | 10050.16 | 20325.81 | 22283.80 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i1024_o1024 | main | 1 | 1024 | 1024 | 12.71 | 5 | 0.39 | 139.93 | 120.49 | 260.41 | 75.65 | 90.47 | 90.97 | 8.11 | 8.19 | 8.21 | 2539.90 | 5592.97 | 5705.34 | 139456/143771 MiB (97.0%) | ✅ |
| c128_i1024_o1024 | main | 128 | 1024 | 1024 | 74.21 | 640 | 8.62 | 4512.43 | 4359.31 | 8871.74 | 224.26 | 719.08 | 837.92 | 26.31 | 30.84 | 33.28 | 13282.76 | 25759.40 | 27367.26 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i1024_o2048 | main | 1 | 1024 | 2048 | 45.78 | 5 | 0.11 | 38.84 | 122.91 | 161.75 | 77.63 | 84.75 | 84.77 | 8.08 | 8.09 | 8.09 | 9154.40 | 13865.78 | 13978.61 | 139456/143771 MiB (97.0%) | ✅ |
| c128_i1024_o2048 | main | 128 | 1024 | 2048 | 132.42 | 640 | 4.83 | 2528.71 | 4909.69 | 7438.40 | 249.17 | 979.55 | 1002.42 | 22.76 | 24.25 | 25.28 | 23085.59 | 43805.99 | 47303.18 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i1024_o4096 | main | 1 | 1024 | 4096 | 95.72 | 5 | 0.05 | 18.57 | 122.97 | 141.55 | 77.88 | 82.10 | 82.14 | 8.09 | 8.12 | 8.13 | 19143.13 | 29602.89 | 30482.21 | 139456/143771 MiB (97.0%) | ✅ |
| c128_i1024_o4096 | main | 128 | 1024 | 4096 | 257.05 | 640 | 2.49 | 1302.66 | 5110.60 | 6413.26 | 197.57 | 737.03 | 754.71 | 21.93 | 23.01 | 23.26 | 44808.43 | 84504.69 | 90427.39 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i4096_o128 | main | 1 | 4096 | 128 | 3.47 | 5 | 1.44 | 2872.58 | 109.20 | 2981.78 | 83.99 | 96.00 | 96.62 | 8.16 | 8.23 | 8.24 | 692.79 | 1028.22 | 1054.23 | 139456/143771 MiB (97.0%) | ✅ |
| c64_i4096_o128 | main | 64 | 4096 | 128 | 30.68 | 320 | 10.43 | 22020.97 | 682.69 | 22703.66 | 584.11 | 1904.20 | 2262.19 | 88.50 | 123.52 | 173.68 | 5949.69 | 11822.35 | 12463.74 | 139456/143771 MiB (97.0%) | ⚠️ |
| c1_i4096_o256 | main | 1 | 4096 | 256 | 4.51 | 5 | 1.11 | 2209.73 | 112.37 | 2322.10 | 83.55 | 91.50 | 91.90 | 8.16 | 8.23 | 8.24 | 900.98 | 1513.84 | 1604.71 | 139456/143771 MiB (97.0%) | ✅ |
| c64_i4096_o256 | main | 64 | 4096 | 256 | 31.29 | 320 | 10.23 | 21590.10 | 1323.77 | 22913.87 | 346.30 | 1198.05 | 1234.72 | 45.94 | 59.77 | 82.58 | 5923.11 | 11845.66 | 13085.09 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i4096_o512 | main | 1 | 4096 | 512 | 4.55 | 5 | 1.10 | 2191.28 | 111.43 | 2302.71 | 91.00 | 104.51 | 107.11 | 8.16 | 8.24 | 8.25 | 908.65 | 1527.91 | 1621.06 | 139456/143771 MiB (97.0%) | ✅ |
| c64_i4096_o512 | main | 64 | 4096 | 512 | 39.52 | 320 | 8.10 | 17097.00 | 2143.08 | 19240.08 | 277.89 | 946.13 | 974.08 | 26.25 | 32.55 | 35.60 | 7228.62 | 14145.53 | 15173.06 | 139456/143771 MiB (97.0%) | ✅ |
| c1_i4096_o1024 | main | 1 | 4096 | 1024 | 12.82 | 5 | 0.39 | 777.58 | 119.41 | 896.98 | 82.49 | 92.40 | 93.46 | 8.15 | 8.23 | 8.24 | 2563.02 | 5648.98 | 5771.30 | 139456/143771 MiB (97.0%) | ✅ |
| c64_i4096_o1024 | main | 64 | 4096 | 1024 | 58.73 | 320 | 5.45 | 11503.60 | 2915.21 | 14418.81 | 261.52 | 1023.51 | 1212.54 | 19.18 | 21.78 | 24.49 | 10436.37 | 20083.65 | 21920.34 | 139824/143771 MiB (97.3%) | ✅ |
| c1_i4096_o2048 | main | 1 | 4096 | 2048 | 46.18 | 5 | 0.11 | 215.88 | 121.84 | 337.72 | 89.17 | 105.61 | 108.34 | 8.13 | 8.16 | 8.16 | 9235.07 | 14025.34 | 14152.06 | 139824/143771 MiB (97.3%) | ✅ |
| c64_i4096_o2048 | main | 64 | 4096 | 2048 | 105.18 | 320 | 3.04 | 6423.33 | 3165.95 | 9589.28 | 219.53 | 856.07 | 869.36 | 17.64 | 18.77 | 20.69 | 18415.93 | 35312.14 | 37188.62 | 139824/143771 MiB (97.3%) | ✅ |
| c1_i4096_o4096 | main | 1 | 4096 | 4096 | 96.28 | 5 | 0.05 | 103.55 | 122.26 | 225.82 | 92.47 | 110.67 | 114.17 | 8.14 | 8.17 | 8.17 | 19254.34 | 29794.95 | 30710.72 | 139824/143771 MiB (97.3%) | ✅ |
| c64_i4096_o4096 | main | 64 | 4096 | 4096 | 196.97 | 320 | 1.62 | 3430.14 | 3323.06 | 6753.20 | 191.83 | 727.11 | 753.66 | 16.53 | 17.22 | 17.64 | 33830.38 | 65711.11 | 68039.71 | 139824/143771 MiB (97.3%) | ✅ |
| c1_i16384_o128 | main | 1 | 16384 | 128 | 4.39 | 5 | 1.14 | 9724.92 | 86.24 | 9811.16 | 272.22 | 512.76 | 518.98 | 8.11 | 8.23 | 8.25 | 877.50 | 1371.53 | 1441.62 | 139824/143771 MiB (97.3%) | ✅ |
| c32_i16384_o128 | main | 32 | 16384 | 128 | 31.01 | 160 | 5.16 | 41639.17 | 340.76 | 41979.93 | 908.42 | 3479.51 | 3760.82 | 85.23 | 160.17 | 235.33 | 6030.16 | 11373.02 | 12999.09 | 139824/143771 MiB (97.3%) | ❌ |
| c1_i16384_o256 | main | 1 | 16384 | 256 | 5.10 | 5 | 0.98 | 8380.37 | 99.42 | 8479.79 | 203.25 | 407.06 | 440.10 | 8.14 | 8.23 | 8.24 | 1018.57 | 1804.68 | 1937.46 | 139824/143771 MiB (97.3%) | ✅ |
| c32_i16384_o256 | main | 32 | 16384 | 256 | 29.56 | 160 | 5.41 | 43677.33 | 751.48 | 44428.80 | 597.59 | 1701.48 | 2572.97 | 38.88 | 55.35 | 126.86 | 5607.62 | 10467.65 | 11645.26 | 140338/143771 MiB (97.6%) | ✅ |
| c1_i16384_o512 | main | 1 | 16384 | 512 | 4.66 | 5 | 1.07 | 9162.19 | 108.69 | 9270.88 | 114.10 | 155.92 | 157.11 | 8.16 | 8.23 | 8.25 | 931.42 | 1574.32 | 1661.52 | 140338/143771 MiB (97.6%) | ✅ |
| c32_i16384_o512 | main | 32 | 16384 | 512 | 29.42 | 160 | 5.44 | 43889.80 | 1373.00 | 45262.81 | 313.79 | 965.45 | 1010.01 | 20.31 | 25.61 | 28.12 | 5422.34 | 10895.57 | 12002.66 | 140854/143771 MiB (98.0%) | ✅ |
| c1_i16384_o1024 | main | 1 | 16384 | 1024 | 13.04 | 5 | 0.38 | 3277.50 | 117.41 | 3394.91 | 114.24 | 158.31 | 161.09 | 8.16 | 8.23 | 8.24 | 2606.54 | 5717.71 | 5837.98 | 140854/143771 MiB (98.0%) | ✅ |
| c32_i16384_o1024 | main | 32 | 16384 | 1024 | 49.82 | 160 | 3.21 | 25915.11 | 1704.78 | 27619.89 | 303.13 | 1131.20 | 1660.57 | 17.14 | 21.79 | 37.19 | 9072.32 | 16361.33 | 18824.49 | 140854/143771 MiB (98.0%) | ✅ |
| c1_i16384_o2048 | main | 1 | 16384 | 2048 | 46.44 | 5 | 0.11 | 920.20 | 121.16 | 1041.36 | 117.87 | 155.57 | 156.44 | 8.14 | 8.18 | 8.19 | 9287.27 | 14109.44 | 14233.84 | 140854/143771 MiB (98.0%) | ✅ |
| c32_i16384_o2048 | main | 32 | 16384 | 2048 | 92.42 | 160 | 1.73 | 13970.52 | 1982.72 | 15953.24 | 323.18 | 1143.92 | 2172.49 | 14.60 | 16.68 | 25.49 | 16690.84 | 28308.53 | 30322.94 | 140854/143771 MiB (98.0%) | ✅ |
| c1_i65536_o128 | main | 1 | 65536 | 128 | 8.91 | 5 | 0.56 | 19510.10 | 42.54 | 19552.65 | 1193.16 | 2211.56 | 2223.60 | 7.89 | 8.16 | 8.19 | 1780.28 | 3036.84 | 3092.42 | 140854/143771 MiB (98.0%) | ✅ |
| c8_i65536_o128 | main | 8 | 65536 | 128 | 42.45 | 40 | 0.94 | 34665.79 | 60.49 | 34726.28 | 2187.88 | 5620.08 | 5766.46 | 118.91 | 301.63 | 309.58 | 8428.69 | 16394.59 | 16549.56 | 140854/143771 MiB (98.0%) | ❌ |
| c1_i65536_o256 | main | 1 | 65536 | 256 | 4.96 | 5 | 1.01 | 35030.52 | 102.18 | 35132.70 | 190.89 | 274.65 | 283.52 | 7.97 | 8.23 | 8.25 | 990.97 | 1580.31 | 1647.86 | 140854/143771 MiB (98.0%) | ✅ |
| c8_i65536_o256 | main | 8 | 65536 | 256 | 30.60 | 40 | 1.31 | 48097.46 | 171.79 | 48269.25 | 1187.02 | 3353.05 | 4860.71 | 44.63 | 113.59 | 141.58 | 5938.64 | 14918.88 | 15303.64 | 140854/143771 MiB (98.0%) | ⚠️ |
| c1_i65536_o512 | main | 1 | 65536 | 512 | 5.10 | 5 | 0.98 | 34094.52 | 99.45 | 34193.97 | 221.62 | 326.62 | 330.25 | 7.91 | 8.17 | 8.18 | 1018.20 | 1589.95 | 1655.09 | 140854/143771 MiB (98.0%) | ✅ |
| c8_i65536_o512 | main | 8 | 65536 | 512 | 27.16 | 40 | 1.47 | 54172.15 | 410.24 | 54582.39 | 493.38 | 1809.22 | 2638.39 | 17.90 | 23.23 | 79.99 | 4993.77 | 8917.36 | 9676.38 | 140854/143771 MiB (98.0%) | ✅ |
| c1_i65536_o1024 | main | 1 | 65536 | 1024 | 13.63 | 5 | 0.37 | 12750.53 | 112.31 | 12862.85 | 229.69 | 358.50 | 364.11 | 8.07 | 8.26 | 8.26 | 2724.95 | 5751.59 | 5842.34 | 140854/143771 MiB (98.0%) | ✅ |
| c8_i65536_o1024 | main | 8 | 65536 | 1024 | 41.44 | 40 | 0.97 | 35510.70 | 528.38 | 36039.07 | 576.47 | 2177.55 | 2783.15 | 13.13 | 20.42 | 32.65 | 7344.83 | 12591.94 | 13878.42 | 140854/143771 MiB (98.0%) | ✅ |
| c1_i131072_o128 | main | 1 | 131072 | 128 | 13.24 | 5 | 0.38 | 27974.16 | 28.62 | 28002.78 | 2074.69 | 3498.43 | 3510.98 | 7.74 | 8.18 | 8.22 | 2646.89 | 4297.81 | 4353.06 | 141374/143771 MiB (98.3%) | ⚠️ |
| c4_i131072_o128 | main | 4 | 131072 | 128 | 32.85 | 20 | 0.61 | 44885.93 | 34.68 | 44920.61 | 2993.02 | 7688.92 | 7746.95 | 55.15 | 151.40 | 157.05 | 6305.32 | 12451.58 | 13546.32 | 141374/143771 MiB (98.3%) | ❌ |
| c1_i131072_o256 | main | 1 | 131072 | 256 | 8.19 | 5 | 0.61 | 45250.75 | 61.94 | 45312.69 | 851.06 | 2612.01 | 3033.01 | 7.81 | 8.23 | 8.24 | 1636.29 | 3468.17 | 3829.62 | 141374/143771 MiB (98.3%) | ✅ |
| c4_i131072_o256 | main | 4 | 131072 | 256 | 25.17 | 20 | 0.79 | 58583.41 | 101.20 | 58684.61 | 1578.82 | 6019.33 | 6092.09 | 20.62 | 43.90 | 46.90 | 4833.02 | 10204.22 | 10875.06 | 141374/143771 MiB (98.3%) | ⚠️ |
| c1_i131072_o512 | main | 1 | 131072 | 512 | 5.60 | 5 | 0.89 | 66152.93 | 90.55 | 66243.48 | 333.30 | 555.04 | 573.87 | 7.81 | 8.24 | 8.27 | 1118.43 | 1626.84 | 1649.69 | 141374/143771 MiB (98.3%) | ✅ |
| c4_i131072_o512 | main | 4 | 131072 | 512 | 19.45 | 20 | 1.03 | 75794.92 | 302.02 | 76096.94 | 489.33 | 724.28 | 1360.67 | 9.94 | 13.97 | 16.16 | 3412.52 | 5788.93 | 5861.60 | 141374/143771 MiB (98.3%) | ✅ |
| c1_i262144_o128 | main | 1 | 262144 | 128 | 24.68 | 5 | 0.20 | 30936.55 | 15.35 | 30951.90 | 4393.17 | 8221.07 | 8613.48 | 6.70 | 7.64 | 7.66 | 4935.30 | 9036.75 | 9509.11 | 141374/143771 MiB (98.3%) | ⚠️ |
| c2_i262144_o128 | main | 2 | 262144 | 128 | 30.76 | 10 | 0.33 | 45318.61 | 24.58 | 45343.19 | 3243.14 | 7994.94 | 8377.33 | 25.23 | 70.28 | 72.19 | 5572.85 | 9102.02 | 9106.67 | 141374/143771 MiB (98.3%) | ⚠️ |
| c1_i262144_o256 | main | 1 | 262144 | 256 | 7.02 | 5 | 0.71 | 108820.99 | 72.25 | 108893.24 | 643.55 | 962.12 | 1036.06 | 6.92 | 7.88 | 7.93 | 1401.77 | 2062.13 | 2080.08 | 141374/143771 MiB (98.3%) | ✅ |
| c2_i262144_o256 | main | 2 | 262144 | 256 | 20.99 | 10 | 0.48 | 66416.34 | 66.52 | 66482.86 | 1808.18 | 6399.54 | 6424.61 | 10.75 | 22.96 | 32.38 | 3491.75 | 7972.48 | 8063.51 | 141374/143771 MiB (98.3%) | ⚠️ |
| c1_i524288_o128 | main | 1 | 524288 | 128 | 23.69 | 5 | 0.21 | 43291.93 | 16.00 | 43307.93 | 4207.11 | 14806.51 | 17551.38 | 6.63 | 7.42 | 7.46 | 4736.94 | 15565.54 | 18281.61 | 141374/143771 MiB (98.3%) | ⚠️ |
| c2_i524288_o128 | main | 2 | 524288 | 128 | 15.61 | 10 | 0.64 | 106086.70 | 48.43 | 106135.13 | 1630.83 | 6241.67 | 9313.98 | 44.54 | 213.86 | 348.24 | 3078.79 | 10277.55 | 10806.98 | 141374/143771 MiB (98.3%) | ⚠️ |
| c1_i1048576_o128 | main | 1 | 1048576 | 128 | 77.70 | 5 | 0.06 | 19948.43 | 4.88 | 19953.31 | 15035.42 | 43228.08 | 48170.56 | 6.14 | 7.51 | 7.56 | 15539.21 | 43599.43 | 48471.96 | 141374/143771 MiB (98.3%) | ⚠️ |
| c2_i1048576_o128 | main | 2 | 1048576 | 128 | 165.96 | 10 | 0.06 | 19455.73 | 4.56 | 19460.29 | 19476.11 | 51577.89 | 52987.83 | 277.94 | 766.41 | 858.05 | 31427.72 | 55532.93 | 56546.05 | 141374/143771 MiB (98.3%) | ❌ |
SLO: S2 tier — TTFT P95 < 3000ms, TPOT mean < 50ms. ✅ pass, ⚠️ partial, ❌ fail.

3500
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp4_dp2/results.json Normal file
View File

File diff suppressed because it is too large Load Diff

57
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp4_dp2/scenarios.tsv Normal file
View File

@ -0,0 +1,57 @@
mark input_len output_len concurrency num_prompts
Y 1024 128 1 5
Y 1024 128 128 640
Y 1024 256 1 5
Y 1024 256 128 640
Y 1024 512 1 5
Y 1024 512 128 640
Y 1024 1024 1 5
Y 1024 1024 128 640
Y 1024 2048 1 5
Y 1024 2048 128 640
Y 1024 4096 1 5
Y 1024 4096 128 640
Y 4096 128 1 5
Y 4096 128 64 320
Y 4096 256 1 5
Y 4096 256 64 320
Y 4096 512 1 5
Y 4096 512 64 320
Y 4096 1024 1 5
Y 4096 1024 64 320
Y 4096 2048 1 5
Y 4096 2048 64 320
Y 4096 4096 1 5
Y 4096 4096 64 320
Y 16384 128 1 5
Y 16384 128 32 160
Y 16384 256 1 5
Y 16384 256 32 160
Y 16384 512 1 5
Y 16384 512 32 160
Y 16384 1024 1 5
Y 16384 1024 32 160
Y 16384 2048 1 5
Y 16384 2048 32 160
Y 65536 128 1 5
Y 65536 128 8 40
Y 65536 256 1 5
Y 65536 256 8 40
Y 65536 512 1 5
Y 65536 512 8 40
Y 65536 1024 1 5
Y 65536 1024 8 40
Y 131072 128 1 5
Y 131072 128 4 20
Y 131072 256 1 5
Y 131072 256 4 20
Y 131072 512 1 5
Y 131072 512 4 20
Y 262144 128 1 5
Y 262144 128 2 10
Y 262144 256 1 5
Y 262144 256 2 10
Y 524288 128 1 5
Y 524288 128 2 10
Y 1048576 128 1 5
Y 1048576 128 2 10
1 mark input_len output_len concurrency num_prompts
2 Y 1024 128 1 5
3 Y 1024 128 128 640
4 Y 1024 256 1 5
5 Y 1024 256 128 640
6 Y 1024 512 1 5
7 Y 1024 512 128 640
8 Y 1024 1024 1 5
9 Y 1024 1024 128 640
10 Y 1024 2048 1 5
11 Y 1024 2048 128 640
12 Y 1024 4096 1 5
13 Y 1024 4096 128 640
14 Y 4096 128 1 5
15 Y 4096 128 64 320
16 Y 4096 256 1 5
17 Y 4096 256 64 320
18 Y 4096 512 1 5
19 Y 4096 512 64 320
20 Y 4096 1024 1 5
21 Y 4096 1024 64 320
22 Y 4096 2048 1 5
23 Y 4096 2048 64 320
24 Y 4096 4096 1 5
25 Y 4096 4096 64 320
26 Y 16384 128 1 5
27 Y 16384 128 32 160
28 Y 16384 256 1 5
29 Y 16384 256 32 160
30 Y 16384 512 1 5
31 Y 16384 512 32 160
32 Y 16384 1024 1 5
33 Y 16384 1024 32 160
34 Y 16384 2048 1 5
35 Y 16384 2048 32 160
36 Y 65536 128 1 5
37 Y 65536 128 8 40
38 Y 65536 256 1 5
39 Y 65536 256 8 40
40 Y 65536 512 1 5
41 Y 65536 512 8 40
42 Y 65536 1024 1 5
43 Y 65536 1024 8 40
44 Y 131072 128 1 5
45 Y 131072 128 4 20
46 Y 131072 256 1 5
47 Y 131072 256 4 20
48 Y 131072 512 1 5
49 Y 131072 512 4 20
50 Y 262144 128 1 5
51 Y 262144 128 2 10
52 Y 262144 256 1 5
53 Y 262144 256 2 10
54 Y 524288 128 1 5
55 Y 524288 128 2 10
56 Y 1048576 128 1 5
57 Y 1048576 128 2 10

70
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp8_dp1/report.md Normal file
View File

@ -0,0 +1,70 @@
# 8x NVIDIA H200 143GB VLLM Benchmark Report
- Result root: `/data/user1/yy/experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp8_dp1`
- Model: `/data/models/DeepSeek-V4-Flash`
- Backend: VLLM
- Benchmark client: `sglang.bench_serving --backend vllm`
## Results
| Scenario | Phase | Concurrency | Input | Output | Duration(s) | Success | Req/s | In tok/s | Out tok/s | Total tok/s | Mean TTFT(ms) | P95 TTFT(ms) | P99 TTFT(ms) | Mean TPOT(ms) | P95 TPOT(ms) | P99 TPOT(ms) | Mean E2E(ms) | P95 E2E(ms) | P99 E2E(ms) | Peak GPU mem | SLO |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| c1_i1024_o128 | main | 1 | 1024 | 128 | 3.11 | 5 | 1.61 | 571.42 | 121.80 | 693.22 | 117.30 | 274.43 | 314.97 | 6.74 | 6.78 | 6.79 | 620.97 | 851.31 | 871.45 | 131900/143771 MiB (91.7%) | ✅ |
| c128_i1024_o128 | main | 128 | 1024 | 128 | 25.39 | 640 | 25.21 | 13188.24 | 1644.87 | 14833.11 | 486.29 | 1647.93 | 2127.36 | 71.51 | 108.98 | 146.42 | 4876.35 | 10467.31 | 11987.25 | 132930/143771 MiB (92.5%) | ⚠️ |
| c1_i1024_o256 | main | 1 | 1024 | 256 | 3.58 | 5 | 1.40 | 497.17 | 141.77 | 638.94 | 40.11 | 44.63 | 45.58 | 6.71 | 6.72 | 6.72 | 713.89 | 1209.48 | 1281.54 | 132930/143771 MiB (92.5%) | ✅ |
| c128_i1024_o256 | main | 128 | 1024 | 256 | 27.46 | 640 | 23.31 | 12194.74 | 3082.56 | 15277.30 | 258.20 | 725.46 | 741.75 | 37.61 | 47.17 | 56.15 | 5081.28 | 9930.41 | 10662.58 | 132930/143771 MiB (92.5%) | ✅ |
| c1_i1024_o512 | main | 1 | 1024 | 512 | 3.69 | 5 | 1.36 | 482.38 | 137.55 | 619.93 | 60.69 | 65.04 | 65.69 | 6.74 | 6.78 | 6.79 | 735.69 | 1233.35 | 1305.74 | 132930/143771 MiB (92.5%) | ✅ |
| c128_i1024_o512 | main | 128 | 1024 | 512 | 37.45 | 640 | 17.09 | 8940.37 | 4330.93 | 13271.30 | 211.37 | 760.28 | 777.02 | 26.29 | 29.37 | 34.23 | 6728.56 | 13132.89 | 14073.81 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i1024_o1024 | main | 1 | 1024 | 1024 | 10.48 | 5 | 0.48 | 169.65 | 146.08 | 315.74 | 48.51 | 60.67 | 60.68 | 6.71 | 6.73 | 6.73 | 2094.65 | 4631.94 | 4727.62 | 132932/143771 MiB (92.5%) | ✅ |
| c128_i1024_o1024 | main | 128 | 1024 | 1024 | 68.42 | 640 | 9.35 | 4893.76 | 4727.69 | 9621.45 | 198.71 | 724.06 | 738.02 | 24.27 | 25.99 | 28.67 | 12260.92 | 23755.36 | 25355.91 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i1024_o2048 | main | 1 | 1024 | 2048 | 37.94 | 5 | 0.13 | 46.87 | 148.32 | 195.19 | 46.74 | 60.19 | 62.81 | 6.71 | 6.72 | 6.72 | 7585.93 | 11492.67 | 11586.89 | 132932/143771 MiB (92.5%) | ✅ |
| c128_i1024_o2048 | main | 128 | 1024 | 2048 | 133.47 | 640 | 4.80 | 2508.84 | 4871.12 | 7379.96 | 198.26 | 746.05 | 761.60 | 23.37 | 24.26 | 24.89 | 23639.36 | 44752.06 | 48201.59 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i1024_o4096 | main | 1 | 1024 | 4096 | 79.46 | 5 | 0.06 | 22.38 | 148.14 | 170.51 | 42.13 | 46.60 | 47.48 | 6.73 | 6.76 | 6.76 | 15890.77 | 24601.03 | 25332.98 | 132932/143771 MiB (92.5%) | ✅ |
| c128_i1024_o4096 | main | 128 | 1024 | 4096 | 275.08 | 640 | 2.33 | 1217.30 | 4775.68 | 5992.98 | 218.44 | 736.09 | 756.63 | 23.99 | 25.92 | 26.62 | 48923.09 | 92762.87 | 97316.79 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i4096_o128 | main | 1 | 4096 | 128 | 3.09 | 5 | 1.62 | 3231.34 | 122.84 | 3354.18 | 113.28 | 148.51 | 155.20 | 6.72 | 6.74 | 6.74 | 615.65 | 898.15 | 918.13 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o128 | main | 64 | 4096 | 128 | 27.10 | 320 | 11.81 | 24931.54 | 772.92 | 25704.46 | 705.87 | 2978.27 | 3725.28 | 76.32 | 108.07 | 233.29 | 5252.69 | 10435.23 | 11989.51 | 132932/143771 MiB (92.5%) | ⚠️ |
| c1_i4096_o256 | main | 1 | 4096 | 256 | 3.66 | 5 | 1.37 | 2724.98 | 138.57 | 2863.55 | 54.71 | 67.18 | 67.54 | 6.73 | 6.77 | 6.78 | 731.14 | 1243.27 | 1320.30 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o256 | main | 64 | 4096 | 256 | 16.45 | 320 | 19.45 | 41074.35 | 2518.43 | 43592.78 | 216.47 | 804.28 | 815.33 | 22.25 | 25.88 | 28.53 | 2990.66 | 5802.25 | 6644.63 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i4096_o512 | main | 1 | 4096 | 512 | 3.66 | 5 | 1.37 | 2724.45 | 138.55 | 2863.00 | 53.85 | 64.63 | 65.10 | 6.73 | 6.78 | 6.78 | 730.54 | 1243.77 | 1320.67 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o512 | main | 64 | 4096 | 512 | 27.32 | 320 | 11.71 | 24731.28 | 3100.03 | 27831.30 | 174.83 | 584.91 | 601.95 | 17.83 | 18.78 | 19.95 | 4860.53 | 9051.41 | 9668.74 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i4096_o1024 | main | 1 | 4096 | 1024 | 10.58 | 5 | 0.47 | 942.72 | 144.76 | 1087.48 | 54.50 | 66.58 | 66.81 | 6.73 | 6.78 | 6.79 | 2113.75 | 4687.47 | 4791.74 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o1024 | main | 64 | 4096 | 1024 | 52.14 | 320 | 6.14 | 12958.58 | 3283.93 | 16242.51 | 197.14 | 789.79 | 800.08 | 17.06 | 17.95 | 19.33 | 9239.51 | 17164.77 | 17906.82 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i4096_o2048 | main | 1 | 4096 | 2048 | 38.30 | 5 | 0.13 | 260.29 | 146.91 | 407.20 | 55.00 | 67.22 | 67.42 | 6.75 | 6.79 | 6.80 | 7659.17 | 11647.12 | 11755.04 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o2048 | main | 64 | 4096 | 2048 | 97.69 | 320 | 3.28 | 6916.20 | 3408.88 | 10325.08 | 162.14 | 570.75 | 582.03 | 16.56 | 17.06 | 18.07 | 17265.87 | 32918.48 | 34336.54 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i4096_o4096 | main | 1 | 4096 | 4096 | 79.98 | 5 | 0.06 | 124.66 | 147.18 | 271.85 | 55.41 | 66.70 | 66.85 | 6.77 | 6.80 | 6.80 | 15993.58 | 24761.64 | 25525.53 | 132932/143771 MiB (92.5%) | ✅ |
| c64_i4096_o4096 | main | 64 | 4096 | 4096 | 192.01 | 320 | 1.67 | 3518.79 | 3408.94 | 6927.73 | 163.81 | 576.41 | 599.00 | 16.44 | 16.79 | 16.98 | 33560.76 | 65515.45 | 68183.67 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i16384_o128 | main | 1 | 16384 | 128 | 3.74 | 5 | 1.34 | 11422.24 | 101.29 | 11523.54 | 244.24 | 433.34 | 440.01 | 6.72 | 6.80 | 6.80 | 746.96 | 1184.46 | 1242.36 | 132932/143771 MiB (92.5%) | ✅ |
| c32_i16384_o128 | main | 32 | 16384 | 128 | 29.67 | 160 | 5.39 | 43522.36 | 356.17 | 43878.53 | 996.18 | 3789.36 | 4532.55 | 78.86 | 150.73 | 226.61 | 5789.68 | 10661.97 | 11830.35 | 132932/143771 MiB (92.5%) | ❌ |
| c1_i16384_o256 | main | 1 | 16384 | 256 | 3.82 | 5 | 1.31 | 11173.76 | 132.55 | 11306.32 | 87.44 | 122.35 | 122.83 | 6.73 | 6.76 | 6.76 | 763.51 | 1299.07 | 1372.37 | 132932/143771 MiB (92.5%) | ✅ |
| c32_i16384_o256 | main | 32 | 16384 | 256 | 12.46 | 160 | 12.84 | 103604.79 | 1782.54 | 105387.34 | 200.10 | 636.48 | 644.96 | 14.84 | 16.94 | 20.78 | 2229.45 | 3832.96 | 4262.99 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i16384_o512 | main | 1 | 16384 | 512 | 3.81 | 5 | 1.31 | 11209.67 | 132.98 | 11342.65 | 85.20 | 118.65 | 119.23 | 6.73 | 6.76 | 6.76 | 761.16 | 1293.05 | 1365.21 | 132932/143771 MiB (92.5%) | ✅ |
| c32_i16384_o512 | main | 32 | 16384 | 512 | 20.43 | 160 | 7.83 | 63203.51 | 1977.19 | 65180.70 | 190.69 | 640.86 | 739.06 | 14.04 | 15.70 | 18.53 | 3673.45 | 7038.21 | 7504.82 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i16384_o1024 | main | 1 | 16384 | 1024 | 10.85 | 5 | 0.46 | 3939.68 | 141.13 | 4080.81 | 96.12 | 132.08 | 134.06 | 6.75 | 6.80 | 6.81 | 2168.26 | 4767.81 | 4873.19 | 132932/143771 MiB (92.5%) | ✅ |
| c32_i16384_o1024 | main | 32 | 16384 | 1024 | 40.02 | 160 | 4.00 | 32262.84 | 2122.35 | 34385.19 | 226.08 | 789.94 | 950.40 | 13.38 | 14.23 | 22.02 | 7223.57 | 13034.14 | 13797.93 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i16384_o2048 | main | 1 | 16384 | 2048 | 38.62 | 5 | 0.13 | 1106.58 | 145.70 | 1252.28 | 92.60 | 132.65 | 135.00 | 6.77 | 6.82 | 6.82 | 7722.67 | 11757.03 | 11867.20 | 132932/143771 MiB (92.5%) | ✅ |
| c32_i16384_o2048 | main | 32 | 16384 | 2048 | 82.31 | 160 | 1.94 | 15686.47 | 2226.25 | 17912.73 | 187.67 | 628.02 | 806.24 | 12.88 | 13.39 | 14.85 | 14894.46 | 25291.54 | 26578.49 | 132932/143771 MiB (92.5%) | ✅ |
| c1_i65536_o128 | main | 1 | 65536 | 128 | 5.98 | 5 | 0.84 | 29066.05 | 63.38 | 29129.43 | 697.62 | 1783.81 | 1903.90 | 6.67 | 6.75 | 6.75 | 1194.51 | 2550.76 | 2686.50 | 133260/143771 MiB (92.7%) | ✅ |
| c8_i65536_o128 | main | 8 | 65536 | 128 | 39.65 | 40 | 1.01 | 37118.27 | 64.77 | 37183.05 | 3209.81 | 7445.34 | 8645.53 | 88.42 | 248.75 | 263.80 | 7821.82 | 14289.92 | 16484.17 | 133780/143771 MiB (93.1%) | ❌ |
| c1_i65536_o256 | main | 1 | 65536 | 256 | 4.21 | 5 | 1.19 | 41330.67 | 120.56 | 41451.23 | 161.62 | 254.47 | 256.20 | 6.74 | 6.79 | 6.80 | 839.71 | 1306.35 | 1352.80 | 133780/143771 MiB (93.1%) | ✅ |
| c8_i65536_o256 | main | 8 | 65536 | 256 | 8.64 | 40 | 4.63 | 170252.44 | 608.09 | 170860.53 | 356.04 | 749.11 | 864.77 | 9.14 | 10.21 | 10.30 | 1581.50 | 2865.09 | 2998.37 | 133780/143771 MiB (93.1%) | ✅ |
| c1_i65536_o512 | main | 1 | 65536 | 512 | 4.23 | 5 | 1.18 | 41120.06 | 119.95 | 41240.01 | 184.79 | 288.97 | 290.20 | 6.54 | 6.74 | 6.75 | 843.98 | 1310.36 | 1356.99 | 133780/143771 MiB (93.1%) | ✅ |
| c8_i65536_o512 | main | 8 | 65536 | 512 | 15.79 | 40 | 2.53 | 93186.96 | 705.69 | 93892.65 | 274.50 | 661.39 | 679.12 | 9.10 | 9.63 | 10.39 | 2813.45 | 5054.25 | 5200.71 | 133780/143771 MiB (93.1%) | ✅ |
| c1_i65536_o1024 | main | 1 | 65536 | 1024 | 11.22 | 5 | 0.45 | 15489.07 | 136.44 | 15625.51 | 161.17 | 254.42 | 254.84 | 6.76 | 6.85 | 6.86 | 2242.90 | 4765.81 | 4841.93 | 133780/143771 MiB (93.1%) | ✅ |
| c8_i65536_o1024 | main | 8 | 65536 | 1024 | 29.80 | 40 | 1.34 | 49382.35 | 734.78 | 50117.12 | 281.66 | 671.08 | 707.77 | 8.91 | 9.20 | 10.49 | 5146.09 | 8994.81 | 9476.99 | 133780/143771 MiB (93.1%) | ✅ |
| c1_i131072_o128 | main | 1 | 131072 | 128 | 9.02 | 5 | 0.55 | 41083.10 | 42.03 | 41125.13 | 1332.75 | 3042.40 | 3072.01 | 6.34 | 6.70 | 6.72 | 1801.84 | 3782.08 | 3832.09 | 133780/143771 MiB (93.1%) | ⚠️ |
| c4_i131072_o128 | main | 4 | 131072 | 128 | 24.36 | 20 | 0.82 | 60525.08 | 46.76 | 60571.83 | 2268.91 | 5665.45 | 7563.45 | 46.92 | 130.55 | 135.39 | 4786.17 | 9054.74 | 9263.16 | 134302/143771 MiB (93.4%) | ⚠️ |
| c1_i131072_o256 | main | 1 | 131072 | 256 | 4.85 | 5 | 1.03 | 76336.33 | 104.48 | 76440.81 | 324.28 | 526.13 | 531.79 | 6.38 | 6.74 | 6.75 | 969.07 | 1348.49 | 1365.14 | 134302/143771 MiB (93.4%) | ✅ |
| c4_i131072_o256 | main | 4 | 131072 | 256 | 8.32 | 20 | 2.40 | 177141.68 | 306.01 | 177447.69 | 542.06 | 826.11 | 873.84 | 7.11 | 8.42 | 8.69 | 1532.39 | 2573.00 | 2602.99 | 134302/143771 MiB (93.4%) | ✅ |
| c1_i131072_o512 | main | 1 | 131072 | 512 | 4.87 | 5 | 1.03 | 76048.83 | 104.09 | 76152.92 | 327.91 | 528.58 | 529.52 | 6.38 | 6.74 | 6.75 | 972.74 | 1353.22 | 1365.66 | 134302/143771 MiB (93.4%) | ✅ |
| c4_i131072_o512 | main | 4 | 131072 | 512 | 15.36 | 20 | 1.30 | 95969.05 | 382.40 | 96351.45 | 384.13 | 581.21 | 686.73 | 7.67 | 8.27 | 8.30 | 2674.27 | 3889.94 | 4365.16 | 134302/143771 MiB (93.4%) | ✅ |
| c1_i262144_o128 | main | 1 | 262144 | 128 | 21.96 | 5 | 0.23 | 34775.53 | 17.26 | 34792.78 | 3949.96 | 7547.16 | 7933.13 | 5.31 | 6.25 | 6.26 | 4389.70 | 8216.37 | 8667.71 | 134304/143771 MiB (93.4%) | ⚠️ |
| c2_i262144_o128 | main | 2 | 262144 | 128 | 17.30 | 10 | 0.58 | 80576.29 | 43.70 | 80619.99 | 1811.47 | 6405.29 | 6864.92 | 12.04 | 40.84 | 63.02 | 2911.30 | 7493.52 | 7624.83 | 134304/143771 MiB (93.4%) | ⚠️ |
| c1_i262144_o256 | main | 1 | 262144 | 256 | 5.49 | 5 | 0.91 | 139043.08 | 92.31 | 139135.39 | 483.52 | 741.06 | 802.19 | 5.45 | 6.50 | 6.55 | 1096.55 | 1644.68 | 1664.47 | 134304/143771 MiB (93.4%) | ✅ |
| c2_i262144_o256 | main | 2 | 262144 | 256 | 8.16 | 10 | 1.23 | 170921.30 | 171.18 | 171092.48 | 588.20 | 872.87 | 960.16 | 6.69 | 7.16 | 7.25 | 1528.35 | 2101.10 | 2191.39 | 134304/143771 MiB (93.4%) | ✅ |
| c1_i524288_o128 | main | 1 | 524288 | 128 | 21.47 | 5 | 0.23 | 47769.55 | 17.65 | 47787.20 | 3868.11 | 13982.40 | 16622.77 | 5.18 | 6.10 | 6.17 | 4292.93 | 14596.89 | 17210.26 | 134304/143771 MiB (93.4%) | ⚠️ |
| c2_i524288_o128 | main | 2 | 524288 | 128 | 5.67 | 10 | 1.76 | 292108.80 | 133.35 | 292242.15 | 645.47 | 1167.12 | 1232.75 | 5.64 | 6.90 | 6.94 | 1126.39 | 1682.93 | 1708.86 | 134306/143771 MiB (93.4%) | ✅ |
| c1_i1048576_o128 | main | 1 | 1048576 | 128 | 52.22 | 5 | 0.10 | 29684.37 | 7.26 | 29691.63 | 10048.05 | 38265.96 | 45702.81 | 4.67 | 6.04 | 6.08 | 10442.12 | 38536.13 | 45912.47 | 134310/143771 MiB (93.4%) | ⚠️ |
| c2_i1048576_o128 | main | 2 | 1048576 | 128 | 122.76 | 10 | 0.08 | 26303.08 | 6.16 | 26309.24 | 23067.17 | 59936.57 | 60383.98 | 15.73 | 56.69 | 68.50 | 24549.49 | 63918.14 | 66073.78 | 134312/143771 MiB (93.4%) | ⚠️ |
SLO: S2 tier — TTFT P95 < 3000ms, TPOT mean < 50ms. ✅ pass, ⚠️ partial, ❌ fail.

3500
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp8_dp1/results.json Normal file
View File

File diff suppressed because it is too large Load Diff

57
experiments/dsv4_h200_vllm_tp_dp_matrix/results/20260709-152419_matrix/tp8_dp1/scenarios.tsv Normal file
View File

@ -0,0 +1,57 @@
mark input_len output_len concurrency num_prompts
Y 1024 128 1 5
Y 1024 128 128 640
Y 1024 256 1 5
Y 1024 256 128 640
Y 1024 512 1 5
Y 1024 512 128 640
Y 1024 1024 1 5
Y 1024 1024 128 640
Y 1024 2048 1 5
Y 1024 2048 128 640
Y 1024 4096 1 5
Y 1024 4096 128 640
Y 4096 128 1 5
Y 4096 128 64 320
Y 4096 256 1 5
Y 4096 256 64 320
Y 4096 512 1 5
Y 4096 512 64 320
Y 4096 1024 1 5
Y 4096 1024 64 320
Y 4096 2048 1 5
Y 4096 2048 64 320
Y 4096 4096 1 5
Y 4096 4096 64 320
Y 16384 128 1 5
Y 16384 128 32 160
Y 16384 256 1 5
Y 16384 256 32 160
Y 16384 512 1 5
Y 16384 512 32 160
Y 16384 1024 1 5
Y 16384 1024 32 160
Y 16384 2048 1 5
Y 16384 2048 32 160
Y 65536 128 1 5
Y 65536 128 8 40
Y 65536 256 1 5
Y 65536 256 8 40
Y 65536 512 1 5
Y 65536 512 8 40
Y 65536 1024 1 5
Y 65536 1024 8 40
Y 131072 128 1 5
Y 131072 128 4 20
Y 131072 256 1 5
Y 131072 256 4 20
Y 131072 512 1 5
Y 131072 512 4 20
Y 262144 128 1 5
Y 262144 128 2 10
Y 262144 256 1 5
Y 262144 256 2 10
Y 524288 128 1 5
Y 524288 128 2 10
Y 1048576 128 1 5
Y 1048576 128 2 10
1 mark input_len output_len concurrency num_prompts
2 Y 1024 128 1 5
3 Y 1024 128 128 640
4 Y 1024 256 1 5
5 Y 1024 256 128 640
6 Y 1024 512 1 5
7 Y 1024 512 128 640
8 Y 1024 1024 1 5
9 Y 1024 1024 128 640
10 Y 1024 2048 1 5
11 Y 1024 2048 128 640
12 Y 1024 4096 1 5
13 Y 1024 4096 128 640
14 Y 4096 128 1 5
15 Y 4096 128 64 320
16 Y 4096 256 1 5
17 Y 4096 256 64 320
18 Y 4096 512 1 5
19 Y 4096 512 64 320
20 Y 4096 1024 1 5
21 Y 4096 1024 64 320
22 Y 4096 2048 1 5
23 Y 4096 2048 64 320
24 Y 4096 4096 1 5
25 Y 4096 4096 64 320
26 Y 16384 128 1 5
27 Y 16384 128 32 160
28 Y 16384 256 1 5
29 Y 16384 256 32 160
30 Y 16384 512 1 5
31 Y 16384 512 32 160
32 Y 16384 1024 1 5
33 Y 16384 1024 32 160
34 Y 16384 2048 1 5
35 Y 16384 2048 32 160
36 Y 65536 128 1 5
37 Y 65536 128 8 40
38 Y 65536 256 1 5
39 Y 65536 256 8 40
40 Y 65536 512 1 5
41 Y 65536 512 8 40
42 Y 65536 1024 1 5
43 Y 65536 1024 8 40
44 Y 131072 128 1 5
45 Y 131072 128 4 20
46 Y 131072 256 1 5
47 Y 131072 256 4 20
48 Y 131072 512 1 5
49 Y 131072 512 4 20
50 Y 262144 128 1 5
51 Y 262144 128 2 10
52 Y 262144 256 1 5
53 Y 262144 256 2 10
54 Y 524288 128 1 5
55 Y 524288 128 2 10
56 Y 1048576 128 1 5
57 Y 1048576 128 2 10