qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/bench_results/dspark_grid_20260707-132641/report.md
Benchmark Bot eb431d3925 Initial commit: benchmark scripts and final reports
2026-07-08 02:17:13 +00:00

13 KiB
Raw Blame History

vllm-dspark Benchmark Grid Report

  • Result root: /data/user1/yy/bench_results/dspark_grid_20260707-132641
  • Model: /data/models/DeepSeek-V4-Flash-DSpark
  • Backend: vllm-dspark (TP=8, FP8 KV cache, spec-method=dspark, spec-tokens=5)
  • Benchmark client: sglang.bench_serving --backend vllm

p1_quick

chat_standard (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 65.33 32 0.49 195.58 71.28 266.86 2040.65 8241.20 10715.53 1412.48 6903.30 8430.71 6.41 32.72 51.29 16.61
8 29.42 128 4.35 2130.85 569.52 2700.37 1819.35 8891.79 15165.63 327.62 379.39 4404.92 11.58 51.06 82.44 51.00
16 30.76 256 8.32 4144.89 1144.74 5289.63 1885.15 8902.10 10491.31 443.50 2779.49 8567.07 12.10 23.11 83.70 46.18
32 51.18 512 10.00 5017.88 1331.65 6349.53 3156.73 10438.92 12936.08 560.77 5750.71 7129.66 22.09 66.23 171.57 85.81

generation_standard (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 36.24 32 0.88 352.60 479.25 831.85 1131.63 1979.98 2576.42 66.57 141.80 177.56 1.94 2.85 2.88 8.50
8 27.43 128 4.67 2285.21 2362.35 4647.56 1649.16 3162.33 3844.70 80.88 157.80 160.50 3.23 5.31 7.09 14.77
16 35.39 256 7.23 3602.52 3654.79 7257.31 2133.90 4187.67 5500.20 88.78 169.39 199.78 4.37 6.97 11.81 19.28
32 48.16 512 10.63 5333.41 5371.85 10705.25 2908.89 5849.64 6938.03 112.84 211.64 405.78 5.84 9.30 12.47 25.89

summarization (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 33.63 32 0.95 4247.40 516.49 4763.90 1049.88 1783.33 1848.92 178.56 292.66 314.18 1.60 1.78 2.41 8.62
4 10.99 32 2.91 13000.06 1580.84 14580.90 1301.15 2238.30 2324.70 101.68 187.78 188.57 2.18 2.47 3.49 11.86
8 33.60 128 3.81 15369.23 1908.96 17278.19 2072.22 3978.11 4531.26 214.01 502.25 630.15 3.70 5.52 7.98 19.60
16 46.93 256 5.45 22698.72 2761.38 25460.10 2840.57 6243.39 7179.11 235.75 552.81 740.45 5.45 9.62 12.75 27.77

p2_core

chat_short (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 11.39 32 2.81 717.30 408.87 1126.18 355.00 685.00 815.55 48.57 105.64 105.97 2.07 3.10 3.69 8.51
8 10.62 128 12.06 3320.19 1591.90 4912.09 647.05 1401.33 1648.15 78.91 151.78 155.67 4.59 7.20 12.54 17.85
16 14.89 256 17.19 4533.71 2394.66 6928.36 902.72 1879.32 2211.23 95.95 165.09 217.14 6.19 10.26 15.12 23.65
32 24.90 512 20.56 5458.26 2723.85 8182.10 1508.89 3076.09 3881.52 161.04 292.00 383.41 10.80 18.19 25.45 41.86
64 21.27 512 24.07 6391.14 3189.39 9580.52 2547.81 5606.07 6543.97 268.68 543.21 588.63 18.24 30.39 39.57 70.29

chat_standard (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
64 19.87 512 25.76 12922.80 3429.47 16352.28 2380.94 4838.43 6315.98 291.17 794.65 947.26 16.87 30.96 42.55 68.46

generation_standard (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
64 43.59 512 11.75 5892.47 5934.93 11827.40 5136.14 10244.70 12575.20 209.70 536.96 606.58 10.33 17.41 21.20 45.47

long_context_probe (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 23.79 32 1.34 10684.78 366.30 11051.09 742.62 1237.28 1418.82 316.59 604.23 617.85 1.54 1.72 1.85 8.53
2 9.79 32 3.27 25971.67 890.38 26862.06 604.82 1055.84 1116.41 110.65 194.62 206.64 1.77 2.04 2.13 9.90
4 6.11 32 5.24 41616.05 1426.72 43042.77 741.94 1262.13 1447.68 127.38 222.70 235.35 2.26 2.69 3.69 12.39
8 36.07 128 3.55 28357.82 897.65 29255.46 2232.38 4946.97 5371.22 453.93 1089.99 1182.47 7.06 13.03 20.03 38.51
16 49.08 256 5.22 41910.25 1382.43 43292.68 3028.26 7651.28 9531.10 444.14 1212.92 1368.04 10.00 20.71 30.74 53.07

rag_medium (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 18.21 32 1.76 3568.86 494.63 4063.49 568.24 966.60 1201.99 110.94 151.11 154.78 1.63 1.95 2.50 8.60
4 6.31 32 5.07 10300.85 1427.65 11728.51 751.19 1259.07 1593.86 92.09 161.03 187.47 2.34 3.25 4.02 12.33
8 19.13 128 6.69 14147.80 1692.32 15840.12 1166.59 2199.67 2982.23 148.09 253.10 350.67 4.30 6.78 9.92 21.06
16 25.89 256 9.89 20561.53 2622.36 23183.89 1587.09 3316.74 4019.02 164.29 303.79 389.34 5.50 8.80 14.38 27.81
32 39.47 512 12.97 26523.95 3335.30 29859.24 2415.10 5069.76 6195.14 227.81 444.76 510.18 8.64 13.54 20.29 44.49

summarization (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
32 91.68 512 5.58 22445.97 2821.71 25267.67 5587.21 11177.97 12257.18 430.38 967.32 2238.28 10.56 16.13 23.53 54.40

p3_extension

decode_heavy (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 56.16 32 0.57 145.47 546.26 691.72 1754.17 2994.25 3175.70 52.30 107.66 108.83 1.88 2.68 3.14 8.53
8 54.33 128 2.36 648.72 2526.47 3175.19 3302.64 6408.14 7166.98 74.21 143.24 149.17 3.05 4.37 4.79 14.12
16 65.74 256 3.89 1026.90 4012.07 5038.96 3934.72 7216.03 9638.96 81.02 165.95 197.11 3.98 5.82 7.21 17.79
32 92.36 512 5.54 1471.70 5761.81 7233.51 5562.57 10498.47 12822.35 107.17 234.99 426.35 5.40 8.14 9.03 24.38

long_rag (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
1 30.84 32 1.04 15823.63 281.34 16104.98 962.70 1688.61 1788.29 549.72 1124.54 1163.96 1.49 1.72 1.82 8.52
2 11.49 32 2.78 42460.67 754.95 43215.62 708.16 1202.62 1253.12 192.57 352.96 409.32 2.05 2.37 6.62 10.76
4 7.09 32 4.51 68816.98 1223.56 70040.54 854.49 1516.80 1562.42 191.57 426.45 427.69 2.62 3.20 7.33 13.69
8 60.48 128 2.12 33604.85 536.85 34141.70 3732.80 8420.06 9705.36 930.98 2125.14 2742.48 11.12 22.70 39.17 61.41

stress_generation (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
96 81.20 768 9.46 4842.45 4635.63 9478.08 9772.15 20945.93 25314.28 326.63 782.50 795.30 20.35 33.33 42.50 88.91
128 92.26 1024 11.10 5587.83 5514.57 11102.40 11012.89 22012.44 28276.20 360.94 812.12 1024.93 22.53 35.59 42.09 99.75

stress_standard (20260707-132641)

Concurrency Duration(s) Success Req/s In tok/s Out tok/s Total tok/s Mean E2E(ms) P95 E2E(ms) P99 E2E(ms) Mean TTFT(ms) P95 TTFT(ms) P99 TTFT(ms) Mean TPOT(ms) P95 TPOT(ms) P99 TPOT(ms) Mean ITL(ms)
96 30.50 768 25.18 12891.05 3175.02 16066.07 3689.97 9383.09 11958.09 435.22 1225.11 2581.90 29.46 67.30 123.42 113.64
128 36.86 1024 27.78 13985.42 3479.38 17464.80 4417.43 11825.67 14850.43 456.55 1032.19 1922.87 35.20 76.85 141.17 136.57