qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/experiments/dsv4_h200_long_context_matrix/results/20260709-020227/comparison.md
yy-fighting 7e614fb702 Add DSpark backend to long-context matrix experiment 
- Add start_dspark.sh with --spec-method dspark --spec-tokens 5
- Add BACKENDS env var to run only selected backends
- Fix client/backend mapping for DSpark (bench client uses vllm, parser uses vllm)
- Add three-way compare.py for sglang/vllm/dspark
- Reuse existing sglang/vllm results from 20260708-100016
- Add DSpark long-context results and comparison
2026-07-09 02:36:57 +00:00

8.6 KiB
Raw Blame History

SGLang vs vLLM vs DSpark long-context matrix (8x NVIDIA H200 143GB)

Summary

  • Model: /data/models/DeepSeek-V4-Flash
  • DSpark model: /data/models/DeepSeek-V4-Flash-DSpark
  • Hardware: 8x NVIDIA H200 143GB
  • Benchmark client: sglang.bench_serving
  • DSpark flags: --spec-method dspark --spec-model <model> --spec-tokens 5
  • SLO reference: TTFT P95 < 3000.0ms, TPOT mean < 50.0ms

Side-by-side results

Scenario Backend Conc Input Output Req/s OutTok/s TTFT P95(ms) TTFT P99(ms) TPOT Mean(ms) TPOT P95(ms) TPOT P99(ms) E2E P99(ms) SLO
c10_i131072_o1024 sglang 10 131072 1024 0.68 319.74 4147.51 7409.82 25.69 49.89 58.92 26432.66 ⚠️
c10_i131072_o1024 vllm 10 131072 1024 0.39 183.08 6596.15 6603.12 47.10 77.53 84.06 48426.64 ⚠️
c10_i131072_o1024 dspark 10 131072 1024 0.66 310.66 11615.81 11676.24 24.58 46.24 46.56 29271.39 ⚠️
c10_i131072_o256 sglang 10 131072 256 0.34 42.70 26046.62 26064.38 1374.50 1810.56 19553.62 58326.41 ❌
c10_i131072_o256 vllm 10 131072 256 0.38 48.09 23250.03 24238.33 150.22 299.91 301.72 51439.08 ❌
c10_i131072_o256 dspark 10 131072 256 0.40 51.33 24693.20 27039.11 59.88 78.22 316.95 36603.69 ❌
c10_i131072_o4096 sglang 10 131072 4096 0.27 517.96 6177.42 6920.28 14.21 19.92 25.87 54286.63 ⚠️
c10_i131072_o4096 vllm 10 131072 4096 0.23 445.81 2096.29 2168.81 18.57 21.47 21.61 67041.13 ✅
c10_i131072_o4096 dspark 10 131072 4096 1.05 1998.23 1543.09 1546.01 3.34 4.81 4.84 14731.59 ✅
c10_i262144_o1024 sglang 10 262144 1024 0.16 76.30 39446.96 43965.67 108.42 202.43 202.90 120180.87 ❌
c10_i262144_o1024 vllm 10 262144 1024 0.18 84.63 34130.22 38144.40 98.87 142.57 266.08 108658.89 ❌
c10_i262144_o1024 dspark 10 262144 1024 0.20 96.79 39321.30 40783.01 48.19 67.60 68.13 91123.07 ⚠️
c10_i262144_o256 sglang 10 262144 256 0.15 18.70 58074.31 62652.59 2953.70 3625.59 41185.32 134628.33 ❌
c10_i262144_o256 vllm 10 262144 256 0.18 22.76 50632.44 51883.02 229.20 339.27 344.95 110216.92 ❌
c10_i262144_o256 dspark 10 262144 256 0.18 23.12 64654.53 67877.62 52.05 73.84 151.28 80196.26 ❌
c10_i262144_o4096 sglang 10 262144 4096 0.14 272.46 18088.12 22579.09 28.00 43.87 45.07 120183.75 ⚠️
c10_i262144_o4096 vllm 10 262144 4096 0.11 215.86 34100.32 38022.94 37.18 56.07 62.00 157944.21 ⚠️
c10_i262144_o4096 dspark 10 262144 4096 0.18 349.94 33426.83 37512.71 20.67 35.94 44.44 103440.52 ⚠️
c25_i65536_o1024 sglang 25 65536 1024 0.67 346.70 19996.43 24478.11 149.27 395.00 2037.09 69586.94 ❌
c25_i65536_o1024 vllm 25 65536 1024 0.64 332.61 24861.20 28942.75 72.36 196.24 271.32 72689.84 ❌
c25_i65536_o1024 dspark 25 65536 1024 0.80 412.74 26078.12 29135.60 42.82 56.19 133.89 60995.55 ⚠️
c25_i65536_o256 sglang 25 65536 256 0.71 84.45 27219.06 31612.70 292.84 801.18 938.43 68854.26 ❌
c25_i65536_o256 vllm 25 65536 256 0.81 95.60 24902.72 28974.45 180.78 273.59 278.16 60781.45 ❌
c25_i65536_o256 dspark 25 65536 256 0.84 98.79 27635.11 29212.48 51.68 65.52 134.94 39437.04 ❌
c25_i65536_o4096 sglang 25 65536 4096 0.41 950.01 18559.58 23249.71 23.78 42.50 82.84 96670.12 ⚠️
c25_i65536_o4096 vllm 25 65536 4096 0.32 741.71 24847.97 28840.96 28.80 46.66 63.40 129654.04 ⚠️
c25_i65536_o4096 dspark 25 65536 4096 0.61 1409.20 25048.60 29143.30 14.72 30.57 42.07 73496.43 ⚠️
c2_i131072_o1024 sglang 2 131072 1024 0.60 136.10 479.99 482.72 8.56 9.72 9.79 5480.98 ✅
c2_i131072_o1024 vllm 2 131072 1024 0.70 158.36 676.14 695.90 6.49 8.20 8.34 4803.24 ✅
c2_i131072_o1024 dspark 2 131072 1024 2.29 514.96 679.81 691.56 1.10 1.60 1.63 1450.74 ✅
c2_i131072_o256 sglang 2 131072 256 1.43 139.48 589.14 605.33 8.29 8.96 9.05 1750.12 ✅
c2_i131072_o256 vllm 2 131072 256 1.51 147.12 684.63 700.28 7.08 9.27 9.45 1556.46 ✅
c2_i131072_o256 dspark 2 131072 256 3.10 301.11 617.17 634.08 1.00 1.41 1.42 780.85 ✅
c2_i131072_o4096 sglang 2 131072 4096 0.11 244.38 584.43 599.22 7.53 7.61 7.62 28360.35 ✅
c2_i131072_o4096 vllm 2 131072 4096 0.12 262.22 671.34 688.15 7.09 7.17 7.18 26496.04 ✅
c2_i131072_o4096 dspark 2 131072 4096 0.47 1076.96 540.43 541.94 1.76 2.03 2.08 6313.36 ✅
c2_i262144_o1024 sglang 2 262144 1024 0.51 114.94 1007.07 1024.10 8.36 9.03 9.06 6240.65 ✅
c2_i262144_o1024 vllm 2 262144 1024 0.19 43.14 8722.76 9009.70 20.17 52.24 57.91 13826.13 ⚠️
c2_i262144_o1024 dspark 2 262144 1024 0.25 55.65 10466.90 10984.16 14.36 43.30 48.52 15361.85 ⚠️
c2_i262144_o256 sglang 2 262144 256 0.29 28.44 9538.91 9980.78 14.10 26.98 29.16 11941.34 ⚠️
c2_i262144_o256 vllm 2 262144 256 0.23 22.74 8625.18 8952.31 10.18 23.01 25.44 10378.60 ⚠️
c2_i262144_o256 dspark 2 262144 256 0.26 25.25 10459.44 10978.82 17.58 45.28 48.94 14620.43 ⚠️
c2_i262144_o4096 sglang 2 262144 4096 0.08 189.78 9453.57 9896.38 7.82 8.48 8.60 38723.14 ⚠️
c2_i262144_o4096 vllm 2 262144 4096 0.08 180.76 8759.63 9062.80 8.63 11.53 12.01 35881.75 ⚠️
c2_i262144_o4096 dspark 2 262144 4096 0.17 384.42 8709.00 8987.60 3.23 6.44 6.99 15113.78 ⚠️
c5_i131072_o1024 sglang 5 131072 1024 0.87 321.54 1231.23 1298.36 9.03 10.47 10.71 7298.25 ✅
c5_i131072_o1024 vllm 5 131072 1024 0.90 331.65 856.18 917.80 8.93 10.48 10.62 6989.27 ✅
c5_i131072_o1024 dspark 5 131072 1024 2.75 1019.06 1080.05 1176.83 2.53 3.98 4.18 2383.07 ✅
c5_i131072_o256 sglang 5 131072 256 1.05 146.97 3270.23 4685.65 19.91 40.81 46.10 7815.89 ⚠️
c5_i131072_o256 vllm 5 131072 256 0.39 54.07 9384.23 9677.91 51.54 107.16 117.25 19860.69 ❌
c5_i131072_o256 dspark 5 131072 256 0.44 60.98 10128.32 10501.28 37.60 60.86 61.78 17256.59 ⚠️
c5_i131072_o4096 sglang 5 131072 4096 0.20 378.76 4667.56 6275.51 11.98 22.07 23.60 31176.81 ⚠️
c5_i131072_o4096 vllm 5 131072 4096 0.23 447.85 1054.01 1104.66 8.48 9.13 9.14 28818.20 ✅
c5_i131072_o4096 dspark 5 131072 4096 0.70 1335.09 1156.91 1249.38 3.20 4.19 4.21 11315.66 ✅
c5_i262144_o1024 sglang 5 262144 1024 0.15 56.42 20522.02 25944.29 54.11 118.98 124.02 53444.22 ❌
c5_i262144_o1024 vllm 5 262144 1024 0.16 60.32 22564.89 26639.92 47.49 100.69 109.86 53790.59 ⚠️
c5_i262144_o1024 dspark 5 262144 1024 0.19 69.32 22995.12 26781.97 38.56 57.19 57.43 40847.11 ⚠️
c5_i262144_o256 sglang 5 262144 256 0.16 21.94 25875.40 30596.95 97.05 231.75 252.98 43756.53 ❌
c5_i262144_o256 vllm 5 262144 256 0.19 25.85 21986.99 26381.86 123.42 202.78 215.56 45761.90 ❌
c5_i262144_o256 dspark 5 262144 256 0.19 26.70 29425.38 30576.66 41.74 60.42 62.17 37097.84 ⚠️
c5_i262144_o4096 sglang 5 262144 4096 0.11 209.50 16539.50 20323.40 25.05 56.69 60.22 60383.57 ⚠️
c5_i262144_o4096 vllm 5 262144 4096 0.11 205.12 22171.78 26533.69 25.69 63.32 68.33 64086.27 ⚠️
c5_i262144_o4096 dspark 5 262144 4096 0.16 299.79 22250.95 26667.05 17.46 45.41 49.74 51820.15 ⚠️

Notes

  • SLO check uses TTFT P95 and TPOT mean.
  • A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.