# SGLang vs vLLM vs DSpark long-context matrix (8x NVIDIA H200 143GB)

## Summary

- Model: `/data/models/DeepSeek-V4-Flash`
- DSpark model: `/data/models/DeepSeek-V4-Flash-DSpark`
- Hardware: 8x NVIDIA H200 143GB
- Benchmark client: `sglang.bench_serving`
- DSpark flags: `--spec-method dspark --spec-model <model> --spec-tokens 5`
- SLO reference: TTFT P95 < 3000.0ms, TPOT mean < 50.0ms

## Side-by-side results

| Scenario | Backend | Conc | Input | Output | Req/s | OutTok/s | TTFT P95(ms) | TTFT P99(ms) | TPOT Mean(ms) | TPOT P95(ms) | TPOT P99(ms) | E2E P99(ms) | SLO |
|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| c10_i131072_o1024 | sglang | 10 | 131072 | 1024 | 0.68 | 319.74 | 4147.51 | 7409.82 | 25.69 | 49.89 | 58.92 | 26432.66 | ⚠️ |
| c10_i131072_o1024 | vllm | 10 | 131072 | 1024 | 0.39 | 183.08 | 6596.15 | 6603.12 | 47.10 | 77.53 | 84.06 | 48426.64 | ⚠️ |
| c10_i131072_o1024 | dspark | 10 | 131072 | 1024 | 0.66 | 310.66 | 11615.81 | 11676.24 | 24.58 | 46.24 | 46.56 | 29271.39 | ⚠️ |
| c10_i131072_o256 | sglang | 10 | 131072 | 256 | 0.34 | 42.70 | 26046.62 | 26064.38 | 1374.50 | 1810.56 | 19553.62 | 58326.41 | ❌ |
| c10_i131072_o256 | vllm | 10 | 131072 | 256 | 0.38 | 48.09 | 23250.03 | 24238.33 | 150.22 | 299.91 | 301.72 | 51439.08 | ❌ |
| c10_i131072_o256 | dspark | 10 | 131072 | 256 | 0.40 | 51.33 | 24693.20 | 27039.11 | 59.88 | 78.22 | 316.95 | 36603.69 | ❌ |
| c10_i131072_o4096 | sglang | 10 | 131072 | 4096 | 0.27 | 517.96 | 6177.42 | 6920.28 | 14.21 | 19.92 | 25.87 | 54286.63 | ⚠️ |
| c10_i131072_o4096 | vllm | 10 | 131072 | 4096 | 0.23 | 445.81 | 2096.29 | 2168.81 | 18.57 | 21.47 | 21.61 | 67041.13 | ✅ |
| c10_i131072_o4096 | dspark | 10 | 131072 | 4096 | 1.05 | 1998.23 | 1543.09 | 1546.01 | 3.34 | 4.81 | 4.84 | 14731.59 | ✅ |
| c10_i262144_o1024 | sglang | 10 | 262144 | 1024 | 0.16 | 76.30 | 39446.96 | 43965.67 | 108.42 | 202.43 | 202.90 | 120180.87 | ❌ |
| c10_i262144_o1024 | vllm | 10 | 262144 | 1024 | 0.18 | 84.63 | 34130.22 | 38144.40 | 98.87 | 142.57 | 266.08 | 108658.89 | ❌ |
| c10_i262144_o1024 | dspark | 10 | 262144 | 1024 | 0.20 | 96.79 | 39321.30 | 40783.01 | 48.19 | 67.60 | 68.13 | 91123.07 | ⚠️ |
| c10_i262144_o256 | sglang | 10 | 262144 | 256 | 0.15 | 18.70 | 58074.31 | 62652.59 | 2953.70 | 3625.59 | 41185.32 | 134628.33 | ❌ |
| c10_i262144_o256 | vllm | 10 | 262144 | 256 | 0.18 | 22.76 | 50632.44 | 51883.02 | 229.20 | 339.27 | 344.95 | 110216.92 | ❌ |
| c10_i262144_o256 | dspark | 10 | 262144 | 256 | 0.18 | 23.12 | 64654.53 | 67877.62 | 52.05 | 73.84 | 151.28 | 80196.26 | ❌ |
| c10_i262144_o4096 | sglang | 10 | 262144 | 4096 | 0.14 | 272.46 | 18088.12 | 22579.09 | 28.00 | 43.87 | 45.07 | 120183.75 | ⚠️ |
| c10_i262144_o4096 | vllm | 10 | 262144 | 4096 | 0.11 | 215.86 | 34100.32 | 38022.94 | 37.18 | 56.07 | 62.00 | 157944.21 | ⚠️ |
| c10_i262144_o4096 | dspark | 10 | 262144 | 4096 | 0.18 | 349.94 | 33426.83 | 37512.71 | 20.67 | 35.94 | 44.44 | 103440.52 | ⚠️ |
| c25_i65536_o1024 | sglang | 25 | 65536 | 1024 | 0.67 | 346.70 | 19996.43 | 24478.11 | 149.27 | 395.00 | 2037.09 | 69586.94 | ❌ |
| c25_i65536_o1024 | vllm | 25 | 65536 | 1024 | 0.64 | 332.61 | 24861.20 | 28942.75 | 72.36 | 196.24 | 271.32 | 72689.84 | ❌ |
| c25_i65536_o1024 | dspark | 25 | 65536 | 1024 | 0.80 | 412.74 | 26078.12 | 29135.60 | 42.82 | 56.19 | 133.89 | 60995.55 | ⚠️ |
| c25_i65536_o256 | sglang | 25 | 65536 | 256 | 0.71 | 84.45 | 27219.06 | 31612.70 | 292.84 | 801.18 | 938.43 | 68854.26 | ❌ |
| c25_i65536_o256 | vllm | 25 | 65536 | 256 | 0.81 | 95.60 | 24902.72 | 28974.45 | 180.78 | 273.59 | 278.16 | 60781.45 | ❌ |
| c25_i65536_o256 | dspark | 25 | 65536 | 256 | 0.84 | 98.79 | 27635.11 | 29212.48 | 51.68 | 65.52 | 134.94 | 39437.04 | ❌ |
| c25_i65536_o4096 | sglang | 25 | 65536 | 4096 | 0.41 | 950.01 | 18559.58 | 23249.71 | 23.78 | 42.50 | 82.84 | 96670.12 | ⚠️ |
| c25_i65536_o4096 | vllm | 25 | 65536 | 4096 | 0.32 | 741.71 | 24847.97 | 28840.96 | 28.80 | 46.66 | 63.40 | 129654.04 | ⚠️ |
| c25_i65536_o4096 | dspark | 25 | 65536 | 4096 | 0.61 | 1409.20 | 25048.60 | 29143.30 | 14.72 | 30.57 | 42.07 | 73496.43 | ⚠️ |
| c2_i131072_o1024 | sglang | 2 | 131072 | 1024 | 0.60 | 136.10 | 479.99 | 482.72 | 8.56 | 9.72 | 9.79 | 5480.98 | ✅ |
| c2_i131072_o1024 | vllm | 2 | 131072 | 1024 | 0.70 | 158.36 | 676.14 | 695.90 | 6.49 | 8.20 | 8.34 | 4803.24 | ✅ |
| c2_i131072_o1024 | dspark | 2 | 131072 | 1024 | 2.29 | 514.96 | 679.81 | 691.56 | 1.10 | 1.60 | 1.63 | 1450.74 | ✅ |
| c2_i131072_o256 | sglang | 2 | 131072 | 256 | 1.43 | 139.48 | 589.14 | 605.33 | 8.29 | 8.96 | 9.05 | 1750.12 | ✅ |
| c2_i131072_o256 | vllm | 2 | 131072 | 256 | 1.51 | 147.12 | 684.63 | 700.28 | 7.08 | 9.27 | 9.45 | 1556.46 | ✅ |
| c2_i131072_o256 | dspark | 2 | 131072 | 256 | 3.10 | 301.11 | 617.17 | 634.08 | 1.00 | 1.41 | 1.42 | 780.85 | ✅ |
| c2_i131072_o4096 | sglang | 2 | 131072 | 4096 | 0.11 | 244.38 | 584.43 | 599.22 | 7.53 | 7.61 | 7.62 | 28360.35 | ✅ |
| c2_i131072_o4096 | vllm | 2 | 131072 | 4096 | 0.12 | 262.22 | 671.34 | 688.15 | 7.09 | 7.17 | 7.18 | 26496.04 | ✅ |
| c2_i131072_o4096 | dspark | 2 | 131072 | 4096 | 0.47 | 1076.96 | 540.43 | 541.94 | 1.76 | 2.03 | 2.08 | 6313.36 | ✅ |
| c2_i262144_o1024 | sglang | 2 | 262144 | 1024 | 0.51 | 114.94 | 1007.07 | 1024.10 | 8.36 | 9.03 | 9.06 | 6240.65 | ✅ |
| c2_i262144_o1024 | vllm | 2 | 262144 | 1024 | 0.19 | 43.14 | 8722.76 | 9009.70 | 20.17 | 52.24 | 57.91 | 13826.13 | ⚠️ |
| c2_i262144_o1024 | dspark | 2 | 262144 | 1024 | 0.25 | 55.65 | 10466.90 | 10984.16 | 14.36 | 43.30 | 48.52 | 15361.85 | ⚠️ |
| c2_i262144_o256 | sglang | 2 | 262144 | 256 | 0.29 | 28.44 | 9538.91 | 9980.78 | 14.10 | 26.98 | 29.16 | 11941.34 | ⚠️ |
| c2_i262144_o256 | vllm | 2 | 262144 | 256 | 0.23 | 22.74 | 8625.18 | 8952.31 | 10.18 | 23.01 | 25.44 | 10378.60 | ⚠️ |
| c2_i262144_o256 | dspark | 2 | 262144 | 256 | 0.26 | 25.25 | 10459.44 | 10978.82 | 17.58 | 45.28 | 48.94 | 14620.43 | ⚠️ |
| c2_i262144_o4096 | sglang | 2 | 262144 | 4096 | 0.08 | 189.78 | 9453.57 | 9896.38 | 7.82 | 8.48 | 8.60 | 38723.14 | ⚠️ |
| c2_i262144_o4096 | vllm | 2 | 262144 | 4096 | 0.08 | 180.76 | 8759.63 | 9062.80 | 8.63 | 11.53 | 12.01 | 35881.75 | ⚠️ |
| c2_i262144_o4096 | dspark | 2 | 262144 | 4096 | 0.17 | 384.42 | 8709.00 | 8987.60 | 3.23 | 6.44 | 6.99 | 15113.78 | ⚠️ |
| c5_i131072_o1024 | sglang | 5 | 131072 | 1024 | 0.87 | 321.54 | 1231.23 | 1298.36 | 9.03 | 10.47 | 10.71 | 7298.25 | ✅ |
| c5_i131072_o1024 | vllm | 5 | 131072 | 1024 | 0.90 | 331.65 | 856.18 | 917.80 | 8.93 | 10.48 | 10.62 | 6989.27 | ✅ |
| c5_i131072_o1024 | dspark | 5 | 131072 | 1024 | 2.75 | 1019.06 | 1080.05 | 1176.83 | 2.53 | 3.98 | 4.18 | 2383.07 | ✅ |
| c5_i131072_o256 | sglang | 5 | 131072 | 256 | 1.05 | 146.97 | 3270.23 | 4685.65 | 19.91 | 40.81 | 46.10 | 7815.89 | ⚠️ |
| c5_i131072_o256 | vllm | 5 | 131072 | 256 | 0.39 | 54.07 | 9384.23 | 9677.91 | 51.54 | 107.16 | 117.25 | 19860.69 | ❌ |
| c5_i131072_o256 | dspark | 5 | 131072 | 256 | 0.44 | 60.98 | 10128.32 | 10501.28 | 37.60 | 60.86 | 61.78 | 17256.59 | ⚠️ |
| c5_i131072_o4096 | sglang | 5 | 131072 | 4096 | 0.20 | 378.76 | 4667.56 | 6275.51 | 11.98 | 22.07 | 23.60 | 31176.81 | ⚠️ |
| c5_i131072_o4096 | vllm | 5 | 131072 | 4096 | 0.23 | 447.85 | 1054.01 | 1104.66 | 8.48 | 9.13 | 9.14 | 28818.20 | ✅ |
| c5_i131072_o4096 | dspark | 5 | 131072 | 4096 | 0.70 | 1335.09 | 1156.91 | 1249.38 | 3.20 | 4.19 | 4.21 | 11315.66 | ✅ |
| c5_i262144_o1024 | sglang | 5 | 262144 | 1024 | 0.15 | 56.42 | 20522.02 | 25944.29 | 54.11 | 118.98 | 124.02 | 53444.22 | ❌ |
| c5_i262144_o1024 | vllm | 5 | 262144 | 1024 | 0.16 | 60.32 | 22564.89 | 26639.92 | 47.49 | 100.69 | 109.86 | 53790.59 | ⚠️ |
| c5_i262144_o1024 | dspark | 5 | 262144 | 1024 | 0.19 | 69.32 | 22995.12 | 26781.97 | 38.56 | 57.19 | 57.43 | 40847.11 | ⚠️ |
| c5_i262144_o256 | sglang | 5 | 262144 | 256 | 0.16 | 21.94 | 25875.40 | 30596.95 | 97.05 | 231.75 | 252.98 | 43756.53 | ❌ |
| c5_i262144_o256 | vllm | 5 | 262144 | 256 | 0.19 | 25.85 | 21986.99 | 26381.86 | 123.42 | 202.78 | 215.56 | 45761.90 | ❌ |
| c5_i262144_o256 | dspark | 5 | 262144 | 256 | 0.19 | 26.70 | 29425.38 | 30576.66 | 41.74 | 60.42 | 62.17 | 37097.84 | ⚠️ |
| c5_i262144_o4096 | sglang | 5 | 262144 | 4096 | 0.11 | 209.50 | 16539.50 | 20323.40 | 25.05 | 56.69 | 60.22 | 60383.57 | ⚠️ |
| c5_i262144_o4096 | vllm | 5 | 262144 | 4096 | 0.11 | 205.12 | 22171.78 | 26533.69 | 25.69 | 63.32 | 68.33 | 64086.27 | ⚠️ |
| c5_i262144_o4096 | dspark | 5 | 262144 | 4096 | 0.16 | 299.79 | 22250.95 | 26667.05 | 17.46 | 45.41 | 49.74 | 51820.15 | ⚠️ |

## Notes

- SLO check uses TTFT P95 and TPOT mean.
- A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.