sskj/experiments/dsv4_h200_sglang_vs_vllm/results/20260708-083346/comparison.md

SGLang vs vLLM on DeepSeek-V4-Flash (H200, TP=8)

Summary

• Model: /data/models/DeepSeek-V4-Flash
• Hardware: 8x NVIDIA H200 143GB
• Tensor Parallelism: 8
• Benchmark client: sglang.bench_serving
• SLO reference: S2 tier (TTFT P95 < 3s, TPOT < 50ms)

Side-by-side results

Scenario	Backend	Conc	Input	Output	Req/s	OutTok/s	TTFT P95(ms)	TTFT P99(ms)	TPOT Mean(ms)	TPOT P95(ms)	TPOT P99(ms)	E2E P99(ms)	SLO
c128_i512_o256	sglang	128	512	256	27.96	3691.88	1046.81	1048.77	19.84	30.70	78.39	4494.67	✅
c128_i512_o256	vllm	128	512	256	1.82	240.43	65775.64	68346.91	7.51	8.88	9.95	69371.23	⚠️
c1_i131072_o1024	sglang	1	131072	1024	0.21	117.45	610.97	623.90	7.64	7.71	7.72	7216.79	✅
c1_i131072_o1024	vllm	1	131072	1024	0.24	133.99	549.11	565.06	6.79	6.84	6.84	6410.99	✅
c1_i200000_o1024	sglang	1	200000	1024	0.25	109.00	619.26	619.26	7.72	7.72	7.72	3976.45	✅
c1_i200000_o1024	vllm	1	200000	1024	0.28	121.52	608.85	608.85	6.84	6.84	6.84	3584.28	✅
c1_i32768_o1024	sglang	1	32768	1024	0.32	107.76	1088.23	1185.18	7.55	7.58	7.58	4928.38	✅
c1_i32768_o1024	vllm	1	32768	1024	0.36	120.28	1007.75	1099.08	6.70	6.76	6.76	4391.50	✅
c1_i32768_o256	sglang	1	32768	256	0.48	122.41	225.25	228.79	7.43	7.44	7.44	2126.29	✅
c1_i32768_o256	vllm	1	32768	256	0.53	135.36	319.19	332.05	6.66	6.66	6.67	2026.78	✅
c1_i4000_o512	sglang	1	4000	512	0.43	121.90	245.14	250.72	7.50	7.58	7.59	4062.79	✅
c1_i4000_o512	vllm	1	4000	512	0.49	138.92	195.78	198.56	6.68	6.73	6.73	3604.09	✅
c1_i512_o256	sglang	1	512	256	0.77	112.74	228.17	229.05	7.45	7.47	7.62	2105.10	✅
c1_i512_o256	vllm	1	512	256	0.92	133.24	188.08	196.56	6.67	6.71	6.73	1877.61	✅
c1_i65536_o1024	sglang	1	65536	1024	0.43	97.35	1391.18	1526.74	7.56	7.62	7.63	5333.41	✅
c1_i65536_o1024	vllm	1	65536	1024	0.50	113.04	1199.14	1320.28	6.11	6.66	6.67	4685.90	✅
c32_i32768_o256	sglang	32	32768	256	9.11	2331.04	894.53	897.13	11.22	11.79	12.38	3706.48	✅
c32_i32768_o256	vllm	32	32768	256	1.03	263.21	29346.73	29758.65	7.18	7.46	7.48	31587.45	⚠️
c32_i4000_o512	sglang	32	4000	512	4.60	1298.23	647.63	650.43	19.83	25.78	27.20	11422.18	✅
c32_i4000_o512	vllm	32	4000	512	0.93	263.43	33396.12	33949.08	7.20	7.61	8.33	36115.49	⚠️
c32_i512_o256	sglang	32	512	256	5.71	753.27	448.83	573.80	38.34	53.13	76.13	11783.40	✅
c32_i512_o256	vllm	32	512	256	1.81	239.27	17481.79	18082.76	7.54	9.09	9.97	19259.08	⚠️

Notes

• SLO check uses TTFT P95 and TPOT mean (the same criteria as docs/SLO_STANDARDS.md).
• A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.