sskj/experiments/dsv4_h200_long_context_matrix/results/20260708-100016/comparison.md

SGLang vs vLLM on DeepSeek-V4-Flash (H200, TP=8)

Summary

• Model: /data/models/DeepSeek-V4-Flash
• Hardware: 8x NVIDIA H200 143GB
• Tensor Parallelism: 8
• Benchmark client: sglang.bench_serving
• SLO reference: S2 tier (TTFT P95 < 3s, TPOT < 50ms)

Side-by-side results

Scenario	Backend	Conc	Input	Output	Req/s	OutTok/s	TTFT P95(ms)	TTFT P99(ms)	TPOT Mean(ms)	TPOT P95(ms)	TPOT P99(ms)	E2E P99(ms)	SLO
c10_i131072_o1024	sglang	10	131072	1024	0.68	319.74	4147.51	7409.82	25.69	49.89	58.92	26432.66	⚠️
c10_i131072_o1024	vllm	10	131072	1024	0.39	183.08	6596.15	6603.12	47.10	77.53	84.06	48426.64	⚠️
c10_i131072_o256	sglang	10	131072	256	0.34	42.70	26046.62	26064.38	1374.50	1810.56	19553.62	58326.41	❌
c10_i131072_o256	vllm	10	131072	256	0.38	48.09	23250.03	24238.33	150.22	299.91	301.72	51439.08	❌
c10_i131072_o4096	sglang	10	131072	4096	0.27	517.96	6177.42	6920.28	14.21	19.92	25.87	54286.63	⚠️
c10_i131072_o4096	vllm	10	131072	4096	0.23	445.81	2096.29	2168.81	18.57	21.47	21.61	67041.13	✅
c10_i262144_o1024	sglang	10	262144	1024	0.16	76.30	39446.96	43965.67	108.42	202.43	202.90	120180.87	❌
c10_i262144_o1024	vllm	10	262144	1024	0.18	84.63	34130.22	38144.40	98.87	142.57	266.08	108658.89	❌
c10_i262144_o256	sglang	10	262144	256	0.15	18.70	58074.31	62652.59	2953.70	3625.59	41185.32	134628.33	❌
c10_i262144_o256	vllm	10	262144	256	0.18	22.76	50632.44	51883.02	229.20	339.27	344.95	110216.92	❌
c10_i262144_o4096	sglang	10	262144	4096	0.14	272.46	18088.12	22579.09	28.00	43.87	45.07	120183.75	⚠️
c10_i262144_o4096	vllm	10	262144	4096	0.11	215.86	34100.32	38022.94	37.18	56.07	62.00	157944.21	⚠️
c25_i65536_o1024	sglang	25	65536	1024	0.67	346.70	19996.43	24478.11	149.27	395.00	2037.09	69586.94	❌
c25_i65536_o1024	vllm	25	65536	1024	0.64	332.61	24861.20	28942.75	72.36	196.24	271.32	72689.84	❌
c25_i65536_o256	sglang	25	65536	256	0.71	84.45	27219.06	31612.70	292.84	801.18	938.43	68854.26	❌
c25_i65536_o256	vllm	25	65536	256	0.81	95.60	24902.72	28974.45	180.78	273.59	278.16	60781.45	❌
c25_i65536_o4096	sglang	25	65536	4096	0.41	950.01	18559.58	23249.71	23.78	42.50	82.84	96670.12	⚠️
c25_i65536_o4096	vllm	25	65536	4096	0.32	741.71	24847.97	28840.96	28.80	46.66	63.40	129654.04	⚠️
c2_i131072_o1024	sglang	2	131072	1024	0.60	136.10	479.99	482.72	8.56	9.72	9.79	5480.98	✅
c2_i131072_o1024	vllm	2	131072	1024	0.70	158.36	676.14	695.90	6.49	8.20	8.34	4803.24	✅
c2_i131072_o256	sglang	2	131072	256	1.43	139.48	589.14	605.33	8.29	8.96	9.05	1750.12	✅
c2_i131072_o256	vllm	2	131072	256	1.51	147.12	684.63	700.28	7.08	9.27	9.45	1556.46	✅
c2_i131072_o4096	sglang	2	131072	4096	0.11	244.38	584.43	599.22	7.53	7.61	7.62	28360.35	✅
c2_i131072_o4096	vllm	2	131072	4096	0.12	262.22	671.34	688.15	7.09	7.17	7.18	26496.04	✅
c2_i262144_o1024	sglang	2	262144	1024	0.51	114.94	1007.07	1024.10	8.36	9.03	9.06	6240.65	✅
c2_i262144_o1024	vllm	2	262144	1024	0.19	43.14	8722.76	9009.70	20.17	52.24	57.91	13826.13	⚠️
c2_i262144_o256	sglang	2	262144	256	0.29	28.44	9538.91	9980.78	14.10	26.98	29.16	11941.34	⚠️
c2_i262144_o256	vllm	2	262144	256	0.23	22.74	8625.18	8952.31	10.18	23.01	25.44	10378.60	⚠️
c2_i262144_o4096	sglang	2	262144	4096	0.08	189.78	9453.57	9896.38	7.82	8.48	8.60	38723.14	⚠️
c2_i262144_o4096	vllm	2	262144	4096	0.08	180.76	8759.63	9062.80	8.63	11.53	12.01	35881.75	⚠️
c5_i131072_o1024	sglang	5	131072	1024	0.87	321.54	1231.23	1298.36	9.03	10.47	10.71	7298.25	✅
c5_i131072_o1024	vllm	5	131072	1024	0.90	331.65	856.18	917.80	8.93	10.48	10.62	6989.27	✅
c5_i131072_o256	sglang	5	131072	256	1.05	146.97	3270.23	4685.65	19.91	40.81	46.10	7815.89	⚠️
c5_i131072_o256	vllm	5	131072	256	0.39	54.07	9384.23	9677.91	51.54	107.16	117.25	19860.69	❌
c5_i131072_o4096	sglang	5	131072	4096	0.20	378.76	4667.56	6275.51	11.98	22.07	23.60	31176.81	⚠️
c5_i131072_o4096	vllm	5	131072	4096	0.23	447.85	1054.01	1104.66	8.48	9.13	9.14	28818.20	✅
c5_i262144_o1024	sglang	5	262144	1024	0.15	56.42	20522.02	25944.29	54.11	118.98	124.02	53444.22	❌
c5_i262144_o1024	vllm	5	262144	1024	0.16	60.32	22564.89	26639.92	47.49	100.69	109.86	53790.59	⚠️
c5_i262144_o256	sglang	5	262144	256	0.16	21.94	25875.40	30596.95	97.05	231.75	252.98	43756.53	❌
c5_i262144_o256	vllm	5	262144	256	0.19	25.85	21986.99	26381.86	123.42	202.78	215.56	45761.90	❌
c5_i262144_o4096	sglang	5	262144	4096	0.11	209.50	16539.50	20323.40	25.05	56.69	60.22	60383.57	⚠️
c5_i262144_o4096	vllm	5	262144	4096	0.11	205.12	22171.78	26533.69	25.69	63.32	68.33	64086.27	⚠️

Notes

• SLO check uses TTFT P95 and TPOT mean (the same criteria as docs/SLO_STANDARDS.md).
• A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.