sskj/experiments/dsv4_h200_256k_4k_probe/results/20260708-094838/comparison.md

SGLang vs vLLM on DeepSeek-V4-Flash (H200, TP=8)

Summary

• Model: /data/models/DeepSeek-V4-Flash
• Hardware: 8x NVIDIA H200 143GB
• Tensor Parallelism: 8
• Benchmark client: sglang.bench_serving
• SLO reference: S2 tier (TTFT P95 < 3s, TPOT < 50ms)

Side-by-side results

Scenario	Backend	Conc	Input	Output	Req/s	OutTok/s	TTFT P95(ms)	TTFT P99(ms)	TPOT Mean(ms)	TPOT P95(ms)	TPOT P99(ms)	E2E P99(ms)	SLO
c2_i262144_o4096	sglang	2	262144	4096	0.07	165.58	9734.97	10056.38	9.32	12.82	13.41	38983.24	⚠️
c2_i262144_o4096	vllm	2	262144	4096	0.08	180.27	8742.36	9023.10	8.64	11.56	12.05	35853.05	⚠️

Notes

• SLO check uses TTFT P95 and TPOT mean (the same criteria as scripts/SLO_STANDARDS.md).
• A ⚠️ indicates one of the two metrics is out of target; ❌ indicates both are out.