qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/bench_results/dsv4_backend_comparison_20260707/README.md
Benchmark Bot eb431d3925 Initial commit: benchmark scripts and final reports
2026-07-08 02:17:13 +00:00

3.9 KiB
Raw Blame History

DSV4 Backend Comparison Raw Outputs (2026-07-07)

Location

/data/user1/yy/bench_results/dsv4_backend_comparison_20260707

Contents

  • raw_outputs/ — raw per-benchmark JSONL files produced by sglang.bench_serving --output-file --output-details.
  • README.md — this file.

File Naming Convention

Each file follows the pattern:

{backend}_0707_{concurrency}_{input_len}_{output_len}.jsonl

Where:

  • {backend}: sglang or vllm
  • {concurrency}: max concurrent requests (max-concurrency)
  • {input_len}: --random-input-len
  • {output_len}: --random-output-len

Output File Inventory

SGLang backend

File Concurrency Input len Output len
sglang_0707_32_512_256.jsonl 32 512 256
sglang_0707_128_512_256.jsonl 128 512 256
sglang_0707_256_512_256.jsonl 256 512 256
sglang_0707_512_512_256.jsonl 512 512 256
sglang_0707_512_512_2000.jsonl 512 512 2000
sglang_0707_32_512_2000.jsonl 32 512 2000
sglang_0707_32_4000_512.jsonl 32 4000 512
sglang_0707_128_4000_512.jsonl 128 4000 512
sglang_0707_512_4000_512.jsonl 512 4000 512
sglang_0707_32_16000_512.jsonl 32 16000 512
sglang_0707_128_16000_512.jsonl 128 16000 512
sglang_0707_512_1000_256.jsonl 512 1000 256
sglang_0707_768_1000_256.jsonl 768 1000 256
sglang_0707_1024_1000_256.jsonl 1024 1000 256
sglang_0707_512_1000_1000.jsonl 512 1000 1000

vLLM backend

File Concurrency Input len Output len
vllm_0707_32_512_256.jsonl 32 512 256
vllm_0707_128_512_256.jsonl 128 512 256
vllm_0707_256_512_256.jsonl 256 512 256
vllm_0707_128_512_2000.jsonl 128 512 2000
vllm_0707_256_512_2000.jsonl 256 512 2000
vllm_0707_512_512_2000.jsonl 512 512 2000
vllm_0707_32_1000_256.jsonl 32 1000 256
vllm_0707_128_1000_256.jsonl 128 1000 256
vllm_0707_256_1000_256.jsonl 256 1000 256
vllm_0707_512_1000_256.jsonl 512 1000 256
vllm_0707_768_1000_256.jsonl 768 1000 256
vllm_0707_1024_1000_256.jsonl 1024 1000 256
vllm_0707_32_1000_1000.jsonl 32 1000 1000
vllm_0707_128_1000_1000.jsonl 128 1000 1000
vllm_0707_256_1000_1000.jsonl 256 1000 1000
vllm_0707_512_1000_1000.jsonl 512 1000 1000
vllm_0707_1024_1000_1000.jsonl 1024 1000 1000
vllm_0707_32_4000_512.jsonl 32 4000 512
vllm_0707_128_4000_512.jsonl 128 4000 512
vllm_0707_256_4000_512.jsonl 256 4000 512
vllm_0707_512_4000_512.jsonl 512 4000 512
vllm_0707_32_8000_1000.jsonl 32 8000 1000
vllm_0707_128_8000_1000.jsonl 128 8000 1000
vllm_0707_256_8000_1000.jsonl 256 8000 1000
vllm_0707_512_8000_1000.jsonl 512 8000 1000
vllm_0707_32_16000_512.jsonl 32 16000 512
vllm_0707_128_16000_512.jsonl 128 16000 512
vllm_0707_256_16000_512.jsonl 256 16000 512
vllm_0707_32_32000_512.jsonl 32 32000 512
vllm_0707_128_32000_512.jsonl 128 32000 512

Provenance

These files are legacy raw outputs from an ad-hoc SGLang vs vLLM backend comparison sweep run on 2026-07-07. The exact generator command/script that produced them was not preserved in the repository. They were generated using sglang.bench_serving with --output-file and --output-details, against:

  • Model: /data/models/DeepSeek-V4-Flash
  • SGLang port: 30000
  • vLLM port: 8000 (inferred from file content)
  • Dataset: random

For future reproducible comparisons, use the script:

scripts/benchmark_dsv4_backend_comparison.sh

Related Reports

  • /data/user1/yy/dsv4_inference_comparison_report.md — earlier DSV4 inference comparison report.
  • /data/user1/yy/bench_results/dsv4_comparison_20260705_152221/ — earlier comparison run outputs (.json/.log).