qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
83 Commits 1 Branch 0 Tags
Commit Graph

6 Commits

Author SHA1 Message Date
yy-fighting
1482601ae5 Add num_prompts = 5×concurrency convention and update all experiments 
- Document the rule in docs/EXPERIMENT_GUIDE.md section 7
- Update all config.env SCENARIOS to follow the 5x rule
- Migrate dsv4_h200_sglang, dsv4_h200_vllm, dsv4_p800_sglang
  from global NUM_PROMPTS to per-scenario num_prompts
- Keep long-context phase2 as exception (low counts documented)
- Add legacy 3-field fallback in run_bench.sh loops
 2026-07-09 09:26:41 +00:00
yy-fighting
7d2ed7f05d Add vLLM+MTP speculative decoding benchmark vs default on H200 
- Experiment: experiments/dsv4_h200_vllm_mtp_vs_default/
- Use official DeepSeek-V4 recipe: --speculative-config '{method: mtp, num_speculative_tokens: 1}'
- Reuse default-side results from dsv4_h200_vllm_dspark_vs_default
- Add TTFT analysis for DSpark vs default trade-offs
 2026-07-08 16:13:01 +00:00
yy-fighting
cd0b083a10 results(vllm-dspark-vs-default): complete TTFT comparison run 20260708-142121 2026-07-08 14:35:14 +00:00
yy-fighting
0bb8adf2d9 feat: add SLO status field to per-backend reports and results.json 
- parse_backend.py for sglang_vs_vllm and dspark_vs_default:
  - compute slo_status per scenario (TTFT P95 < 3000ms, TPOT mean < 50ms)
  - append slo_status to results.json
  - add SLO column to report.md
- BENCHMARK_WORKFLOW.md documents slo_status in schema and checklist
 2026-07-08 08:28:37 +00:00
yy-fighting
162168da3c feat: record exact server launch args in results.json for reproducibility 
- sglang_vs_vllm/run_bench.sh: add phase1/phase2 server_args per backend
- dspark_vs_default/run_bench.sh: add server_args for dspark/default
- READMEs document where to find the recorded server args
 2026-07-08 08:21:55 +00:00
yy-fighting
5d1cd9c266 feat: add vLLM+DSpark vs vLLM-default TTFT comparison experiment 
- New experiments/dsv4_h200_vllm_dspark_vs_default/
- start_dspark.sh: vllm serve with --spec-method dspark --spec-tokens 5
- start_default.sh: plain vllm serve baseline
- Unified scenario matrix (512/4000 input, 1/32/128 concurrency)
- parse_backend.py + compare.py focused on TTFT differences
- README.md + root README entries
 2026-07-08 08:15:02 +00:00