yy-fighting 1482601ae5 Add num_prompts = 5×concurrency convention and update all experiments
- Document the rule in docs/EXPERIMENT_GUIDE.md section 7
- Update all config.env SCENARIOS to follow the 5x rule
- Migrate dsv4_h200_sglang, dsv4_h200_vllm, dsv4_p800_sglang
  from global NUM_PROMPTS to per-scenario num_prompts
- Keep long-context phase2 as exception (low counts documented)
- Add legacy 3-field fallback in run_bench.sh loops
2026-07-09 09:26:41 +00:00
..

TEMPLATESGLang vs vLLM 实验模板

这是一个最小可运行的实验模板。要新增一个实验:

cp -r experiments/TEMPLATE experiments/<your_experiment_name>
# 修改 config.env 里的参数
# 按需调整 start_sglang.sh / start_vllm.sh 的启动参数
bash experiments/<your_experiment_name>/run_bench.sh

目录约定

  • config.env:实验参数唯一来源
  • start_sglang.sh / start_vllm.shserver 启动脚本
  • run_bench.sh:编排入口
  • results/<run_id>/:自动生成的结果目录
    • sglang/, vllm/:各自 backend 的结果
    • comparison.md:横向对比表

通用工具位置

不要在本目录下复制解析脚本,直接引用:

  • 预热:../../scripts/common/warmup.py
  • 结果解析:../../scripts/common/parse_backend.py
  • 对比报告:../../scripts/common/compare.py
  • 公共函数:../../scripts/common/lib.sh, ../../scripts/common/platform.sh

提交规范

只提交最终文件:

git add experiments/<your_experiment_name>/config.env \
        experiments/<your_experiment_name>/run_bench.sh \
        experiments/<your_experiment_name>/start_*.sh \
        experiments/<your_experiment_name>/results/<run_id>/comparison.md \
        experiments/<your_experiment_name>/results/<run_id>/sglang/results.json \
        experiments/<your_experiment_name>/results/<run_id>/sglang/report.md \
        experiments/<your_experiment_name>/results/<run_id>/vllm/results.json \
        experiments/<your_experiment_name>/results/<run_id>/vllm/report.md

raw_outputs/logs/ 已在 .gitignore 中忽略。