- Document the rule in docs/EXPERIMENT_GUIDE.md section 7 - Update all config.env SCENARIOS to follow the 5x rule - Migrate dsv4_h200_sglang, dsv4_h200_vllm, dsv4_p800_sglang from global NUM_PROMPTS to per-scenario num_prompts - Keep long-context phase2 as exception (low counts documented) - Add legacy 3-field fallback in run_bench.sh loops
TEMPLATE:SGLang vs vLLM 实验模板
这是一个最小可运行的实验模板。要新增一个实验:
cp -r experiments/TEMPLATE experiments/<your_experiment_name>
# 修改 config.env 里的参数
# 按需调整 start_sglang.sh / start_vllm.sh 的启动参数
bash experiments/<your_experiment_name>/run_bench.sh
目录约定
config.env:实验参数唯一来源start_sglang.sh/start_vllm.sh:server 启动脚本run_bench.sh:编排入口results/<run_id>/:自动生成的结果目录sglang/,vllm/:各自 backend 的结果comparison.md:横向对比表
通用工具位置
不要在本目录下复制解析脚本,直接引用:
- 预热:
../../scripts/common/warmup.py - 结果解析:
../../scripts/common/parse_backend.py - 对比报告:
../../scripts/common/compare.py - 公共函数:
../../scripts/common/lib.sh,../../scripts/common/platform.sh
提交规范
只提交最终文件:
git add experiments/<your_experiment_name>/config.env \
experiments/<your_experiment_name>/run_bench.sh \
experiments/<your_experiment_name>/start_*.sh \
experiments/<your_experiment_name>/results/<run_id>/comparison.md \
experiments/<your_experiment_name>/results/<run_id>/sglang/results.json \
experiments/<your_experiment_name>/results/<run_id>/sglang/report.md \
experiments/<your_experiment_name>/results/<run_id>/vllm/results.json \
experiments/<your_experiment_name>/results/<run_id>/vllm/report.md
raw_outputs/ 和 logs/ 已在 .gitignore 中忽略。