qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

workflow: document what to commit for each experiment

Browse Source 
Add rule 10: commit experiment code together with final results.json
and .md reports; do not commit logs/raw_outputs/gpu_logs. Also update
the pre-commit checklist accordingly.
This commit is contained in:
yy-fighting 2026-07-10 02:22:30 +00:00
parent 666c583844
commit 938b2f1e88
1 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
BENCHMARK_WORKFLOW.md
View File

@ -83,6 +83,12 @@
- This is required for cross-platform reproduction: when the same experiment is run on H200 and P800, the only differences should be model paths, ports, and device IDs. - This is required for cross-platform reproduction: when the same experiment is run on H200 and P800, the only differences should be model paths, ports, and device IDs.
- If an experiment uses `start_server.sh`, that script should be self-contained and its command line should be reproducible from `results.json` alone. - If an experiment uses `start_server.sh`, that script should be self-contained and its command line should be reproducible from `results.json` alone.
10. **Version control: commit code and final artifacts only.**
- Always commit the experiment code (`config.env`, `run_bench.sh`, `start_*.sh`, parsers, etc.) together with the run's final artifacts.
- Final artifacts to commit: `results.json` (structured data) and `report.md` / `comparison.md` (human-readable summaries).
- Do **not** commit intermediate logs, per-request JSONL raw outputs, or GPU sampling CSVs. These are already ignored by `.gitignore` (`raw_outputs/`, `logs/`, `gpu_logs/`).
- `scenarios.tsv` and `skipped_after_oom.csv` may be committed if they are useful for reproducing the test plan, but they are optional.
## Naming Conventions ## Naming Conventions
### Experiment result directories ### Experiment result directories
@ -340,3 +346,4 @@ See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough.
- [ ] `<run>/README.md` exists and documents provenance (or the report itself covers provenance). - [ ] `<run>/README.md` exists and documents provenance (or the report itself covers provenance).
- [ ] Scripts either live under `experiments/<name>/` or in `scripts/` (or `scripts/<group>/`). - [ ] Scripts either live under `experiments/<name>/` or in `scripts/` (or `scripts/<group>/`).
- [ ] Script path references updated after moving. - [ ] Script path references updated after moving.
- [ ] Commits include the experiment code plus final `results.json` and `.md` reports; logs/raw outputs are left ignored.