evalstone

sora/evalstone

Fork 0

Commit Graph

Author	SHA1	Message	Date
sora	e26fa0fe44	feat: restructure output layout and add --folder-name - Add --folder-name CLI arg; defaults to safe model name, or {model}_THINKING when --thinking is enabled. - Move benchmark outputs under output/{folder_name}/{benchmark}/... - Remove redundant {model_name} subfolder from predictions/reports/reviews. - Update collect_results.py to write both CSV and Excel to results/{folder_name}.csv/xlsx. - Update perf backup/restore paths to match the new layout. - Patch evalscope CacheManager to drop model_name from cache paths.	2026-07-23 03:06:52 +00:00
sora	2ee0af0728	docs: add resource prep section and sync docker/bash helpers - Rewrite myread.md with proper markdown and four download sources: code, data, evalscope-complete-py312 image, execution images. - Add bash/collect_results.py, perf_backup.py, pull_swe_bench_images.py for result aggregation, breakpoint perf recovery, and SWE-bench image pre-pulling. - Update bash/run.py with multi-run suites, perf backup/restore, and whitelist-based summary. - Update config/dpv4-int8_nothinking.yaml benchmark parameters. - Ignore /docker_images and /results in .gitignore.	2026-07-23 02:18:15 +00:00

Author

SHA1

Message

Date

sora

e26fa0fe44

feat: restructure output layout and add --folder-name

- Add --folder-name CLI arg; defaults to safe model name, or
  {model}_THINKING when --thinking is enabled.
- Move benchmark outputs under output/{folder_name}/{benchmark}/...
- Remove redundant {model_name} subfolder from predictions/reports/reviews.
- Update collect_results.py to write both CSV and Excel to
  results/{folder_name}.csv/xlsx.
- Update perf backup/restore paths to match the new layout.
- Patch evalscope CacheManager to drop model_name from cache paths.

2026-07-23 03:06:52 +00:00

sora

2ee0af0728

docs: add resource prep section and sync docker/bash helpers

- Rewrite myread.md with proper markdown and four download sources:
  code, data, evalscope-complete-py312 image, execution images.
- Add bash/collect_results.py, perf_backup.py, pull_swe_bench_images.py
  for result aggregation, breakpoint perf recovery, and SWE-bench
  image pre-pulling.
- Update bash/run.py with multi-run suites, perf backup/restore, and
  whitelist-based summary.
- Update config/dpv4-int8_nothinking.yaml benchmark parameters.
- Ignore /docker_images and /results in .gitignore.

2026-07-23 02:18:15 +00:00

2 Commits