sora/evalstone
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
62 Commits 2 Branches 0 Tags
Commit Graph

7 Commits

Author SHA1 Message Date
sora
13274243a0 Bump vendored EvalScope and add K3-ready DPV4 configs. 
Keep K3 suite selection and report-schema scoring in bash, merge K3/vision dataset_args into dpv4 yamls, and pin EvalScope at 735d920ee911 with local patches.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-02 07:30:48 +00:00
aaxiyao
8b7dff96e7 add fingerprint benchmark 2026-08-25 02:06:23 +00:00
sora
ba2b5c1772 fix: terminal_bench_v2_1 result collection, apt mirror, and restore run.py env var 2026-08-05 03:56:44 +00:00
sora
b1eb4e116a chore: update evalscope runner, thinking config and webui 2026-07-31 02:36:51 +00:00
sora
991a4a5f7f Add sandbox images download from ModelScope; fix Excel sample count total 2026-07-27 05:34:16 +00:00
sora
e26fa0fe44 feat: restructure output layout and add --folder-name 
- Add --folder-name CLI arg; defaults to safe model name, or
  {model}_THINKING when --thinking is enabled.
- Move benchmark outputs under output/{folder_name}/{benchmark}/...
- Remove redundant {model_name} subfolder from predictions/reports/reviews.
- Update collect_results.py to write both CSV and Excel to
  results/{folder_name}.csv/xlsx.
- Update perf backup/restore paths to match the new layout.
- Patch evalscope CacheManager to drop model_name from cache paths.
 2026-07-23 03:06:52 +00:00
sora
2ee0af0728 docs: add resource prep section and sync docker/bash helpers 
- Rewrite myread.md with proper markdown and four download sources:
  code, data, evalscope-complete-py312 image, execution images.
- Add bash/collect_results.py, perf_backup.py, pull_swe_bench_images.py
  for result aggregation, breakpoint perf recovery, and SWE-bench
  image pre-pulling.
- Update bash/run.py with multi-run suites, perf backup/restore, and
  whitelist-based summary.
- Update config/dpv4-int8_nothinking.yaml benchmark parameters.
- Ignore /docker_images and /results in .gitignore.
 2026-07-23 02:18:15 +00:00