Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Scoring milestones: only when no live bar

Browse Source 
The bar's 'scoring N/164' counter and 21 milestone log lines were both
active on terminals -- the lines are noise there. Emit them only for
pipes/redirects where no bar exists (every ~10% now).

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-14 03:48:01 +00:00
parent 841b7b1ffb
commit 25e51c5b30
1 changed files with 6 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
evalharness/cli.py
View File

@ -720,11 +720,13 @@ def _cmd_eval_run(args) -> int:
for minutes with zero feedback otherwise (bar sits at
'generating 100%' and looks hung)."""
if _reporter is not None:
# live terminal: the bar carries the counter -- log
# lines here are pure noise (21 lines per bench)
_reporter.set_phase(f'scoring {done}/{total_s}')
# milestone lines: FIRST completion reports immediately
# (docker-slow runs otherwise look frozen for minutes),
# then every ~5%
step = max(1, total_s // 20) if total_s else 1
return
# pipes/redirects (no live bar): milestone lines instead,
# first completion immediately then every ~10%
step = max(1, total_s // 10) if total_s else 1
if _cb and (done == 1 or done % step == 0
or done == total_s):
_cb(f'Scoring {done}/{total_s} samples')