Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Gate evidence: 2x level exactly (drop the 5-sample floor)

Browse Source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-15 03:51:48 +00:00
parent 8fc7df5b26
commit e021c94f44
1 changed files with 2 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
evalharness/model/pool.py
View File

@ -320,9 +320,8 @@ class AdaptiveGate:
lvl = max(1, int(self.limit)) lvl = max(1, int(self.limit))
# not enough evidence yet at this level: keep measuring. # not enough evidence yet at this level: keep measuring.
# need = max(MIN_OK, 2x level): rate noise shrinks only with # evidence = 2 x level completions, no other floor
# samples proportional to the concurrency being judged need_ok = max(2, lvl * 2)
need_ok = max(self.MIN_OK, lvl * 2)
# dwell fallback scales with the OBSERVED completion cadence: # dwell fallback scales with the OBSERVED completion cadence:
# a 60s-per-request bench needs minutes, not 25s, before a # a 60s-per-request bench needs minutes, not 25s, before a
# single-sample judgment is acceptable # single-sample judgment is acceptable