sora adb46d4927 Mirror chain: +xuanyuan/dockerhub.icu (CN) + mirror.gcr.io/public.ecr (intl) ...

16 total. The intl entries need real egress (usually blocked on this
host) but cost only 10s each with the idle watchdog on networks that
have them. Also noted: a 10s watchdog can kill sources mid slow
negotiation (rat.dev emitted 'Pulling fs layer' then went quiet) --
EVALHARNESS_PULL_IDLE_S=30 softens that.

Co-Authored-By: Claude <noreply@anthropic.com>

2026-09-17 05:51:16 +00:00

..

agent

Usability round: progress plugin, CLI provider flags, top-level run(), vendored BFCL checker

2026-09-10 05:46:45 +00:00

config

thinking.yaml: 64k budgets for the full-thinking ladder (es parity)

2026-09-17 02:10:50 +00:00

data

Fix NameError _fmt (module-level helper restored); aime/hmmt max_tokens 32768->8192 (non-stream path, gateway honors nothink)

2026-09-11 07:20:02 +00:00

eval

bigcodebench: override the image ENTRYPOINT so our runner executes

2026-09-17 03:05:13 +00:00

fingerprint

fingerprint: integrate fp_fusion model fingerprint benchmark

2026-09-11 03:52:21 +00:00

model

Retarget the bar to 0/N the moment scoring starts

2026-09-17 02:53:56 +00:00

progress

Unified duration format: elapsed/cur use the same tiered format as eta

2026-09-15 06:37:45 +00:00

sandbox

Mirror chain: +xuanyuan/dockerhub.icu (CN) + mirror.gcr.io/public.ecr (intl)

2026-09-17 05:51:16 +00:00

themes

Fixes: theme markup closing tags ([bold green]...[/bold green]); evaluate() runs off the event loop (sympy equivalence on hard math froze the progress clock for the whole bench)

2026-09-11 04:36:51 +00:00

third_party/bfcl

Usability round: progress plugin, CLI provider flags, top-level run(), vendored BFCL checker

2026-09-10 05:46:45 +00:00

viz

Unified, nicer result tables + conda env setup in README

2026-09-10 06:30:22 +00:00

__init__.py

Usability round: progress plugin, CLI provider flags, top-level run(), vendored BFCL checker

2026-09-10 05:46:45 +00:00

__main__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

cli.py

--thinking: unified switch (off/low/medium/high/max/full)

2026-09-16 04:02:34 +00:00

hooks.py

Pluginize the run shell: @register_progress (rich/plain), @register_theme, @register_hook (on_benchmark_failed/done), @register_prober; CLI gains --progress-plugin/--theme

2026-09-11 03:26:19 +00:00