EvalHarness

History

sora 3cc0158236 hf_raw: fall back to cached blobs when the mirror listing degrades

The tree API intermittently returns truncated listings (9 parquets
listed as [.gitattributes, README.md] at night); previously that failed
the bench even though 5 of the files were already in the shared blob
store. Empty selection now uses matching cached blobs with a notice;
only a truly cold cache still raises.

Co-Authored-By: Claude <noreply@anthropic.com>

2026-09-17 06:37:55 +00:00

datasets

Fix NameError _fmt (module-level helper restored); aime/hmmt max_tokens 32768->8192 (non-stream path, gateway honors nothink)

2026-09-11 07:20:02 +00:00

__init__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

dataset.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

loader.py

hf_raw: fall back to cached blobs when the mirror listing degrades