sora 63ffd1b613 Fix NameError _fmt (module-level helper restored); aime/hmmt max_tokens 32768->8192 (non-stream path, gateway honors nothink) ...

Co-Authored-By: Claude <noreply@anthropic.com>

2026-09-11 07:20:02 +00:00

..

datasets

Fix NameError _fmt (module-level helper restored); aime/hmmt max_tokens 32768->8192 (non-stream path, gateway honors nothink)

2026-09-11 07:20:02 +00:00

__init__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

dataset.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

loader.py

HF endpoint auto-fallback: when huggingface.co is unreachable, probe mirrors (hf-mirror.com) and switch automatically with a one-line notice; error only if all fail

2026-09-10 10:57:16 +00:00

registry.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

sample.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

spec.py

GPQA 0.15->0.55: (1) per-bench prompt_style cot_letter (es GPQA CoT template, matches its 'Think step by step' contract), (2) mcq_letter extractor case-insensitive answer-tail (ANSWER: vs answer), (3) deterministic choice shuffle

2026-08-28 18:01:16 +00:00