sora 60b2bfc0dc GPQA deterministic per-question choice shuffle (es-adapter parity: sha256(question) seed, position-bias protection) — was fixed-A; lb2 subset grouping aligned to es (length field, 60 samples)

2026-08-28 16:24:11 +00:00

..

agent

ladder20 complete 28/28 (Qwen3-8B pool): fixes en route — tau2 off-loop executor, double-encoded tool args, shared adapter cache; swe real-model 0.0 (oracle path verified 1.0)

2026-08-26 10:37:54 +00:00

data

GPQA deterministic per-question choice shuffle (es-adapter parity: sha256(question) seed, position-bias protection) — was fixed-A; lb2 subset grouping aligned to es (length field, 60 samples)

2026-08-28 16:24:11 +00:00

eval

!perf spec flag; perf_stats full dashboard columns (success_rate, latency/ttft/tpot mean+P90+P99, tokens mean/total, output TPS, request QPS); summary.csv perf columns aligned

2026-08-27 02:49:55 +00:00

model

GPQA deterministic per-question choice shuffle (es-adapter parity: sha256(question) seed, position-bias protection) — was fixed-A; lb2 subset grouping aligned to es (length field, 60 samples)

2026-08-28 16:24:11 +00:00

sandbox

Fix images_for_samples export (swe prefetch import error)

2026-08-27 15:27:30 +00:00

viz

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00

__init__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

__main__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

cli.py

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00