sora 7f3c42d39c Drop summary.md from outputs; reports switch to report.jsonl (header line + one sample per line, round-trip verified) ...

Co-Authored-By: Claude <noreply@anthropic.com>

2026-09-10 10:40:00 +00:00

..

_lcb_official

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

recipes

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

__init__.py

Add evaluation layer + visualization: extract/score/aggregate plugins, 28 recipes, official-aligned scorers (PRM800K math, DROP Hungarian EM/F1, SimpleQA A/B/C judge), report artifacts, console renderers, CLI eval/viz, regression tests

2026-08-24 06:09:38 +00:00

aggregator.py

!perf spec flag; perf_stats full dashboard columns (success_rate, latency/ttft/tpot mean+P90+P99, tokens mean/total, output TPS, request QPS); summary.csv perf columns aligned

2026-08-27 02:49:55 +00:00

checkpoint.py

tau2 via OFFICIAL engine: self-running env plugin (run_task hook + needs_adapter generic dispatch, no name hardcoding), deep generate() patch, official ToolCall shape, data plugin keeps verbatim Task json, official reward scoring; checkpoint plugin (per-sample resume) + summary csv with time/categories

2026-08-25 11:06:08 +00:00

extractor.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

math_grader.py

Add evaluation layer + visualization: extract/score/aggregate plugins, 28 recipes, official-aligned scorers (PRM800K math, DROP Hungarian EM/F1, SimpleQA A/B/C judge), report artifacts, console renderers, CLI eval/viz, regression tests

2026-08-24 06:09:38 +00:00

recipe.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

record.py

Drop summary.md from outputs; reports switch to report.jsonl (header line + one sample per line, round-trip verified)

2026-09-10 10:40:00 +00:00

registry.py

Add evaluation layer + visualization: extract/score/aggregate plugins, 28 recipes, official-aligned scorers (PRM800K math, DROP Hungarian EM/F1, SimpleQA A/B/C judge), report artifacts, console renderers, CLI eval/viz, regression tests

2026-08-24 06:09:38 +00:00

runner.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00

scorer.py

dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes

2026-09-08 05:55:04 +00:00