sora f7c7536400 Fix images_for_samples export (swe prefetch import error)

2026-08-27 15:27:30 +00:00

..

agent

ladder20 complete 28/28 (Qwen3-8B pool): fixes en route — tau2 off-loop executor, double-encoded tool args, shared adapter cache; swe real-model 0.0 (oracle path verified 1.0)

2026-08-26 10:37:54 +00:00

data

Align with local evalscope run.py (full read-through): (1) shuffle+seed42 sample selection identical to es _shuffle_in_place -- same exact sample subset both frameworks; (2) official MATH_PROMPT_TEMPLATE CoT boxed suffix on 7 math benches via DatasetSpec.prompt_suffix; (3) per-bench gen_config (temp/max_tokens from dpv4 yaml) on DatasetSpec, default-merged into gen_kwargs

2026-08-27 11:25:17 +00:00

eval

!perf spec flag; perf_stats full dashboard columns (success_rate, latency/ttft/tpot mean+P90+P99, tokens mean/total, output TPS, request QPS); summary.csv perf columns aligned

2026-08-27 02:49:55 +00:00

model

Align with local evalscope run.py (full read-through): (1) shuffle+seed42 sample selection identical to es _shuffle_in_place -- same exact sample subset both frameworks; (2) official MATH_PROMPT_TEMPLATE CoT boxed suffix on 7 math benches via DatasetSpec.prompt_suffix; (3) per-bench gen_config (temp/max_tokens from dpv4 yaml) on DatasetSpec, default-merged into gen_kwargs

2026-08-27 11:25:17 +00:00

sandbox

Fix images_for_samples export (swe prefetch import error)

2026-08-27 15:27:30 +00:00

viz

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00

__init__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

__main__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

cli.py

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00