sora c4473a1e1c Token-level middle truncation (ported from evalside run.py: head+tail 128k, mrcr message-window); max_input_tokens param; perf fields (ttft/itl/status/retries) verified persisted in checkpoints

2026-08-27 03:23:33 +00:00

..

agent

ladder20 complete 28/28 (Qwen3-8B pool): fixes en route — tau2 off-loop executor, double-encoded tool args, shared adapter cache; swe real-model 0.0 (oracle path verified 1.0)

2026-08-26 10:37:54 +00:00

data

swe_bench_verified end-to-end verified (oracle patch -> official sweb image -> resolved); image naming mapped to local swebench/ set; ladder2 complete 28/28

2026-08-26 06:40:55 +00:00

eval

!perf spec flag; perf_stats full dashboard columns (success_rate, latency/ttft/tpot mean+P90+P99, tokens mean/total, output TPS, request QPS); summary.csv perf columns aligned

2026-08-27 02:49:55 +00:00

model

Token-level middle truncation (ported from evalside run.py: head+tail 128k, mrcr message-window); max_input_tokens param; perf fields (ttft/itl/status/retries) verified persisted in checkpoints

2026-08-27 03:23:33 +00:00

sandbox

Multi-endpoint model pool (openai-pool/{lo..hi} round-robin with failover), nothink/textools spec flags (chat_template_kwargs enable_thinking=false; text-protocol tool calls for backends without --enable-auto-tool-choice), global cross-bench queue in ladder runs

2026-08-26 06:26:52 +00:00

viz

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00

__init__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

__main__.py

EvalHarness data layer: 28 dataset plugins, lazy materialize cache (raw/ + samples.jsonl + meta.json), ModelScope native loader, CLI list/fetch/unload/stats/show

2026-08-24 03:35:11 +00:00

cli.py

Excel renderer (xlsxwriter, 4 sheets: Summary dashboard with 21 columns, Perf detail, Categories, Samples drill-down); !perf spec flag; viz --style excel

2026-08-27 02:56:07 +00:00