sora

0 Followers · 1 Following

• Joined on 2026-07-22

Repositories

1

Projects Packages Public Activity Starred Repositories

2

sora pushed to main at Meta-Eval/EvalHarness 2026-09-14 02:30:41 +00:00

e9d7e7f4eb README overhaul: features overview, config section, perf stats, FAQ

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 13:38:47 +00:00

a17dd88611 Untrack pip build/ artifacts, restore gen_profiles.yaml

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 13:38:10 +00:00

370953729b Fix perf stats (wrong import path), per-repeat checkpoints, README

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:58:58 +00:00

52872bcbc7 Per-benchmark Excel: each <bench>/<bench>.xlsx (4 sheets) alongside report.jsonl

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:51:31 +00:00

f118f4092b Remove detail.md from per-benchmark output (user preference: report.jsonl + summary.xlsx only)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:50:06 +00:00

ed675e790f Auto-stream threshold 8192→100000 (streaming strips all thinking params on GLM gateway; non-streaming works for 32k); EVALHARNESS_NO_AUTOSTREAM=1 to disable entirely

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:45:03 +00:00

f5aaddb870 Progress bar: colored fields (cur=yellow, elapsed=green, eta=cyan, retries=red, rate=dim)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:43:38 +00:00

624b2fd37e aime/hmmt max_tokens 32768→8192: stays on non-stream path where GLM gateway honors enable_thinking:false (thinking off, 15s/题)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:28:14 +00:00

3f555878b8 Fix: max_input_tokens extracted from YAML config and passed as the separate run_eval param (was going into gen_kwargs where truncation never saw it -> raw 128k text sent to gateway -> 400)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:24:19 +00:00

ff5d40aad5 Read timeout 300→600s base (GLM gateway 30+s startup on 100k+ token inputs)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:23:43 +00:00

7ada0113ad max_input_tokens back to 128000 (user confirmed same behavior as 120k)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:22:56 +00:00

b75fbc1a12 max_input_tokens 128000→120000 (GLM gateway unstable on very long inputs, 30s timeout at 128k)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:12:43 +00:00

ebc51de274 Rename dp4-nothink.yaml → default.yaml (auto-loaded)

sora transferred repository sora/EvalHarness to Meta-Eval/EvalHarness 2026-09-11 09:09:53 +00:00

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:04:26 +00:00

ed345c5ac1 Config auto-loads: single yaml in config/ becomes default generation params

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 09:00:30 +00:00

fe1852302a --config flag loads evalharness/config/<name>.yaml (per-bench generation params); yaml added to package-data; verified end-to-end

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 08:47:27 +00:00

ebf7a0e7e5 Config: generation params + repeats + max_input_tokens only; judge/env/limit controlled elsewhere

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 08:45:36 +00:00

60bdef3936 Complete config: all fields included (env, limit, limit_per_task, judge, judge_url, repeats, max_input_tokens, max_turns, concurrency)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 08:42:23 +00:00

eb6028e8c9 Config simplified: default block + per-bench overrides only (removed env/limit/judge/concurrency; max_input_tokens 128000)

sora pushed to main at Meta-Eval/EvalHarness 2026-09-11 08:37:19 +00:00

0815f6e4f8 Config: evalscope generation_config format + our extras (repeats, env) at bench level; parallel_tool_calls noted as unsupported in adapter