This website requires JavaScript.
Explore
Help
Register
Sign In
Meta-Eval
/
EvalHarness
Watch
3
Star
0
Fork
0
You've already forked EvalHarness
Code
Issues
Pull Requests
Actions
Packages
Projects
Releases
Wiki
Activity
35
Commits
1
Branch
0
Tags
Commit Graph
4 Commits
Author
SHA1
Message
Date
sora
46bef7d3dd
dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes
...
Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 05:55:04 +00:00
sora
e32da10ce9
longbench_v2 chain of fixes: reasoning_content output fallback (Qwen3 thinking channel), nothink soft switch /no_think (template-400 immune), truncation input+output budget, adaptive context shrink on server 400 (cross-tokenizer margin, body-aware detection), pool fails over on 4xx
2026-08-27 06:27:59 +00:00
sora
f2a914d0ff
Performance profile: TTFT/ITL via SSE streaming (collect_perf flag), retries/http_status per request, pool success/retry/failure rates, perf_stats aggregator (P50/P95/P99 latency+ttft, tokens, retry_rate) in metric_groups['perf']; verified on fake SSE server
2026-08-27 02:45:51 +00:00
sora
7114564301
Multi-endpoint model pool (openai-pool/{lo..hi} round-robin with failover), nothink/textools spec flags (chat_template_kwargs enable_thinking=false; text-protocol tool calls for backends without --enable-auto-tool-choice), global cross-bench queue in ladder runs
2026-08-26 06:26:52 +00:00