2c3672f2cb
- AdaptiveGate rewritten (Netflix Gradient2): window-vs-window per-stream speed gradient, count-driven windows with admission stamps, no thresholds or mode state machine; failures x0.7 + 30s drain pause - session-level admission for multi-turn agents (_SessionGate): in-progress sessions hold slots until done, newcomers queue at the door; capacity follows the model gate's discovered limit (CONCUR-style continuity) - image service: memory-first register (zero docker calls for known images), TTL-cached docker images listing, optimistic ready when the daemon is unreachable (docker save contention no longer kills runs); es tar loading removed in favor of ModelScope shipping (ms_images.py per-image tar upload/pull with round-trip verification) - runner: circuit breaker (12 consecutive failures abort the bench), first-failure error printed immediately - swe_agentic: image wait / docker run / rm off the event loop; exec timeout becomes an observation the agent can react to; container gets curlrc + git low-speed aborts (stalled github downloads fail fast) - eval run excludes its own endpoints from http_proxy (a sick personal proxy read as 'endpoint dead' and killed whole runs) - progress bar shows failed count; swe agentic exec_workers 2 -> 4 Co-Authored-By: Claude <noreply@anthropic.com>
67 lines
1.0 KiB
YAML
67 lines
1.0 KiB
YAML
default:
|
|
temperature: 0.0
|
|
top_p: 1.0
|
|
stream: true
|
|
max_tokens: 200336
|
|
aime24:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 200336
|
|
aime25:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 200336
|
|
aime26:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 200336
|
|
hmmt26:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 200336
|
|
imo_answerbench:
|
|
temperature: 1.0
|
|
gpqa_diamond:
|
|
temperature: 1.0
|
|
repeats: 3
|
|
max_tokens: 200336
|
|
mmlu:
|
|
max_tokens: 200336
|
|
mmlu_pro:
|
|
max_tokens: 200336
|
|
cmmlu:
|
|
max_tokens: 200336
|
|
arc:
|
|
max_tokens: 200336
|
|
hellaswag:
|
|
max_tokens: 200336
|
|
winogrande:
|
|
max_tokens: 200336
|
|
simple_qa:
|
|
max_tokens: 200336
|
|
trivia_qa:
|
|
max_tokens: 200336
|
|
humaneval:
|
|
temperature: 1.0
|
|
repeats: 3
|
|
live_code_bench:
|
|
temperature: 1.0
|
|
longbench_v2:
|
|
max_tokens: 200336
|
|
max_input_tokens: 128000
|
|
openai_mrcr:
|
|
max_tokens: 200336
|
|
max_input_tokens: 128000
|
|
bfcl_v3:
|
|
max_tokens: 4096
|
|
general_fc:
|
|
max_tokens: 4096
|
|
tau2_bench:
|
|
max_tokens: 16384
|
|
|
|
swe_bench_verified_agentic:
|
|
max_tokens: 8196
|
|
temperature: 0.0
|
|
env: swe_agentic
|
|
max_turns: 250
|