- AdaptiveGate rewritten (Netflix Gradient2): window-vs-window per-stream
speed gradient, count-driven windows with admission stamps, no thresholds
or mode state machine; failures x0.7 + 30s drain pause
- session-level admission for multi-turn agents (_SessionGate): in-progress
sessions hold slots until done, newcomers queue at the door; capacity
follows the model gate's discovered limit (CONCUR-style continuity)
- image service: memory-first register (zero docker calls for known
images), TTL-cached docker images listing, optimistic ready when the
daemon is unreachable (docker save contention no longer kills runs);
es tar loading removed in favor of ModelScope shipping (ms_images.py
per-image tar upload/pull with round-trip verification)
- runner: circuit breaker (12 consecutive failures abort the bench),
first-failure error printed immediately
- swe_agentic: image wait / docker run / rm off the event loop; exec
timeout becomes an observation the agent can react to; container gets
curlrc + git low-speed aborts (stalled github downloads fail fast)
- eval run excludes its own endpoints from http_proxy (a sick personal
proxy read as 'endpoint dead' and killed whole runs)
- progress bar shows failed count; swe agentic exec_workers 2 -> 4
Co-Authored-By: Claude <noreply@anthropic.com>
YAML default section and per-bench (tau2_bench: 200) both set it; the
runner's generic 8-starved every agent dialog (tau2 reward-0 root
cause). CLI wiring pops the key into args.max_turns before the
generation keys are stripped.
Co-Authored-By: Claude <noreply@anthropic.com>
- config env entries: bfcl_v3->bfcl_mock, tau2_bench->tau2_official;
the eval loop previously POPPED the env key and discarded it -- now
it feeds args.env (CLI --env still works as default/fallback)
- swe_bench_verified images: per-instance names are never published;
the official docker.io layout is repo-level BASE images
(swebench/sweb.eval.x86_64.{repo}) with per-instance images built
on top -- naming corrected (note: the swebench/* namespace is
currently blocked on every reachable CN mirror all the same)
Co-Authored-By: Claude <noreply@anthropic.com>
syy's es runs: humaneval x3, gpqa x2, aime25/26 x12 (already aligned),
mmlu_pro/longbench_v2 x1 (temp=0 deterministic -- repeats are noise,
and 3x 12k samples is pure cost). temp=1 benches get 3.
Co-Authored-By: Claude <noreply@anthropic.com>
