Per-instance images are 1-4GB each; a --limit 10 run pulls up to 10 of
them and previously stranded the whole footprint on Ctrl+C/exit. Every
image THIS process pulls is now registered and released by an atexit
hook (containers first, then rmi) -- images that already existed
locally are never touched. EVALHARNESS_KEEP_SWE_IMAGES=1 opts out for
prefetch-style runs that want to keep them.
Co-Authored-By: Claude <noreply@anthropic.com>
docker run's implicit pull prints nothing and the 120s start timeout
kills runs on slow mirrors; ensure_image brings the progress display,
mirror chain and watchdog to per-instance pulls (each sample = its own
image, 10 samples = 10 pulls).
Co-Authored-By: Claude <noreply@anthropic.com>
The captured diff was stripped, and git apply rejects diffs whose last
line lacks a trailing newline ('corrupt patch at line N') -- the first
agentic score was resolved=0 with patch_apply_failed despite a correct
fix. Verified: same payload + newline applies clean in the official
container.
Co-Authored-By: Claude <noreply@anthropic.com>
Without it get_env('swe_agentic') raised TypeError at env creation --
and the swallowed exception silently degraded the run to SINGLE-TURN
prose generation (421s of model monologue, no container, empty
trajectory). This was the agentic 'run' that looked like it worked.
Co-Authored-By: Claude <noreply@anthropic.com>
Function-level silencers kept losing the race to whichever 'import
tau2.*' fired first in the process. The filter + TAU2_DATA_DIR now
install when the env registry loads the plugin, strictly before any
tau2 import can happen anywhere.
Co-Authored-By: Claude <noreply@anthropic.com>
The startup noise (.env found note, data directory note) is itself
WARNING-level, so the WARNING filter passed it through. Level is now
ERROR and the data-dir warnings die at the source with the correct
TAU2_DATA_DIR.
Co-Authored-By: Claude <noreply@anthropic.com>
Both reward-0 runs died with 'Termination reason: max_steps' after 3-4
exchanges -- the runner passes its generic default (8) down, far too
few for greet->verify->find->policy->act->confirm. Official tau2 runs
use 40+; floor, don't cap.
Co-Authored-By: Claude <noreply@anthropic.com>
loguru restricted to WARNING+ (every orchestrator step dumped full
messages -- thousands of lines per bench). AssistantMessage.is_final_
chunk now False on tool-call turns (official adapter semantics: True
means the agent is FINISHED talking); always-True handed the turn
back to the user prematurely mid-action-sequence.
Co-Authored-By: Claude <noreply@anthropic.com>
The user simulator (GLM via our adapter) inlines its scenario reasoning
in content as '...instructions...</think>reply' -- passed through
unstripped, the AGENT receives the scenario's secret instructions
(task goal, disclosure strategy), inflating rewards. Both channels now
trimmed at the last </think>.
Co-Authored-By: Claude <noreply@anthropic.com>
Same UX as the docker image chain: try local sources first, then the
GitHub URL, and on total failure print the exact manual commands --
with a warning that PyPI's 'tau2' is an unrelated physics package.
Co-Authored-By: Claude <noreply@anthropic.com>
