2 Commits

Author	SHA1	Message	Date
sora	565ba11790	swe datasets: carry version/patch/hints/created_at in metadata ... Official make_test_spec indexes MAP_REPO_VERSION_TO_SPECS[repo][version] -- a None version KeyError'd the official scorer (first agentic run scored 0 with 'KeyError: None' hidden in details). The base converter never copied these fields; both variants now carry them, verified with a live make_test_spec call (spec + 2KB eval script + correct image key). Co-Authored-By: Claude <noreply@anthropic.com>	2026-09-18 08:11:18 +00:00
sora	f662006517	swe_bench_verified_agentic: multi-turn SWE agent (mini-swe-agent protocol) ... Ports es's swe_bench_agentic_adapter into our plugin architecture: - env swe_agentic: per-sample LONG-RUNNING container (official sweb image, /testbed, bash -lc like the testbed startup files expect), single bash tool via function calling, sentinel-submission protocol (COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT + patch), git-diff fallback; observations capped at 30k chars - dataset swe_bench_verified_agentic: same princeton source/converter, separate bench name so both variants coexist - recipe: recovered patch + OFFICIAL test_patch applied in-container, FAIL_TO_PASS + capped PASS_TO_PASS via conda testbed pytest, 1800s - config: max_turns 250, env swe_agentic Single-turn swe_bench_verified is untouched. Co-Authored-By: Claude <noreply@anthropic.com>	2026-09-18 02:29:35 +00:00