Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

swe_agentic: env_state carries version/hints/env_commit (make_test_spec need)

Browse Source 
MAP_REPO_VERSION_TO_SPECS[repo][version] KeyError'd on empty version --
env_state didn't store the version field, the scorer's fallback to
sample.metadata hit the positional misalign again. The four official
metadata fields now stored alongside the rest.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-18 16:03:21 +00:00
parent 014e8bcd9b
commit 7085624da0
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
evalharness/agent/envs/swe_agentic.py
View File

@ -246,6 +246,10 @@ class SWEAgenticEnvironment(Environment):
'test_patch': md.get('test_patch'), 'test_patch': md.get('test_patch'),
'FAIL_TO_PASS': md.get('FAIL_TO_PASS'), 'FAIL_TO_PASS': md.get('FAIL_TO_PASS'),
'PASS_TO_PASS': md.get('PASS_TO_PASS'), 'PASS_TO_PASS': md.get('PASS_TO_PASS'),
'version': md.get('version') or '',
'hints_text': md.get('hints_text') or '',
'created_at': md.get('created_at') or '',
'environment_setup_commit': md.get('environment_setup_commit') or '',
'image': image, 'image': image,
'turns_used': turns, 'turns_used': turns,
}, },