Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

swe: repo-base image naming (correct indentation this time)

Browse Source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-17 08:33:45 +00:00
parent 5c14d11852
commit 081249d47c
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
evalharness/data/datasets/swe_bench_verified.py
View File

@ -22,7 +22,11 @@ def swe_bench_verified():
instance_id = record['instance_id']
# image naming: local swebench/ set uses {repo}_1776_{repo}-{num}; the
# newer official layout is sweb.eval.x86_64.{repo}__{repo}-{num}
img = f'sweb.eval.x86_64.{instance_id}'
# OFFICIAL docker layout: repo-level BASE images
# (swebench/sweb.eval.x86_64.{repo}) -- per-instance images are
# NEVER published; the harness builds them on top of the base
_repo = instance_id.rsplit('-', 1)[0]
img = f'swebench/sweb.eval.x86_64.{_repo}:latest'
if '__' in instance_id:
repo, num = instance_id.rsplit('-', 1)
r = repo.split('__')[0]