1140 real predictions scored 0.0% because the recipe referenced
'bigcodebench-sandbox:latest' -- a name nothing builds and docker.io
does not have; every sample then burned 3 pull-retries (~200s each).
- image -> bigcodebench/bigcodebench-evaluate:latest (the official hub
image, same one evalscope uses)
- ensure_image() preflight in evaluate(): recipe-level AND sample-level
images are verified/pulled ONCE before any container runs; missing ->
seconds-fast bench failure with a fix hint instead of a silent 0.0%
- docker exec: 'Unable to find image'/'manifest unknown' class errors
are permanent -- no 3x retry amplification
Verified with a bogus image: preflight raises in one pull-attempt with
the fix hint.
Co-Authored-By: Claude <noreply@anthropic.com>
2949e10371