sora c23665fed7 Scoring preflight: auto batch-pull for multi-image benches ...

Single-image benches keep fail-fast; >8 distinct images (swe's 500
per-instance) now pull concurrently with resume state inside the run
itself -- exactly like the old auto-pull behavior, just resilient:
scoring proceeds with whatever images landed, missing ones score 0
and only a total wipeout fails the bench. Ctrl+C-safe (state file),
network recovery resumes automatically on the next run.

Co-Authored-By: Claude <noreply@anthropic.com>

2026-09-17 09:38:16 +00:00

..

__init__.py

Fix images_for_samples export (swe prefetch import error)

2026-08-27 15:27:30 +00:00

base.py

LocalSandbox.exec: accept entrypoint kwarg (signature parity)

2026-09-17 07:24:48 +00:00

bg_prefetch.py

Comparison-driven fixes: MCQ choices in prompt + letter contract, Answer: suffix for QA, markdown answer cleaning, trivia_qa answer_phrase priority, DROP gold-as-alternatives (OR) official semantics, retry on 5xx, numpy/scipy compat

2026-08-24 15:48:42 +00:00

docker.py

Mirror chain trimmed to 6 mainstream sources; manual-load guidance

2026-09-17 05:53:09 +00:00

local.py

LocalSandbox.exec: accept entrypoint kwarg (signature parity)

2026-09-17 07:24:48 +00:00

prefetch.py

docker pull --progress=plain: docker 25+ hides layer progress when piped

2026-09-17 06:26:04 +00:00

pull_swe.py

Scoring preflight: auto batch-pull for multi-image benches

2026-09-17 09:38:16 +00:00