This website requires JavaScript.
Explore
Help
Register
Sign In
Meta-Eval
/
EvalHarness
Watch
3
Star
0
Fork
0
You've already forked EvalHarness
Code
Issues
Pull Requests
Actions
Packages
Projects
Releases
Wiki
Activity
58
Commits
1
Branch
0
Tags
Commit Graph
3 Commits
Author
SHA1
Message
Date
sora
46bef7d3dd
dp4-flash 28-bench alignment: renderer plugin layer, exec_workers, gen_profiles, AIMD pool, BCB/LCB/bfcl/gfc judge fixes
...
Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 05:55:04 +00:00
sora
3d16ab9103
Add SWE-bench single-turn harness (patch apply + FAIL_TO_PASS in official sweb image, sh entries in docker sandbox), bigcodebench official all-libs image, docker default for humaneval/LCB, input truncation (max_input_chars) + context assembly (passage/context), API retry with backoff, judge thread-safe bridge, parquet blob integrity check, summary.csv in out-dir, sandbox prefetch CLI
2026-08-24 08:43:49 +00:00
sora
b2e7133b20
Add sandbox layer (docker exec hard-isolation + serve envs, refcounted acquire/release, atexit teardown, bind-mount sharing) and agent evaluation driver (message pump drive(), Trajectory, bfcl_mock env with official call-sequence scoring, mock:fc oracle); Deployer delegates to sandbox.serve_env; code_any extractor fixes indentation-stripping; CLI --env
2026-08-24 07:11:03 +00:00