5c14d11852
- config env entries: bfcl_v3->bfcl_mock, tau2_bench->tau2_official;
the eval loop previously POPPED the env key and discarded it -- now
it feeds args.env (CLI --env still works as default/fallback)
- swe_bench_verified images: per-instance names are never published;
the official docker.io layout is repo-level BASE images
(swebench/sweb.eval.x86_64.{repo}) with per-instance images built
on top -- naming corrected (note: the swebench/* namespace is
currently blocked on every reachable CN mirror all the same)
Co-Authored-By: Claude <noreply@anthropic.com>
63 lines
1.0 KiB
YAML
63 lines
1.0 KiB
YAML
default:
|
|
temperature: 0.0
|
|
top_p: 1.0
|
|
stream: true
|
|
max_tokens: 32768
|
|
aime24:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 8192
|
|
aime25:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 8192
|
|
aime26:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 8192
|
|
hmmt26:
|
|
temperature: 1.0
|
|
repeats: 12
|
|
max_tokens: 8192
|
|
imo_answerbench:
|
|
temperature: 1.0
|
|
gpqa_diamond:
|
|
temperature: 1.0
|
|
repeats: 3
|
|
max_tokens: 8192
|
|
mmlu:
|
|
max_tokens: 8192
|
|
mmlu_pro:
|
|
max_tokens: 8192
|
|
cmmlu:
|
|
max_tokens: 8192
|
|
arc:
|
|
max_tokens: 8192
|
|
hellaswag:
|
|
max_tokens: 8192
|
|
winogrande:
|
|
max_tokens: 8192
|
|
simple_qa:
|
|
max_tokens: 8192
|
|
trivia_qa:
|
|
max_tokens: 8192
|
|
humaneval:
|
|
temperature: 1.0
|
|
repeats: 3
|
|
live_code_bench:
|
|
temperature: 1.0
|
|
longbench_v2:
|
|
max_tokens: 8192
|
|
max_input_tokens: 128000
|
|
openai_mrcr:
|
|
max_tokens: 8192
|
|
max_input_tokens: 128000
|
|
bfcl_v3:
|
|
max_tokens: 4096
|
|
env: bfcl_mock # agent 模式：工具调用轨迹 + 官方 AST 判分
|
|
general_fc:
|
|
max_tokens: 4096
|
|
tau2_bench:
|
|
max_tokens: 16384
|
|
env: tau2_official # agent 模式：官方引擎
|