80194e845e
The adapter dropped reasoning_effort/thinking from the payload, so the
middle rung of the ladder (es reference: full 98.3 / effort_low 94 /
no-think 82.3 on humaneval) was unreachable. Both keys now pass
through; --reasoning-effort {minimal,low,medium,high,max} overrides
the YAML, and config/effort_low.yaml mirrors default.yaml with
reasoning_effort: low for one-command low-thinking runs.
Probe on the endpoint: same question, default = 319 chars reasoning /
262 tok, low = 47 chars / 117 tok -- the server honors it.
Co-Authored-By: Claude <noreply@anthropic.com>
85 lines
1.6 KiB
YAML
85 lines
1.6 KiB
YAML
# 低思考档：全库参数同 default.yaml，仅追加 reasoning_effort: low
|
||
# 用法: evalharness eval run ... --config effort_low
|
||
default:
|
||
temperature: 0.0
|
||
top_p: 1.0
|
||
stream: true
|
||
max_tokens: 32768
|
||
reasoning_effort: low
|
||
aime24:
|
||
temperature: 1.0
|
||
repeats: 12
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
aime25:
|
||
temperature: 1.0
|
||
repeats: 12
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
aime26:
|
||
temperature: 1.0
|
||
repeats: 12
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
hmmt26:
|
||
temperature: 1.0
|
||
repeats: 12
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
imo_answerbench:
|
||
temperature: 1.0
|
||
reasoning_effort: low
|
||
gpqa_diamond:
|
||
temperature: 1.0
|
||
repeats: 3
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
mmlu:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
mmlu_pro:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
cmmlu:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
arc:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
hellaswag:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
winogrande:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
simple_qa:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
trivia_qa:
|
||
max_tokens: 8192
|
||
reasoning_effort: low
|
||
humaneval:
|
||
temperature: 1.0
|
||
repeats: 3
|
||
reasoning_effort: low
|
||
live_code_bench:
|
||
temperature: 1.0
|
||
reasoning_effort: low
|
||
longbench_v2:
|
||
max_tokens: 8192
|
||
max_input_tokens: 128000
|
||
reasoning_effort: low
|
||
openai_mrcr:
|
||
max_tokens: 8192
|
||
max_input_tokens: 128000
|
||
reasoning_effort: low
|
||
bfcl_v3:
|
||
max_tokens: 4096
|
||
reasoning_effort: low
|
||
general_fc:
|
||
max_tokens: 4096
|
||
reasoning_effort: low
|
||
tau2_bench:
|
||
max_tokens: 16384
|
||
reasoning_effort: low
|