Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

max_input_tokens 128000→120000 (GLM gateway unstable on very long inputs, 30s timeout at 128k)

Browse Source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-11 09:22:55 +00:00
parent ebc51de274
commit b75fbc1a12
1 changed files with 2 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
evalharness/config/default.yaml
View File

@ -3,73 +3,52 @@ default:
top_p: 1.0 top_p: 1.0
stream: true stream: true
max_tokens: 32768 max_tokens: 32768
aime24: aime24:
temperature: 1.0 temperature: 1.0
repeats: 12 repeats: 12
aime25: aime25:
temperature: 1.0 temperature: 1.0
repeats: 12 repeats: 12
aime26: aime26:
temperature: 1.0 temperature: 1.0
repeats: 12 repeats: 12
hmmt26: hmmt26:
temperature: 1.0 temperature: 1.0
repeats: 12 repeats: 12
imo_answerbench: imo_answerbench:
temperature: 1.0 temperature: 1.0
gpqa_diamond: gpqa_diamond:
temperature: 1.0 temperature: 1.0
max_tokens: 8192 max_tokens: 8192
mmlu: mmlu:
max_tokens: 8192 max_tokens: 8192
mmlu_pro: mmlu_pro:
max_tokens: 8192 max_tokens: 8192
cmmlu: cmmlu:
max_tokens: 8192 max_tokens: 8192
arc: arc:
max_tokens: 8192 max_tokens: 8192
hellaswag: hellaswag:
max_tokens: 8192 max_tokens: 8192
winogrande: winogrande:
max_tokens: 8192 max_tokens: 8192
simple_qa: simple_qa:
max_tokens: 8192 max_tokens: 8192
trivia_qa: trivia_qa:
max_tokens: 8192 max_tokens: 8192
humaneval: humaneval:
temperature: 1.0 temperature: 1.0
live_code_bench: live_code_bench:
temperature: 1.0 temperature: 1.0
longbench_v2: longbench_v2:
max_tokens: 8192 max_tokens: 8192
max_input_tokens: 128000 max_input_tokens: 120000
openai_mrcr: openai_mrcr:
max_tokens: 8192 max_tokens: 8192
max_input_tokens: 128000 max_input_tokens: 120000
bfcl_v3: bfcl_v3:
max_tokens: 4096 max_tokens: 4096
general_fc: general_fc:
max_tokens: 4096 max_tokens: 4096
tau2_bench: tau2_bench:
max_tokens: 16384 max_tokens: 16384