Meta-Eval/EvalHarness
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gen_profiles.yaml: YAML profiles (dp4-nothink, qwen3-es-parity, glm53-nothink, t1-short) — edit file to tune parameters, no code changes needed

Browse Source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-11 07:45:51 +00:00
parent 63ffd1b613
commit 471e0ca1a9
1 changed files with 31 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
gen_profiles.yaml Normal file
View File

@ -0,0 +1,31 @@
# Generation-parameter profiles (YAML).
# Usage: evalharness eval run <bench> --model ... --profile <name>
# Resolution: DatasetSpec.gen_config < profile.default < profile.<bench> < gen_kwargs
dp4-nothink:
default: {temperature: 0.0, max_tokens: 32768, top_p: 1.0}
qwen3-es-parity:
default: {temperature: 0.0, max_tokens: 32768}
simple_qa: {max_tokens: 1024}
hle: {max_tokens: 8192}
gpqa_diamond: {temperature: 1.0, max_tokens: 8192}
aime24: {temperature: 1.0, max_tokens: 8192}
aime25: {temperature: 1.0, max_tokens: 8192}
aime26: {temperature: 1.0, max_tokens: 8192}
hmmt26: {temperature: 1.0, max_tokens: 8192}
imo_answerbench: {temperature: 1.0, max_tokens: 8192}
glm53-nothink:
default: {temperature: 0.0, max_tokens: 8192, top_p: 1.0}
aime24: {temperature: 1.0, max_tokens: 8192}
aime25: {temperature: 1.0, max_tokens: 8192}
aime26: {temperature: 1.0, max_tokens: 8192}
hmmt26: {temperature: 1.0, max_tokens: 8192}
imo_answerbench: {temperature: 1.0, max_tokens: 8192}
gpqa_diamond: {temperature: 1.0, max_tokens: 8192}
humaneval: {temperature: 1.0, max_tokens: 32768}
live_code_bench: {temperature: 1.0, max_tokens: 32768}
t1-short:
default: {temperature: 1.0, max_tokens: 8192, top_p: 1.0}