## Overview

Score Summary 4 entries

Dataset | Model | Metric | Score | Num
--- | --- | --- | --- | ---
GPQA-Diamond | qwen-plus | mean_acc | 0.8000 | 10
GSM8K | qwen-plus | mean_acc | 1.0000 | 10
IFEval | qwen-plus | mean_prompt_level_strict | 0.9000 | 10
IQuiz | qwen-plus | mean_acc | 0.8500 | 20

Overall Score Chart
Score Distribution

## Results by Dataset

GPQA-Diamond
0.8000
Description

## Overview

GPQA (Graduate-Level Google-Proof Q&A) Diamond is a challenging benchmark of 198 multiple-choice questions written by domain experts in biology, physics, and chemistry. The questions are designed to be extremely difficult, requiring PhD-level expertise to answer correctly.

## Task Description

- Task Type: Expert-Level Multiple-Choice Q&A
- Input: Graduate-level science question with 4 choices
- Output: Single correct answer letter (A, B, C, or D)
- Domains: Biology, Physics, Chemistry

## Key Features

- 198 questions written and validated by domain PhD experts
- Questions are "Google-proof" - cannot be easily looked up
- Designed to test deep domain knowledge and reasoning
- Diamond subset represents the highest quality questions
- Average human expert accuracy ~65%, non-expert ~34%

## Evaluation Notes

- Default configuration uses 0-shot or 5-shot evaluation
- Supports Chain-of-Thought (CoT) prompting for improved reasoning
- Answer choices are randomly shuffled during evaluation
- Only uses train split (validation set is private)
- Challenging benchmark for measuring expert-level reasoning

## Properties

Property | Value
--- | ---
Benchmark Name | `gpqa_diamond`
Dataset ID | AI-ModelScope/gpqa_diamond
Paper | N/A
Tags | `Knowledge`, `MCQ`
Metrics | `acc`
Default Shots | 0-shot
Evaluation Split | `train`

## Data Statistics

Metric | Value
--- | ---
Total Samples | 198
Prompt Length (Mean) | 841.15 chars
Prompt Length (Min/Max) | 340 / 5845 chars

## Sample Example

Subset: `default`

    {
      "input": [
        {
          "id": "82b448a9",
          "content": "Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of A,B,C,D. Think step by step before answering.\n\nTwo quantum states wi ... [TRUNCATED] ...  and 10^-8 sec, respectively. We want to clearly distinguish these two energy levels. Which one of the following options could be their energy difference so that they can be clearly resolved?\n\n\nA) 10^-4 eV\nB) 10^-9 eV\nC) 10^-8 eV\nD) 10^-11 eV"
        }
      ],
      "choices": [
        "10^-4 eV",
        "10^-9 eV",
        "10^-8 eV",
        "10^-11 eV"
      ],
      "target": "A",
      "id": 0,
      "group_id": 0,
      "subset_key": "",
      "metadata": {
        "correct_answer": "10^-4 eV",
        "incorrect_answers": [
          "10^-11 eV",
          "10^-8 eV\n",
          "10^-9 eV"
        ]
      }
    }

Note: Some content was truncated for display.

## Prompt Template

Prompt Template:

    Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of {letters}. Think step by step before answering.

    {question}

    {choices}

## 概述

GPQA（Graduate-Level Google-Proof Q&A）Diamond 是一个极具挑战性的基准测试，包含 198 道由生物学、物理学和化学领域的专家编写的多项选择题。这些问题设计得极为困难，需要博士级别的专业知识才能正确作答。

## 任务描述

- 任务类型：专家级多项选择问答
- 输入：研究生水平的科学问题，附带 4 个选项
- 输出：单个正确答案字母（A、B、C 或 D）
- 领域：生物学、物理学、化学

## 主要特点

- 198 道题目均由相关领域的博士专家编写并验证
- 题目“无法通过 Google 轻易查到”——难以通过简单搜索获得答案
- 旨在测试深层次的领域知识与推理能力
- Diamond 子集代表了最高质量的问题
- 人类专家平均准确率约为 65%，非专家约为 34%

## 评估说明

- 默认配置使用 0-shot 或 5-shot 评估
- 支持思维链（Chain-of-Thought, CoT）提示以提升推理能力
- 评估过程中答案选项会随机打乱
- 仅使用训练集（验证集为私有）
- 是衡量专家级推理能力的高难度基准测试

## 属性

属性 | 值
--- | ---
基准测试名称 | `gpqa_diamond`
数据集 ID | AI-ModelScope/gpqa_diamond
论文 | N/A
标签 | `Knowledge`, `MCQ`
指标 | `acc`
默认示例数（Shots） | 0-shot
评估划分 | `train`

## 数据统计

指标 | 值
--- | ---
总样本数 | 198
提示词长度（平均） | 841.15 字符
提示词长度（最小/最大） | 340 / 5845 字符

## 样例示例

子集: `default`

    {
      "input": [
        {
          "id": "82b448a9",
          "content": "Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of A,B,C,D. Think step by step before answering.\n\nTwo quantum states wi ... [TRUNCATED] ...  and 10^-8 sec, respectively. We want to clearly distinguish these two energy levels. Which one of the following options could be their energy difference so that they can be clearly resolved?\n\n\nA) 10^-4 eV\nB) 10^-9 eV\nC) 10^-8 eV\nD) 10^-11 eV"
        }
      ],
      "choices": [
        "10^-4 eV",
        "10^-9 eV",
        "10^-8 eV",
        "10^-11 eV"
      ],
      "target": "A",
      "id": 0,
      "group_id": 0,
      "subset_key": "",
      "metadata": {
        "correct_answer": "10^-4 eV",
        "incorrect_answers": [
          "10^-11 eV",
          "10^-8 eV\n",
          "10^-9 eV"
        ]
      }
    }

注：部分内容因展示需要已被截断。

## 提示模板

提示模板：

    Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of {letters}. Think step by step before answering.

    {question}

    {choices}

Category | Subset | Metric | Score | Num
--- | --- | --- | --- | ---
default | default | mean_acc | 0.8000 | 10

GPQA-Diamond · Subset Scores
GSM8K
1.0000
Description

## Overview

GSM8K (Grade School Math 8K) is a high-quality dataset of 8.5K linguistically diverse grade school math word problems created by human problem writers. The dataset is specifically designed to evaluate and improve the multi-step mathematical reasoning capabilities of language models.

## Task Description

- Task Type: Mathematical Word Problem Solving
- Input: Natural language math word problem
- Output: Numerical answer derived through step-by-step reasoning
- Difficulty: Grade school level (2-8 reasoning steps required)

## Key Features

- Problems require basic arithmetic operations (addition, subtraction, multiplication, division)
- Solutions involve 2 to 8 sequential reasoning steps
- High linguistic diversity in problem formulations
- Human-written problems ensuring natural language quality
- Clear numerical answers for objective evaluation

## Evaluation Notes

- Default configuration uses 4-shot examples with Chain-of-Thought (CoT) prompting
- Answers should be formatted within `\boxed{}` for proper extraction
- The metric extracts numerical values for accuracy comparison
- Supports both zero-shot and few-shot evaluation modes

## Properties

Property | Value
--- | ---
Benchmark Name | `gsm8k`
Dataset ID | AI-ModelScope/gsm8k
Paper | Paper
Tags | `Math`, `Reasoning`
Metrics | `acc`
Default Shots | 4-shot
Evaluation Split | `test`
Train Split | `train`

## Data Statistics

Metric | Value
--- | ---
Total Samples | 1,319
Prompt Length (Mean) | 1966.87 chars
Prompt Length (Min/Max) | 1800 / 2575 chars

## Sample Example

Subset: `main`

    {
      "input": [
        {
          "id": "0bc7f97b",
          "content": "Here are some examples of how to solve similar problems:\n\nNatalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?\n\nReasoning:\nNatalia sold 48/ ... [TRUNCATED] ... ds every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?\nPlease reason step by step, and put your final answer within \\boxed{}."
        }
      ],
      "target": "18",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "reasoning": "Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.\nShe makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market."
      }
    }

Note: Some content was truncated for display.

## Prompt Template

Prompt Template:

    {question}
    Please reason step by step, and put your final answer within \boxed{{}}.

Few-shot Template

    Here are some examples of how to solve similar problems:

    {fewshot}

    {question}
    Please reason step by step, and put your final answer within \boxed{{}}.

## 概述

GSM8K（Grade School Math 8K）是一个高质量的数据集，包含由人工编写者创作的 8.5K 道语言多样化的中小学数学应用题。该数据集专门用于评估和提升语言模型的多步数学推理能力。

## 任务描述

- 任务类型：数学应用题求解
- 输入：自然语言描述的数学应用题
- 输出：通过逐步推理得出的数值答案
- 难度：小学水平（需 2–8 步推理）

## 主要特点

- 问题涉及基本算术运算（加、减、乘、除）
- 解答过程包含 2 到 8 个连续推理步骤
- 问题表述具有高度的语言多样性
- 人工编写确保自然语言质量
- 答案为明确的数值，便于客观评估

## 评估说明

- 默认配置使用 4-shot 示例并结合思维链（Chain-of-Thought, CoT）提示
- 答案应使用 `\boxed{}` 格式包裹以便正确提取
- 评估指标会提取数值进行准确率比较
- 支持零样本（zero-shot）和少样本（few-shot）两种评估模式

## 属性

属性 | 值
--- | ---
基准测试名称 | `gsm8k`
数据集ID | AI-ModelScope/gsm8k
论文 | Paper
标签 | `Math`, `Reasoning`
指标 | `acc`
默认样本数 | 4-shot
评估划分 | `test`
训练划分 | `train`

## 数据统计

指标 | 值
--- | ---
总样本数 | 1,319
提示词长度（平均） | 1966.87 字符
提示词长度（最小/最大） | 1800 / 2575 字符

## 样例示例

子集: `main`

    {
      "input": [
        {
          "id": "0bc7f97b",
          "content": "Here are some examples of how to solve similar problems:\n\nNatalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?\n\nReasoning:\nNatalia sold 48/ ... [TRUNCATED] ... ds every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?\nPlease reason step by step, and put your final answer within \\boxed{}."
        }
      ],
      "target": "18",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "reasoning": "Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.\nShe makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market."
      }
    }

注：部分内容因展示需要已被截断。

## 提示模板

提示模板：

    {question}
    Please reason step by step, and put your final answer within \boxed{{}}.

少样本模板

    Here are some examples of how to solve similar problems:

    {fewshot}

    {question}
    Please reason step by step, and put your final answer within \boxed{{}}.

Category | Subset | Metric | Score | Num
--- | --- | --- | --- | ---
default | main | mean_acc | 1.0000 | 10

GSM8K · Subset Scores
IFEval
0.9000
Description

## Overview

IFEval (Instruction-Following Eval) is a benchmark for evaluating how well language models follow explicit, verifiable instructions. It contains prompts with specific formatting, content, or structural requirements that can be objectively verified.

## Task Description

- Task Type: Instruction Following Evaluation
- Input: Prompts with explicit, verifiable constraints
- Output: Response that follows all specified instructions
- Constraint Types: Format, length, keywords, structure, etc.

## Key Features

- ~500 prompts with 25 types of verifiable instructions
- Instructions are objectively checkable (not subjective)
- Examples: "write exactly 3 paragraphs", "include the word X", "use bullet points"
- Tests instruction comprehension and compliance
- No ambiguity in evaluation criteria

## Evaluation Notes

- Default configuration uses 0-shot evaluation
- Four metrics available:
- `prompt_level_strict`: All instructions in prompt must be followed
- `prompt_level_loose`: Some tolerance for minor deviations
- `inst_level_strict`: Per-instruction accuracy (strict)
- `inst_level_loose`: Per-instruction accuracy (loose)
- `prompt_level_strict` is the primary metric
- Automatic verification of instruction compliance

## Properties

Property | Value
--- | ---
Benchmark Name | `ifeval`
Dataset ID | opencompass/ifeval
Paper | N/A
Tags | `InstructionFollowing`
Metrics | `prompt_level_strict`, `inst_level_strict`, `prompt_level_loose`, `inst_level_loose`
Default Shots | 0-shot
Evaluation Split | `train`

## Data Statistics

Metric | Value
--- | ---
Total Samples | 541
Prompt Length (Mean) | 210.75 chars
Prompt Length (Min/Max) | 53 / 1858 chars

## Sample Example

Subset: `default`

    {
      "input": [
        {
          "id": "cb71907f",
          "content": "Write a 300+ word summary of the wikipedia page \"https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli\". Do not use any commas and highlight at least 3 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*."
        }
      ],
      "target": "",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "key": 1000,
        "prompt": "Write a 300+ word summary of the wikipedia page \"https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli\". Do not use any commas and highlight at least 3 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*.",
        "instruction_id_list": [
          "punctuation:no_comma",
          "detectable_format:number_highlighted_sections",
          "length_constraints:number_words"
        ],
        "kwargs": [
          {
            "num_highlights": null,
            "relation": null,
            "num_words": null,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          },
          {
            "num_highlights": 3,
            "relation": null,
            "num_words": null,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          },
          {
            "num_highlights": null,
            "relation": "at least",
            "num_words": 300,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          }
        ]
      }
    }

## Prompt Template

No prompt template defined.

## 概述

IFEval（Instruction-Following Eval）是一个用于评估语言模型遵循明确、可验证指令能力的基准测试。它包含一系列带有特定格式、内容或结构要求的提示（prompts），这些要求可以被客观地验证。

## 任务描述

- 任务类型：指令遵循评估
- 输入：包含明确、可验证约束条件的提示
- 输出：完全遵循所有指定指令的响应
- 约束类型：格式、长度、关键词、结构等

## 主要特点

- 包含约500个提示，涵盖25种可验证的指令类型
- 指令可被客观检查（非主观判断）
- 示例：“写恰好3段文字”、“包含单词X”、“使用项目符号列表”
- 测试模型对指令的理解与遵守能力
- 评估标准无歧义

## 评估说明

- 默认配置采用 0-shot 评估方式
- 提供四种评估指标：
- `prompt_level_strict`：提示中的所有指令都必须被严格遵循
- `prompt_level_loose`：允许对轻微偏差有一定容忍度
- `inst_level_strict`：按每条指令计算准确率（严格）
- `inst_level_loose`：按每条指令计算准确率（宽松）
- 主要指标为 `prompt_level_strict`
- 支持自动验证指令遵循情况

## 属性

属性 | 值
--- | ---
基准测试名称 | `ifeval`
数据集ID | opencompass/ifeval
论文 | N/A
标签 | `InstructionFollowing`
指标 | `prompt_level_strict`, `inst_level_strict`, `prompt_level_loose`, `inst_level_loose`
默认示例数（Shots） | 0-shot
评估划分 | `train`

## 数据统计

指标 | 值
--- | ---
总样本数 | 541
提示词长度（平均） | 210.75 字符
提示词长度（最小/最大） | 53 / 1858 字符

## 样例示例

子集：`default`

    {
      "input": [
        {
          "id": "cb71907f",
          "content": "Write a 300+ word summary of the wikipedia page \"https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli\". Do not use any commas and highlight at least 3 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*."
        }
      ],
      "target": "",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "key": 1000,
        "prompt": "Write a 300+ word summary of the wikipedia page \"https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli\". Do not use any commas and highlight at least 3 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*.",
        "instruction_id_list": [
          "punctuation:no_comma",
          "detectable_format:number_highlighted_sections",
          "length_constraints:number_words"
        ],
        "kwargs": [
          {
            "num_highlights": null,
            "relation": null,
            "num_words": null,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          },
          {
            "num_highlights": 3,
            "relation": null,
            "num_words": null,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          },
          {
            "num_highlights": null,
            "relation": "at least",
            "num_words": 300,
            "num_placeholders": null,
            "prompt_to_repeat": null,
            "num_bullets": null,
            "section_spliter": null,
            "num_sections": null,
            "capital_relation": null,
            "capital_frequency": null,
            "keywords": null,
            "num_paragraphs": null,
            "language": null,
            "let_relation": null,
            "letter": null,
            "let_frequency": null,
            "end_phrase": null,
            "forbidden_words": null,
            "keyword": null,
            "frequency": null,
            "num_sentences": null,
            "postscript_marker": null,
            "first_word": null,
            "nth_paragraph": null
          }
        ]
      }
    }

## 提示模板

未定义提示模板。

Category | Subset | Metric | Score | Num
--- | --- | --- | --- | ---
default | default | mean_prompt_level_strict | 0.9000 | 10

IFEval · Subset Scores
IQuiz
0.8500
Description

## Overview

IQuiz is a Chinese benchmark for evaluating AI models on intelligence quotient (IQ) and emotional quotient (EQ) questions. It tests logical reasoning, pattern recognition, and social-emotional understanding through multiple-choice questions.

## Task Description

- Task Type: Multiple-Choice Question Answering
- Input: Question in Chinese with multiple choice options
- Output: Selected answer with explanation (Chain-of-Thought)
- Language: Chinese

## Key Features

- Dual evaluation of IQ and EQ capabilities
- Chinese-language cognitive assessment
- Multiple difficulty levels
- Requires explanation alongside answer selection
- Tests logical reasoning and emotional understanding

## Evaluation Notes

- Default configuration uses 0-shot evaluation
- Primary metric: Accuracy
- Subsets: IQ (logical reasoning) and EQ (emotional intelligence)
- Uses Chinese Chain-of-Thought prompt template
- Evaluates on test split
- Metadata includes difficulty level information

## Properties

Property | Value
--- | ---
Benchmark Name | `iquiz`
Dataset ID | AI-ModelScope/IQuiz
Paper | N/A
Tags | `Chinese`, `Knowledge`, `MCQ`
Metrics | `acc`
Default Shots | 0-shot
Evaluation Split | `test`

## Data Statistics

Metric | Value
--- | ---
Total Samples | 120
Prompt Length (Mean) | 248.31 chars
Prompt Length (Min/Max) | 146 / 394 chars

Per-Subset Statistics:

Subset | Samples | Prompt Mean | Prompt Min | Prompt Max
--- | --- | --- | --- | ---
`IQ` | 40 | 194.5 | 146 | 323
`EQ` | 80 | 275.21 | 219 | 394

## Sample Example

Subset: `IQ`

    {
      "input": [
        {
          "id": "748f2700",
          "content": "回答下面的单项选择题，请选出其中的正确答案。你的回答的最后一行应该是这样的格式：\"答案：[LETTER]\"（不带引号），其中 [LETTER] 是 A,B,C,D 中的一个。请在回答前进行一步步思考。\n\n问题：天气预报说本周星期三会下雨，昨天果然下雨了，今天星期几？\n选项：\nA) 星期一\nB) 星期二\nC) 星期三\nD) 星期四\n"
        }
      ],
      "choices": [
        "星期一",
        "星期二",
        "星期三",
        "星期四"
      ],
      "target": "D",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "level": 1
      }
    }

## Prompt Template

Prompt Template:

    回答下面的单项选择题，请选出其中的正确答案。你的回答的最后一行应该是这样的格式："答案：[LETTER]"（不带引号），其中 [LETTER] 是 {letters} 中的一个。请在回答前进行一步步思考。

    问题：{question}
    选项：
    {choices}

## 概述

IQuiz 是一个中文基准测试，用于评估 AI 模型在智商（IQ）和情商（EQ）问题上的表现。它通过多项选择题来测试逻辑推理、模式识别以及社会情感理解能力。

## 任务描述

- 任务类型：多项选择题问答
- 输入：中文问题及多个选项
- 输出：所选答案及解释（思维链，Chain-of-Thought）
- 语言：中文

## 主要特点

- 同时评估 IQ 和 EQ 能力
- 中文认知能力测评
- 多种难度级别
- 要求在选择答案的同时提供解释
- 测试逻辑推理与情感理解能力

## 评估说明

- 默认配置使用 0-shot 评估
- 主要指标：准确率（Accuracy）
- 子集：IQ（逻辑推理）和 EQ（情感智能）
- 使用中文思维链（Chain-of-Thought）提示模板
- 在 test 数据划分上进行评估
- 元数据包含难度级别信息

## 属性

属性 | 值
--- | ---
基准测试名称 | `iquiz`
数据集 ID | AI-ModelScope/IQuiz
论文 | N/A
标签 | `Chinese`, `Knowledge`, `MCQ`
指标 | `acc`
默认示例数 | 0-shot
评估数据划分 | `test`

## 数据统计

指标 | 值
--- | ---
总样本数 | 120
提示词长度（平均） | 248.31 字符
提示词长度（最小/最大） | 146 / 394 字符

各子集统计数据：

子集 | 样本数 | 提示平均长度 | 提示最小长度 | 提示最大长度
--- | --- | --- | --- | ---
`IQ` | 40 | 194.5 | 146 | 323
`EQ` | 80 | 275.21 | 219 | 394

## 样例示例

子集: `IQ`

    {
      "input": [
        {
          "id": "748f2700",
          "content": "回答下面的单项选择题，请选出其中的正确答案。你的回答的最后一行应该是这样的格式：\"答案：[LETTER]\"（不带引号），其中 [LETTER] 是 A,B,C,D 中的一个。请在回答前进行一步步思考。\n\n问题：天气预报说本周星期三会下雨，昨天果然下雨了，今天星期几？\n选项：\nA) 星期一\nB) 星期二\nC) 星期三\nD) 星期四\n"
        }
      ],
      "choices": [
        "星期一",
        "星期二",
        "星期三",
        "星期四"
      ],
      "target": "D",
      "id": 0,
      "group_id": 0,
      "metadata": {
        "level": 1
      }
    }

## 提示模板

提示模板：

    回答下面的单项选择题，请选出其中的正确答案。你的回答的最后一行应该是这样的格式："答案：[LETTER]"（不带引号），其中 [LETTER] 是 {letters} 中的一个。请在回答前进行一步步思考。

    问题：{question}
    选项：
    {choices}

Category | Subset | Metric | Score | Num
--- | --- | --- | --- | ---
default | IQ | mean_acc | 1.0000 | 10
default | EQ | mean_acc | 0.7000 | 10

IQuiz · Subset Scores