{
  "meta": {
    "pretty_name": "GDPval",
    "dataset_id": "openai-mirror/gdpval",
    "paper_url": null,
    "tags": [
      "Agent",
      "Knowledge",
      "MultiTurn"
    ],
    "metrics": [
      "submission_ready"
    ],
    "few_shot_num": 0,
    "eval_split": "train",
    "train_split": "",
    "subset_list": [
      "default"
    ],
    "description": "\n## Overview\n\nGDPval evaluates whether models can complete realistic economically valuable work tasks and produce requested\ndeliverable files. This adapter targets OpenAI's public 220-task gold subset mirrored on ModelScope as\n`openai-mirror/gdpval`.\n\n## Task Description\n\n- **Task Type**: Agentic professional work / deliverable generation\n- **Input**: A workplace-style task prompt, optionally with reference files\n- **Output**: Final response text and requested files under `deliverable_files/`\n- **Dataset**: OpenAI public GDPval gold subset with 220 tasks\n\n## Key Features\n\n- Uses the native EvalScope `AgentLoopAdapter` with bash and Python execution tools.\n- Loads records and reference files from ModelScope by default.\n- Mounts selected reference files read-only into the sandbox under `/reference_files`.\n- Extracts files written to `deliverable_files/` before sandbox teardown.\n- Generates a GDPval submission package with `deliverable_text` and `deliverable_files` columns.\n\n## Evaluation Notes\n\n- The default Docker image is `evalscope/gdpval:latest` and is built automatically from the bundled Dockerfile when\n  missing. Set `extra_params.auto_build_docker_image=false` to require a pre-built image, or override\n  `extra_params.docker_image`.\n- `submission_ready` is a local readiness metric: it is 1 when the model produced final text or at least one\n  deliverable file. It is not an official GDPval quality score.\n- EvalScope does not run a local GDPval judge. Use the exported submission package with OpenAI's official GDPval judge\n  to obtain quality scores.\n- Full document/spreadsheet/slide quality depends on the GDPval runtime image. Thin Python images are useful only for\n  plumbing smoke tests.\n\n## Scoring and Submission\n\n- EvalScope writes a local submission folder under the reports directory.\n- The submission contains `deliverable_text` and `deliverable_files` fields in the GDPval dataset format.\n- Official GDPval grading is external. Run OpenAI's official GDPval judge on the exported submission package.\n",
    "prompt_template": "{question}",
    "system_prompt": "",
    "few_shot_prompt_template": "",
    "aggregation": "mean",
    "extra_params": {
      "max_steps": {
        "type": "int",
        "description": "Maximum number of agent steps per sample.",
        "value": 250
      },
      "command_timeout": {
        "type": "float",
        "description": "Default per-command timeout in seconds.",
        "value": 180.0
      },
      "docker_image": {
        "type": "str",
        "description": "Docker image used as the per-sample sandbox.",
        "value": "evalscope/gdpval:latest"
      },
      "auto_build_docker_image": {
        "type": "bool",
        "description": "Automatically build the default GDPval Docker image if it is missing locally.",
        "value": true
      },
      "network_enabled": {
        "type": "bool",
        "description": "Allow the sandbox to access the network.",
        "value": true
      },
      "download_reference_files": {
        "type": "bool",
        "description": "Download each selected sample reference file from the dataset hub before inference.",
        "value": true
      }
    },
    "sandbox_config": {},
    "category": "agent"
  },
  "statistics": {
    "total_samples": 220,
    "subset_stats": [
      {
        "name": "default",
        "sample_count": 220,
        "prompt_length_mean": 2742.59,
        "prompt_length_min": 1058,
        "prompt_length_max": 7160,
        "prompt_length_std": 1013.03,
        "target_length_mean": null
      }
    ],
    "prompt_length": {
      "mean": 2742.59,
      "min": 1058,
      "max": 7160,
      "std": 1013.03
    },
    "target_length_mean": null,
    "computed_at": "2026-06-30T18:01:44.268190"
  },
  "sample_example": {
    "data": {
      "input": [
        {
          "id": "3315415d",
          "content": "You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\n\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 2069 chars] ...  folder named `deliverable_files` in the sandbox working directory.\nWe will grade your final message as part of the deliverable, but requested documents, spreadsheets, slides, media,\nor archives should be actual files in `deliverable_files`.\n"
        }
      ],
      "target": "",
      "id": 0,
      "group_id": 0,
      "tools": [
        {
          "name": "bash",
          "description": "Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.",
          "parameters": {
            "properties": {
              "command": {
                "type": "string",
                "description": "The bash command to execute."
              },
              "timeout": {
                "type": "number",
                "description": "Maximum execution time in seconds (default: 60).",
                "default": 60
              }
            },
            "required": [
              "command"
            ]
          }
        },
        {
          "name": "python_exec",
          "description": "Execute Python source code inside the sandbox environment. Returns stdout and stderr output.",
          "parameters": {
            "properties": {
              "code": {
                "type": "string",
                "description": "Python source code to execute."
              },
              "timeout": {
                "type": "number",
                "description": "Maximum execution time in seconds (default: 60).",
                "default": 60
              }
            },
            "required": [
              "code"
            ]
          }
        }
      ],
      "metadata": {
        "task_id": "83d10b06-26d1-4636-a32c-23f92c57f30b",
        "sector": "Professional, Scientific, and Technical Services",
        "occupation": "Accountants and Auditors",
        "prompt": "You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\n\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 1526 chars] ... across all Divisions and sub-Divisions.\n\n4. Create a new spreadsheet titled ‘Sample’:\n- Tab 1: Selected sample, copied from the original ‘Population’ sheet, with selected rows marked in column K.\n- Tab 2: Workings for sample size calculation.",
        "reference_files": [
          "reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx"
        ],
        "reference_file_urls": [
          "https://huggingface.co/datasets/openai/gdpval/resolve/main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx"
        ],
        "reference_file_hf_uris": [
          "hf://datasets/openai/gdpval@main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx"
        ],
        "reference_paths": [
          "reference_files/Population v2.xlsx"
        ],
        "sandbox_reference_paths": [
          "/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx"
        ],
        "rubric_pretty": "[+2] The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\n\n[+2] The workbook contains a worksheet named exactly 'Sample Size Calculation' (case-insensitive, ignoring surrounding spaces ... [TRUNCATED 4861 chars] ... ntage changes (e.g., |J| ≥ 100%) are made easily identifiable (such as by a separate flag, note, or conditional formatting).\n\n[+1] The first worksheet is named 'Sample' (case-insensitive).\n\n[+5] Overall formatting and style of the deliverable",
        "rubric_json": "[{\"score\": 2, \"criterion\": \"The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\", \"required\": null, \"rubric_item_id\": \"1d43f1eb-4011-47ac-8ad7-a3c467639a6a\", \"author_type\": \"human\", \" ... [TRUNCATED 11817 chars] ... ull}, {\"score\": 5, \"criterion\": \"Overall formatting and style of the deliverable\", \"required\": null, \"rubric_item_id\": \"a64588ed-db04-4b8b-b3b8-3674ddcf10d1\", \"author_type\": \"human\", \"tags\": [\"true\"], \"read_only\": null, \"form_content\": null}]",
        "dataset_id": "openai-mirror/gdpval",
        "dataset_hub": "modelscope"
      }
    },
    "subset": "default",
    "truncated": false
  },
  "readme": {
    "en": "# GDPval\n\n\n## Overview\n\nGDPval evaluates whether models can complete realistic economically valuable work tasks and produce requested\ndeliverable files. This adapter targets OpenAI's public 220-task gold subset mirrored on ModelScope as\n`openai-mirror/gdpval`.\n\n## Task Description\n\n- **Task Type**: Agentic professional work / deliverable generation\n- **Input**: A workplace-style task prompt, optionally with reference files\n- **Output**: Final response text and requested files under `deliverable_files/`\n- **Dataset**: OpenAI public GDPval gold subset with 220 tasks\n\n## Key Features\n\n- Uses the native EvalScope `AgentLoopAdapter` with bash and Python execution tools.\n- Loads records and reference files from ModelScope by default.\n- Mounts selected reference files read-only into the sandbox under `/reference_files`.\n- Extracts files written to `deliverable_files/` before sandbox teardown.\n- Generates a GDPval submission package with `deliverable_text` and `deliverable_files` columns.\n\n## Evaluation Notes\n\n- The default Docker image is `evalscope/gdpval:latest` and is built automatically from the bundled Dockerfile when\n  missing. Set `extra_params.auto_build_docker_image=false` to require a pre-built image, or override\n  `extra_params.docker_image`.\n- `submission_ready` is a local readiness metric: it is 1 when the model produced final text or at least one\n  deliverable file. It is not an official GDPval quality score.\n- EvalScope does not run a local GDPval judge. Use the exported submission package with OpenAI's official GDPval judge\n  to obtain quality scores.\n- Full document/spreadsheet/slide quality depends on the GDPval runtime image. Thin Python images are useful only for\n  plumbing smoke tests.\n\n## Scoring and Submission\n\n- EvalScope writes a local submission folder under the reports directory.\n- The submission contains `deliverable_text` and `deliverable_files` fields in the GDPval dataset format.\n- Official GDPval grading is external. Run OpenAI's official GDPval judge on the exported submission package.\n\n\n## Properties\n\n| Property | Value |\n|----------|-------|\n| **Benchmark Name** | `gdpval` |\n| **Dataset ID** | [openai-mirror/gdpval](https://modelscope.cn/datasets/openai-mirror/gdpval/summary) |\n| **Paper** | N/A |\n| **Tags** | `Agent`, `Knowledge`, `MultiTurn` |\n| **Metrics** | `submission_ready` |\n| **Default Shots** | 0-shot |\n| **Evaluation Split** | `train` |\n\n\n## Data Statistics\n\n| Metric | Value |\n|--------|-------|\n| Total Samples | 220 |\n| Prompt Length (Mean) | 2742.59 chars |\n| Prompt Length (Min/Max) | 1058 / 7160 chars |\n\n## Sample Example\n\n**Subset**: `default`\n\n```json\n{\n  \"input\": [\n    {\n      \"id\": \"3315415d\",\n      \"content\": \"You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\\n\\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 2069 chars] ...  folder named `deliverable_files` in the sandbox working directory.\\nWe will grade your final message as part of the deliverable, but requested documents, spreadsheets, slides, media,\\nor archives should be actual files in `deliverable_files`.\\n\"\n    }\n  ],\n  \"target\": \"\",\n  \"id\": 0,\n  \"group_id\": 0,\n  \"tools\": [\n    {\n      \"name\": \"bash\",\n      \"description\": \"Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.\",\n      \"parameters\": {\n        \"properties\": {\n          \"command\": {\n            \"type\": \"string\",\n            \"description\": \"The bash command to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"command\"\n        ]\n      }\n    },\n    {\n      \"name\": \"python_exec\",\n      \"description\": \"Execute Python source code inside the sandbox environment. Returns stdout and stderr output.\",\n      \"parameters\": {\n        \"properties\": {\n          \"code\": {\n            \"type\": \"string\",\n            \"description\": \"Python source code to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"code\"\n        ]\n      }\n    }\n  ],\n  \"metadata\": {\n    \"task_id\": \"83d10b06-26d1-4636-a32c-23f92c57f30b\",\n    \"sector\": \"Professional, Scientific, and Technical Services\",\n    \"occupation\": \"Accountants and Auditors\",\n    \"prompt\": \"You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\\n\\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 1526 chars] ... across all Divisions and sub-Divisions.\\n\\n4. Create a new spreadsheet titled ‘Sample’:\\n- Tab 1: Selected sample, copied from the original ‘Population’ sheet, with selected rows marked in column K.\\n- Tab 2: Workings for sample size calculation.\",\n    \"reference_files\": [\n      \"reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx\"\n    ],\n    \"reference_file_urls\": [\n      \"https://huggingface.co/datasets/openai/gdpval/resolve/main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx\"\n    ],\n    \"reference_file_hf_uris\": [\n      \"hf://datasets/openai/gdpval@main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx\"\n    ],\n    \"reference_paths\": [\n      \"reference_files/Population v2.xlsx\"\n    ],\n    \"sandbox_reference_paths\": [\n      \"/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx\"\n    ],\n    \"rubric_pretty\": \"[+2] The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\\n\\n[+2] The workbook contains a worksheet named exactly 'Sample Size Calculation' (case-insensitive, ignoring surrounding spaces ... [TRUNCATED 4861 chars] ... ntage changes (e.g., |J| ≥ 100%) are made easily identifiable (such as by a separate flag, note, or conditional formatting).\\n\\n[+1] The first worksheet is named 'Sample' (case-insensitive).\\n\\n[+5] Overall formatting and style of the deliverable\",\n    \"rubric_json\": \"[{\\\"score\\\": 2, \\\"criterion\\\": \\\"The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\\\", \\\"required\\\": null, \\\"rubric_item_id\\\": \\\"1d43f1eb-4011-47ac-8ad7-a3c467639a6a\\\", \\\"author_type\\\": \\\"human\\\", \\\" ... [TRUNCATED 11817 chars] ... ull}, {\\\"score\\\": 5, \\\"criterion\\\": \\\"Overall formatting and style of the deliverable\\\", \\\"required\\\": null, \\\"rubric_item_id\\\": \\\"a64588ed-db04-4b8b-b3b8-3674ddcf10d1\\\", \\\"author_type\\\": \\\"human\\\", \\\"tags\\\": [\\\"true\\\"], \\\"read_only\\\": null, \\\"form_content\\\": null}]\",\n    \"dataset_id\": \"openai-mirror/gdpval\",\n    \"dataset_hub\": \"modelscope\"\n  }\n}\n```\n\n## Prompt Template\n\n**Prompt Template:**\n```text\n{question}\n```\n\n## Extra Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `max_steps` | `int` | `250` | Maximum number of agent steps per sample. |\n| `command_timeout` | `float` | `180.0` | Default per-command timeout in seconds. |\n| `docker_image` | `str` | `evalscope/gdpval:latest` | Docker image used as the per-sample sandbox. |\n| `auto_build_docker_image` | `bool` | `True` | Automatically build the default GDPval Docker image if it is missing locally. |\n| `network_enabled` | `bool` | `True` | Allow the sandbox to access the network. |\n| `download_reference_files` | `bool` | `True` | Download each selected sample reference file from the dataset hub before inference. |\n\n## Usage\n\n### Using CLI\n\n```bash\nevalscope eval \\\n    --model YOUR_MODEL \\\n    --api-url OPENAI_API_COMPAT_URL \\\n    --api-key EMPTY_TOKEN \\\n    --datasets gdpval \\\n    --limit 10  # Remove this line for formal evaluation\n```\n\n### Using Python\n\n```python\nfrom evalscope import run_task\nfrom evalscope.config import TaskConfig\n\ntask_cfg = TaskConfig(\n    model='YOUR_MODEL',\n    api_url='OPENAI_API_COMPAT_URL',\n    api_key='EMPTY_TOKEN',\n    datasets=['gdpval'],\n    dataset_args={\n        'gdpval': {\n            # extra_params: {}  # uses default extra parameters\n        }\n    },\n    limit=10,  # Remove this line for formal evaluation\n)\n\nrun_task(task_cfg=task_cfg)\n```\n\n\n",
    "zh": "# GDPval\n\n\n## 概述\n\nGDPval 用于评估模型是否能够完成具有现实经济价值的工作任务，并生成所要求的交付文件。该适配器针对 OpenAI 公开的 220 个任务的 gold 子集，该子集在 ModelScope 上镜像为 `openai-mirror/gdpval`。\n\n## 任务描述\n\n- **任务类型**：代理式专业工作 / 交付物生成\n- **输入**：职场风格的任务提示，可选附带参考文件\n- **输出**：最终回复文本及位于 `deliverable_files/` 目录下的请求文件\n- **数据集**：OpenAI 公开的 GDPval gold 子集，包含 220 个任务\n\n## 主要特性\n\n- 使用原生 EvalScope 的 `AgentLoopAdapter`，支持 bash 和 Python 执行工具。\n- 默认从 ModelScope 加载记录和参考文件。\n- 将选定的参考文件以只读方式挂载到沙箱中的 `/reference_files` 目录下。\n- 在沙箱销毁前提取写入 `deliverable_files/` 的文件。\n- 生成符合 GDPval 格式的提交包，包含 `deliverable_text` 和 `deliverable_files` 列。\n\n## 评估说明\n\n- 默认 Docker 镜像为 `evalscope/gdpval:latest`，若本地缺失则会根据内置的 Dockerfile 自动构建。设置 `extra_params.auto_build_docker_image=false` 可强制使用预构建镜像，或通过 `extra_params.docker_image` 覆盖默认镜像。\n- `submission_ready` 是一个本地就绪指标：当模型生成了最终文本或至少一个交付文件时，其值为 1。该指标并非官方 GDPval 质量评分。\n- EvalScope 不运行本地 GDPval 评判器。请使用导出的提交包配合 OpenAI 官方 GDPval 评判器获取质量评分。\n- 文档/电子表格/幻灯片等完整文件的质量依赖于 GDPval 运行时镜像。轻量级 Python 镜像仅适用于基础流程的冒烟测试。\n\n## 评分与提交\n\n- EvalScope 会在报告目录下写入一个本地提交文件夹。\n- 提交内容包含符合 GDPval 数据集格式的 `deliverable_text` 和 `deliverable_files` 字段。\n- 官方 GDPval 评分需在外部进行。请对导出的提交包运行 OpenAI 官方 GDPval 评判器。\n\n## 属性\n\n| 属性 | 值 |\n|----------|-------|\n| **基准测试名称** | `gdpval` |\n| **数据集ID** | [openai-mirror/gdpval](https://modelscope.cn/datasets/openai-mirror/gdpval/summary) |\n| **论文** | N/A |\n| **标签** | `Agent`, `Knowledge`, `MultiTurn` |\n| **指标** | `submission_ready` |\n| **默认示例数** | 0-shot |\n| **评估分割** | `train` |\n\n\n## 数据统计\n\n| 指标 | 值 |\n|--------|-------|\n| 总样本数 | 220 |\n| 提示词长度（平均） | 2742.59 字符 |\n| 提示词长度（最小/最大） | 1058 / 7160 字符 |\n\n## 样例示例\n\n**子集**: `default`\n\n```json\n{\n  \"input\": [\n    {\n      \"id\": \"3315415d\",\n      \"content\": \"You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\\n\\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 2069 chars] ...  folder named `deliverable_files` in the sandbox working directory.\\nWe will grade your final message as part of the deliverable, but requested documents, spreadsheets, slides, media,\\nor archives should be actual files in `deliverable_files`.\\n\"\n    }\n  ],\n  \"target\": \"\",\n  \"id\": 0,\n  \"group_id\": 0,\n  \"tools\": [\n    {\n      \"name\": \"bash\",\n      \"description\": \"Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.\",\n      \"parameters\": {\n        \"properties\": {\n          \"command\": {\n            \"type\": \"string\",\n            \"description\": \"The bash command to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"command\"\n        ]\n      }\n    },\n    {\n      \"name\": \"python_exec\",\n      \"description\": \"Execute Python source code inside the sandbox environment. Returns stdout and stderr output.\",\n      \"parameters\": {\n        \"properties\": {\n          \"code\": {\n            \"type\": \"string\",\n            \"description\": \"Python source code to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"code\"\n        ]\n      }\n    }\n  ],\n  \"metadata\": {\n    \"task_id\": \"83d10b06-26d1-4636-a32c-23f92c57f30b\",\n    \"sector\": \"Professional, Scientific, and Technical Services\",\n    \"occupation\": \"Accountants and Auditors\",\n    \"prompt\": \"You are an auditor and as part of an audit engagement, you are tasked with reviewing and testing the accuracy of reported Anti-Financial Crime Risk Metrics.\\n\\nThe attached spreadsheet titled ‘Population’ contains Anti-Financial Crime Risk Metr ... [TRUNCATED 1526 chars] ... across all Divisions and sub-Divisions.\\n\\n4. Create a new spreadsheet titled ‘Sample’:\\n- Tab 1: Selected sample, copied from the original ‘Population’ sheet, with selected rows marked in column K.\\n- Tab 2: Workings for sample size calculation.\",\n    \"reference_files\": [\n      \"reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx\"\n    ],\n    \"reference_file_urls\": [\n      \"https://huggingface.co/datasets/openai/gdpval/resolve/main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx\"\n    ],\n    \"reference_file_hf_uris\": [\n      \"hf://datasets/openai/gdpval@main/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population%20v2.xlsx\"\n    ],\n    \"reference_paths\": [\n      \"reference_files/Population v2.xlsx\"\n    ],\n    \"sandbox_reference_paths\": [\n      \"/reference_files/cc781e4dc0985c8eb327a53ec03b5900/Population v2.xlsx\"\n    ],\n    \"rubric_pretty\": \"[+2] The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\\n\\n[+2] The workbook contains a worksheet named exactly 'Sample Size Calculation' (case-insensitive, ignoring surrounding spaces ... [TRUNCATED 4861 chars] ... ntage changes (e.g., |J| ≥ 100%) are made easily identifiable (such as by a separate flag, note, or conditional formatting).\\n\\n[+1] The first worksheet is named 'Sample' (case-insensitive).\\n\\n[+5] Overall formatting and style of the deliverable\",\n    \"rubric_json\": \"[{\\\"score\\\": 2, \\\"criterion\\\": \\\"The submitted deliverable is an Excel workbook file whose basename is 'Sample' (accept .xlsx, .xls, or .xlsm).\\\", \\\"required\\\": null, \\\"rubric_item_id\\\": \\\"1d43f1eb-4011-47ac-8ad7-a3c467639a6a\\\", \\\"author_type\\\": \\\"human\\\", \\\" ... [TRUNCATED 11817 chars] ... ull}, {\\\"score\\\": 5, \\\"criterion\\\": \\\"Overall formatting and style of the deliverable\\\", \\\"required\\\": null, \\\"rubric_item_id\\\": \\\"a64588ed-db04-4b8b-b3b8-3674ddcf10d1\\\", \\\"author_type\\\": \\\"human\\\", \\\"tags\\\": [\\\"true\\\"], \\\"read_only\\\": null, \\\"form_content\\\": null}]\",\n    \"dataset_id\": \"openai-mirror/gdpval\",\n    \"dataset_hub\": \"modelscope\"\n  }\n}\n```\n\n## 提示模板\n\n**提示模板:**\n```text\n{question}\n```\n\n## 额外参数\n\n| 参数 | 类型 | 默认值 | 描述 |\n|-----------|------|---------|-------------|\n| `max_steps` | `int` | `250` | 每个样本的最大代理步骤数。 |\n| `command_timeout` | `float` | `180.0` | 每条命令的默认超时时间（秒）。 |\n| `docker_image` | `str` | `evalscope/gdpval:latest` | 用作每个样本沙箱的 Docker 镜像。 |\n| `auto_build_docker_image` | `bool` | `True` | 若本地缺失默认 GDPval Docker 镜像，则自动构建。 |\n| `network_enabled` | `bool` | `True` | 允许沙箱访问网络。 |\n| `download_reference_files` | `bool` | `True` | 在推理前从数据集中心下载每个选定样本的参考文件。 |\n\n## 使用方法\n\n### 使用 CLI\n\n```bash\nevalscope eval \\\n    --model YOUR_MODEL \\\n    --api-url OPENAI_API_COMPAT_URL \\\n    --api-key EMPTY_TOKEN \\\n    --datasets gdpval \\\n    --limit 10  # 正式评估时请删除此行\n```\n\n### 使用 Python\n\n```python\nfrom evalscope import run_task\nfrom evalscope.config import TaskConfig\n\ntask_cfg = TaskConfig(\n    model='YOUR_MODEL',\n    api_url='OPENAI_API_COMPAT_URL',\n    api_key='EMPTY_TOKEN',\n    datasets=['gdpval'],\n    dataset_args={\n        'gdpval': {\n            # extra_params: {}  # 使用默认额外参数\n        }\n    },\n    limit=10,  # 正式评估时请删除此行\n)\n\nrun_task(task_cfg=task_cfg)\n```",
    "content_hash": "44951e777aaf601d1b5a0304c2a3aaf3",
    "needs_translation": false
  },
  "updated_at": "2026-06-30T18:01:44.292643",
  "translation_updated_at": "2026-06-30T18:01:47"
}