{
  "meta": {
    "pretty_name": "SWE-bench_Verified_Mini_Agentic",
    "dataset_id": "evalscope/swe-bench-verified-mini",
    "paper_url": null,
    "tags": [
      "Coding"
    ],
    "metrics": [
      "acc"
    ],
    "few_shot_num": 0,
    "eval_split": "test",
    "train_split": "",
    "subset_list": [
      "default"
    ],
    "description": "\n## Overview\n\nSWE-bench Verified Mini Agentic is the agentic-mode evaluation of SWE-bench Verified Mini, a compact 50-sample subset that maintains the same distribution of performance, test pass rates, and difficulty as the full Verified set while requiring only 5GB of storage instead of 130GB. The model must autonomously explore, edit, and submit a patch through a multi-turn agent loop.\n\n## Task Description\n\n- **Task Type**: Automated Software Engineering / Bug Fixing (Agentic)\n- **Input**: GitHub issue description (no oracle file context)\n- **Output**: Code patch (diff format) collected from `git diff` after autonomous editing\n- **Size**: 50 samples (vs 500 in full Verified set)\n\n## Key Features\n\n- Representative 50-sample subset of SWE-bench Verified\n- Same difficulty distribution as the full dataset\n- Dramatically reduced storage requirements (5GB vs 130GB)\n- Multi-turn agent loop with per-instance Docker sandbox\n- Ideal for quick agentic evaluation and development iteration\n\n## Evaluation Notes\n\n- Requires `pip install swebench==4.1.0` before evaluation\n- Docker images are built/pulled automatically\n- See the [usage documentation](https://evalscope.readthedocs.io/en/latest/third_party/swe_bench.html) for detailed setup\n- Good for rapid prototyping of agent strategies and initial model assessment\n\n## Agentic Mode\n\nThis benchmark drives a multi-turn agent loop (mirrors mini-swe-agent's\n`swebench.yaml`) inside a per-instance SWE-bench Docker container. The\nmodel issues `bash` commands to explore `/testbed`, edit source files,\nand finally submits its `git diff` patch by printing the sentinel\n`COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT` followed by the patch contents.\n\n`extra_params.action_protocol` selects between:\n- `toolcall` (default): OpenAI function-calling protocol with a single\n  `bash` tool. Recommended for any model that supports tool calling.\n- `backticks`: text-based fallback expecting one\n  ` ```mswea_bash_command ``` ` block per turn. For models without\n  function-calling support.\n",
    "prompt_template": "{question}",
    "system_prompt": "",
    "few_shot_prompt_template": "",
    "aggregation": "mean",
    "extra_params": {
      "action_protocol": {
        "type": "str",
        "description": "Agent action protocol: \"toolcall\" (mainline OpenAI function-calling, mirrors mini-swe-agent swebench.yaml) or \"backticks\" (textbased mswea_bash_command fallback for models without function-calling support).",
        "value": "toolcall",
        "choices": [
          "toolcall",
          "backticks"
        ]
      },
      "max_steps": {
        "type": "int",
        "description": "Maximum number of agent steps per sample.",
        "value": 250
      },
      "command_timeout": {
        "type": "float",
        "description": "Default per-bash-command timeout in seconds.",
        "value": 60.0
      },
      "build_docker_images": {
        "type": "bool",
        "description": "Build Docker images locally for each sample.",
        "value": true
      },
      "pull_remote_images_if_available": {
        "type": "bool",
        "description": "Attempt to pull existing remote Docker images before building.",
        "value": true
      },
      "force_arch": {
        "type": "str",
        "description": "Optionally force a specific architecture for image build/pull.",
        "value": "",
        "choices": [
          "",
          "arm64",
          "x86_64"
        ]
      },
      "dockerhub_username": {
        "type": "str",
        "description": "DockerHub user/org namespace for remote SWE-bench images.",
        "value": "swebench"
      }
    },
    "sandbox_config": {},
    "category": "agent"
  },
  "statistics": {
    "total_samples": 50,
    "subset_stats": [
      {
        "name": "default",
        "sample_count": 50,
        "prompt_length_mean": 1268.5,
        "prompt_length_min": 257,
        "prompt_length_max": 5362,
        "prompt_length_std": 997.63,
        "target_length_mean": null
      }
    ],
    "prompt_length": {
      "mean": 1268.5,
      "min": 257,
      "max": 5362,
      "std": 997.63
    },
    "target_length_mean": null,
    "computed_at": "2026-05-15T15:00:43.757185"
  },
  "sample_example": {
    "data": {
      "input": [
        {
          "id": "363bb967",
          "content": "AuthenticationForm's username field doesn't set maxlength HTML attribute.\nDescription\n\t\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\n"
        }
      ],
      "id": 0,
      "group_id": 0,
      "tools": [
        {
          "name": "bash",
          "description": "Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.",
          "parameters": {
            "properties": {
              "command": {
                "type": "string",
                "description": "The bash command to execute."
              },
              "timeout": {
                "type": "number",
                "description": "Maximum execution time in seconds (default: 60).",
                "default": 60
              }
            },
            "required": [
              "command"
            ]
          }
        }
      ],
      "metadata": {
        "problem_statement": "AuthenticationForm's username field doesn't set maxlength HTML attribute.\nDescription\n\t\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\n",
        "instance_id": "django__django-11790",
        "base_commit": "b1d6b35e146aea83b171c1b921178bbaae2795ed",
        "patch": "diff --git a/django/contrib/auth/forms.py b/django/contrib/auth/forms.py\n--- a/django/contrib/auth/forms.py\n+++ b/django/contrib/auth/forms.py\n@@ -191,7 +191,9 @@ def __init__(self, request=None, *args, **kwargs):\n \n         # Set the max len ... [TRUNCATED 322 chars] ... username_max_length\n+        self.fields['username'].widget.attrs['maxlength'] = username_max_length\n         if self.fields['username'].label is None:\n             self.fields['username'].label = capfirst(self.username_field.verbose_name)\n \n",
        "PASS_TO_PASS": [
          "test_html_autocomplete_attributes (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_missing_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_non_matching_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_one_password (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_password_whitespace_not_stripped (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_success (auth_tests.test_forms.AdminPasswordChangeFormTest)",
          "test_field_order (auth_tests.test_forms.PasswordChangeFormTest)",
          "test_html_autocomplete_attributes (auth_tests.test_forms.PasswordChangeFormTest)",
          "test_incorrect_password (auth_tests.test_forms.PasswordChangeFormTest)",
          "test_password_verification (auth_tests.test_forms.PasswordChangeFormTest)",
          "... [TRUNCATED 67 more items] ..."
        ],
        "FAIL_TO_PASS": [
          "test_username_field_max_length_defaults_to_254 (auth_tests.test_forms.AuthenticationFormTest)",
          "test_username_field_max_length_matches_user_model (auth_tests.test_forms.AuthenticationFormTest)"
        ],
        "test_patch": "diff --git a/tests/auth_tests/test_forms.py b/tests/auth_tests/test_forms.py\n--- a/tests/auth_tests/test_forms.py\n+++ b/tests/auth_tests/test_forms.py\n@@ -423,6 +423,7 @@ def test_username_field_max_length_matches_user_model(self):\n         C ... [TRUNCATED 543 chars] ... )\n         self.assertEqual(form.fields['username'].max_length, 254)\n+        self.assertEqual(form.fields['username'].widget.attrs.get('maxlength'), 254)\n         self.assertEqual(form.errors, {})\n \n     def test_username_field_label(self):\n",
        "version": "3.1",
        "repo": "django/django",
        "environment_setup_commit": "0668164b4ac93a5be79f5b87fae83c657124d9ab",
        "hints_text": "Regression test.",
        "created_at": "2019-09-17T14:33:44Z",
        "docker_image": "swebench/sweb.eval.arm64.django_1776_django-11790:latest"
      }
    },
    "subset": "default",
    "truncated": false
  },
  "readme": {
    "en": "# SWE-bench_Verified_Mini_Agentic\n\n\n## Overview\n\nSWE-bench Verified Mini Agentic is the agentic-mode evaluation of SWE-bench Verified Mini, a compact 50-sample subset that maintains the same distribution of performance, test pass rates, and difficulty as the full Verified set while requiring only 5GB of storage instead of 130GB. The model must autonomously explore, edit, and submit a patch through a multi-turn agent loop.\n\n## Task Description\n\n- **Task Type**: Automated Software Engineering / Bug Fixing (Agentic)\n- **Input**: GitHub issue description (no oracle file context)\n- **Output**: Code patch (diff format) collected from `git diff` after autonomous editing\n- **Size**: 50 samples (vs 500 in full Verified set)\n\n## Key Features\n\n- Representative 50-sample subset of SWE-bench Verified\n- Same difficulty distribution as the full dataset\n- Dramatically reduced storage requirements (5GB vs 130GB)\n- Multi-turn agent loop with per-instance Docker sandbox\n- Ideal for quick agentic evaluation and development iteration\n\n## Evaluation Notes\n\n- Requires `pip install swebench==4.1.0` before evaluation\n- Docker images are built/pulled automatically\n- See the [usage documentation](https://evalscope.readthedocs.io/en/latest/third_party/swe_bench.html) for detailed setup\n- Good for rapid prototyping of agent strategies and initial model assessment\n\n## Agentic Mode\n\nThis benchmark drives a multi-turn agent loop (mirrors mini-swe-agent's\n`swebench.yaml`) inside a per-instance SWE-bench Docker container. The\nmodel issues `bash` commands to explore `/testbed`, edit source files,\nand finally submits its `git diff` patch by printing the sentinel\n`COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT` followed by the patch contents.\n\n`extra_params.action_protocol` selects between:\n- `toolcall` (default): OpenAI function-calling protocol with a single\n  `bash` tool. Recommended for any model that supports tool calling.\n- `backticks`: text-based fallback expecting one\n  ` ```mswea_bash_command ``` ` block per turn. For models without\n  function-calling support.\n\n\n## Properties\n\n| Property | Value |\n|----------|-------|\n| **Benchmark Name** | `swe_bench_verified_mini_agentic` |\n| **Dataset ID** | [evalscope/swe-bench-verified-mini](https://modelscope.cn/datasets/evalscope/swe-bench-verified-mini/summary) |\n| **Paper** | N/A |\n| **Tags** | `Coding` |\n| **Metrics** | `acc` |\n| **Default Shots** | 0-shot |\n| **Evaluation Split** | `test` |\n\n\n## Data Statistics\n\n| Metric | Value |\n|--------|-------|\n| Total Samples | 50 |\n| Prompt Length (Mean) | 1268.5 chars |\n| Prompt Length (Min/Max) | 257 / 5362 chars |\n\n## Sample Example\n\n**Subset**: `default`\n\n```json\n{\n  \"input\": [\n    {\n      \"id\": \"363bb967\",\n      \"content\": \"AuthenticationForm's username field doesn't set maxlength HTML attribute.\\nDescription\\n\\t\\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\\n\"\n    }\n  ],\n  \"id\": 0,\n  \"group_id\": 0,\n  \"tools\": [\n    {\n      \"name\": \"bash\",\n      \"description\": \"Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.\",\n      \"parameters\": {\n        \"properties\": {\n          \"command\": {\n            \"type\": \"string\",\n            \"description\": \"The bash command to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"command\"\n        ]\n      }\n    }\n  ],\n  \"metadata\": {\n    \"problem_statement\": \"AuthenticationForm's username field doesn't set maxlength HTML attribute.\\nDescription\\n\\t\\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\\n\",\n    \"instance_id\": \"django__django-11790\",\n    \"base_commit\": \"b1d6b35e146aea83b171c1b921178bbaae2795ed\",\n    \"patch\": \"diff --git a/django/contrib/auth/forms.py b/django/contrib/auth/forms.py\\n--- a/django/contrib/auth/forms.py\\n+++ b/django/contrib/auth/forms.py\\n@@ -191,7 +191,9 @@ def __init__(self, request=None, *args, **kwargs):\\n \\n         # Set the max len ... [TRUNCATED 322 chars] ... username_max_length\\n+        self.fields['username'].widget.attrs['maxlength'] = username_max_length\\n         if self.fields['username'].label is None:\\n             self.fields['username'].label = capfirst(self.username_field.verbose_name)\\n \\n\",\n    \"PASS_TO_PASS\": [\n      \"test_html_autocomplete_attributes (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_missing_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_non_matching_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_one_password (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_password_whitespace_not_stripped (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_success (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_field_order (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_html_autocomplete_attributes (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_incorrect_password (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_password_verification (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"... [TRUNCATED 67 more items] ...\"\n    ],\n    \"FAIL_TO_PASS\": [\n      \"test_username_field_max_length_defaults_to_254 (auth_tests.test_forms.AuthenticationFormTest)\",\n      \"test_username_field_max_length_matches_user_model (auth_tests.test_forms.AuthenticationFormTest)\"\n    ],\n    \"test_patch\": \"diff --git a/tests/auth_tests/test_forms.py b/tests/auth_tests/test_forms.py\\n--- a/tests/auth_tests/test_forms.py\\n+++ b/tests/auth_tests/test_forms.py\\n@@ -423,6 +423,7 @@ def test_username_field_max_length_matches_user_model(self):\\n         C ... [TRUNCATED 543 chars] ... )\\n         self.assertEqual(form.fields['username'].max_length, 254)\\n+        self.assertEqual(form.fields['username'].widget.attrs.get('maxlength'), 254)\\n         self.assertEqual(form.errors, {})\\n \\n     def test_username_field_label(self):\\n\",\n    \"version\": \"3.1\",\n    \"repo\": \"django/django\",\n    \"environment_setup_commit\": \"0668164b4ac93a5be79f5b87fae83c657124d9ab\",\n    \"hints_text\": \"Regression test.\",\n    \"created_at\": \"2019-09-17T14:33:44Z\",\n    \"docker_image\": \"swebench/sweb.eval.arm64.django_1776_django-11790:latest\"\n  }\n}\n```\n\n## Prompt Template\n\n**Prompt Template:**\n```text\n{question}\n```\n\n## Extra Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `action_protocol` | `str` | `toolcall` | Agent action protocol: \"toolcall\" (mainline OpenAI function-calling, mirrors mini-swe-agent swebench.yaml) or \"backticks\" (textbased mswea_bash_command fallback for models without function-calling support). Choices: ['toolcall', 'backticks'] |\n| `max_steps` | `int` | `250` | Maximum number of agent steps per sample. |\n| `command_timeout` | `float` | `60.0` | Default per-bash-command timeout in seconds. |\n| `build_docker_images` | `bool` | `True` | Build Docker images locally for each sample. |\n| `pull_remote_images_if_available` | `bool` | `True` | Attempt to pull existing remote Docker images before building. |\n| `force_arch` | `str` | `` | Optionally force a specific architecture for image build/pull. Choices: ['', 'arm64', 'x86_64'] |\n| `dockerhub_username` | `str` | `swebench` | DockerHub user/org namespace for remote SWE-bench images. |\n\n## Usage\n\n### Using CLI\n\n```bash\nevalscope eval \\\n    --model YOUR_MODEL \\\n    --api-url OPENAI_API_COMPAT_URL \\\n    --api-key EMPTY_TOKEN \\\n    --datasets swe_bench_verified_mini_agentic \\\n    --limit 10  # Remove this line for formal evaluation\n```\n\n### Using Python\n\n```python\nfrom evalscope import run_task\nfrom evalscope.config import TaskConfig\n\ntask_cfg = TaskConfig(\n    model='YOUR_MODEL',\n    api_url='OPENAI_API_COMPAT_URL',\n    api_key='EMPTY_TOKEN',\n    datasets=['swe_bench_verified_mini_agentic'],\n    dataset_args={\n        'swe_bench_verified_mini_agentic': {\n            # extra_params: {}  # uses default extra parameters\n        }\n    },\n    limit=10,  # Remove this line for formal evaluation\n)\n\nrun_task(task_cfg=task_cfg)\n```\n\n\n",
    "zh": "# SWE-bench_Verified_Mini_Agentic\n\n\n## 概述\n\nSWE-bench Verified Mini Agentic 是对 SWE-bench Verified Mini 的代理模式（agentic-mode）评估。SWE-bench Verified Mini 是一个精简的 50 样本子集，在保持与完整 Verified 集合相同的性能分布、测试通过率和难度的同时，仅需 5GB 存储空间（而非 130GB）。模型必须通过多轮代理循环，自主探索、编辑代码并提交补丁。\n\n## 任务描述\n\n- **任务类型**：自动化软件工程 / 缺陷修复（代理模式）\n- **输入**：GitHub issue 描述（无 oracle 文件上下文）\n- **输出**：代码补丁（以 `git diff` 生成的 diff 格式）\n- **规模**：50 个样本（完整 Verified 集合为 500 个）\n\n## 主要特性\n\n- SWE-bench Verified 的代表性 50 样本子集\n- 与完整数据集具有相同的难度分布\n- 存储需求大幅降低（5GB vs 130GB）\n- 每个实例使用独立 Docker 沙箱的多轮代理循环\n- 非常适合快速代理评估和开发迭代\n\n## 评估说明\n\n- 评估前需执行 `pip install swebench==4.1.0`\n- Docker 镜像会自动构建或拉取\n- 详细设置请参阅 [使用文档](https://evalscope.readthedocs.io/zh-cn/latest/third_party/swe_bench.html)\n- 适用于代理策略的快速原型设计和模型初步评估\n\n## 代理模式\n\n该基准测试在每个实例专属的 SWE-bench Docker 容器内驱动一个多轮代理循环（与 mini-swe-agent 的 `swebench.yaml` 配置一致）。模型通过发出 `bash` 命令来探索 `/testbed` 目录、编辑源文件，并最终通过打印哨兵字符串 `COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT` 及其后的补丁内容来提交 `git diff` 补丁。\n\n`extra_params.action_protocol` 可选择以下两种协议：\n- `toolcall`（默认）：OpenAI 函数调用协议，仅提供一个 `bash` 工具。推荐用于支持工具调用的模型。\n- `backticks`：基于文本的备用协议，每轮期望一个 ` ```mswea_bash_command ``` ` 代码块。适用于不支持函数调用的模型。\n\n## 属性\n\n| 属性 | 值 |\n|----------|-------|\n| **基准测试名称** | `swe_bench_verified_mini_agentic` |\n| **数据集ID** | [evalscope/swe-bench-verified-mini](https://modelscope.cn/datasets/evalscope/swe-bench-verified-mini/summary) |\n| **论文** | N/A |\n| **标签** | `Coding` |\n| **指标** | `acc` |\n| **默认示例数** | 0-shot |\n| **评估划分** | `test` |\n\n\n## 数据统计\n\n| 指标 | 值 |\n|--------|-------|\n| 总样本数 | 50 |\n| 提示词长度（平均） | 1268.5 字符 |\n| 提示词长度（最小/最大） | 257 / 5362 字符 |\n\n## 样例示例\n\n**子集**: `default`\n\n```json\n{\n  \"input\": [\n    {\n      \"id\": \"363bb967\",\n      \"content\": \"AuthenticationForm's username field doesn't set maxlength HTML attribute.\\nDescription\\n\\t\\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\\n\"\n    }\n  ],\n  \"id\": 0,\n  \"group_id\": 0,\n  \"tools\": [\n    {\n      \"name\": \"bash\",\n      \"description\": \"Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.\",\n      \"parameters\": {\n        \"properties\": {\n          \"command\": {\n            \"type\": \"string\",\n            \"description\": \"The bash command to execute.\"\n          },\n          \"timeout\": {\n            \"type\": \"number\",\n            \"description\": \"Maximum execution time in seconds (default: 60).\",\n            \"default\": 60\n          }\n        },\n        \"required\": [\n          \"command\"\n        ]\n      }\n    }\n  ],\n  \"metadata\": {\n    \"problem_statement\": \"AuthenticationForm's username field doesn't set maxlength HTML attribute.\\nDescription\\n\\t\\nAuthenticationForm's username field doesn't render with maxlength HTML attribute anymore.\\nRegression introduced in #27515 and 5ceaf14686ce626404afb6a5fbd3d8286410bf13.\\n​https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/django-developers/qnfSqro0DlA\\n​https://forum.djangoproject.com/t/possible-authenticationform-max-length-regression-in-django-2-1/241\\n\",\n    \"instance_id\": \"django__django-11790\",\n    \"base_commit\": \"b1d6b35e146aea83b171c1b921178bbaae2795ed\",\n    \"patch\": \"diff --git a/django/contrib/auth/forms.py b/django/contrib/auth/forms.py\\n--- a/django/contrib/auth/forms.py\\n+++ b/django/contrib/auth/forms.py\\n@@ -191,7 +191,9 @@ def __init__(self, request=None, *args, **kwargs):\\n \\n         # Set the max len ... [TRUNCATED 322 chars] ... username_max_length\\n+        self.fields['username'].widget.attrs['maxlength'] = username_max_length\\n         if self.fields['username'].label is None:\\n             self.fields['username'].label = capfirst(self.username_field.verbose_name)\\n \\n\",\n    \"PASS_TO_PASS\": [\n      \"test_html_autocomplete_attributes (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_missing_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_non_matching_passwords (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_one_password (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_password_whitespace_not_stripped (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_success (auth_tests.test_forms.AdminPasswordChangeFormTest)\",\n      \"test_field_order (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_html_autocomplete_attributes (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_incorrect_password (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"test_password_verification (auth_tests.test_forms.PasswordChangeFormTest)\",\n      \"... [TRUNCATED 67 more items] ...\"\n    ],\n    \"FAIL_TO_PASS\": [\n      \"test_username_field_max_length_defaults_to_254 (auth_tests.test_forms.AuthenticationFormTest)\",\n      \"test_username_field_max_length_matches_user_model (auth_tests.test_forms.AuthenticationFormTest)\"\n    ],\n    \"test_patch\": \"diff --git a/tests/auth_tests/test_forms.py b/tests/auth_tests/test_forms.py\\n--- a/tests/auth_tests/test_forms.py\\n+++ b/tests/auth_tests/test_forms.py\\n@@ -423,6 +423,7 @@ def test_username_field_max_length_matches_user_model(self):\\n         C ... [TRUNCATED 543 chars] ... )\\n         self.assertEqual(form.fields['username'].max_length, 254)\\n+        self.assertEqual(form.fields['username'].widget.attrs.get('maxlength'), 254)\\n         self.assertEqual(form.errors, {})\\n \\n     def test_username_field_label(self):\\n\",\n    \"version\": \"3.1\",\n    \"repo\": \"django/django\",\n    \"environment_setup_commit\": \"0668164b4ac93a5be79f5b87fae83c657124d9ab\",\n    \"hints_text\": \"Regression test.\",\n    \"created_at\": \"2019-09-17T14:33:44Z\",\n    \"docker_image\": \"swebench/sweb.eval.arm64.django_1776_django-11790:latest\"\n  }\n}\n```\n\n## 提示模板\n\n**提示模板：**\n```text\n{question}\n```\n\n## 额外参数\n\n| 参数 | 类型 | 默认值 | 描述 |\n|-----------|------|---------|-------------|\n| `action_protocol` | `str` | `toolcall` | 代理动作协议：\"toolcall\"（主流 OpenAI 函数调用方式，与 mini-swe-agent 的 swebench.yaml 一致）或 \"backticks\"（针对不支持函数调用的模型的文本式 mswea_bash_command 回退方案）。可选值：['toolcall', 'backticks'] |\n| `max_steps` | `int` | `250` | 每个样本的最大代理步数。 |\n| `command_timeout` | `float` | `60.0` | 每个 bash 命令的默认超时时间（秒）。 |\n| `build_docker_images` | `bool` | `True` | 为每个样本在本地构建 Docker 镜像。 |\n| `pull_remote_images_if_available` | `bool` | `True` | 在构建前尝试拉取已存在的远程 Docker 镜像。 |\n| `force_arch` | `str` | `` | 可选地强制指定镜像构建/拉取的架构。可选值：['', 'arm64', 'x86_64'] |\n| `dockerhub_username` | `str` | `swebench` | 远程 SWE-bench 镜像的 DockerHub 用户/组织命名空间。 |\n\n## 使用方法\n\n### 使用 CLI\n\n```bash\nevalscope eval \\\n    --model YOUR_MODEL \\\n    --api-url OPENAI_API_COMPAT_URL \\\n    --api-key EMPTY_TOKEN \\\n    --datasets swe_bench_verified_mini_agentic \\\n    --limit 10  # 正式评估时请删除此行\n```\n\n### 使用 Python\n\n```python\nfrom evalscope import run_task\nfrom evalscope.config import TaskConfig\n\ntask_cfg = TaskConfig(\n    model='YOUR_MODEL',\n    api_url='OPENAI_API_COMPAT_URL',\n    api_key='EMPTY_TOKEN',\n    datasets=['swe_bench_verified_mini_agentic'],\n    dataset_args={\n        'swe_bench_verified_mini_agentic': {\n            # extra_params: {}  # 使用默认额外参数\n        }\n    },\n    limit=10,  # 正式评估时请删除此行\n)\n\nrun_task(task_cfg=task_cfg)\n```",
    "content_hash": "0f8056724238f62bc7de991132e04b38",
    "needs_translation": false
  },
  "updated_at": "2026-06-16T10:56:20.668415",
  "translation_updated_at": "2026-06-16T10:56:45"
}