114 lines
2.9 KiB
Markdown
114 lines
2.9 KiB
Markdown
# EvalScope Docker 评测快速入口
|
||
|
||
> 详细流程、参数说明、踩坑记录和三个版本（Lite/Standard/Full）的完整命令，请见 `work.md`。
|
||
>
|
||
> 本文档只保留最常用的命令，方便复制粘贴。
|
||
|
||
## 快速开始
|
||
|
||
### 0. 配置 Docker 国内镜像加速（可选但强烈建议）
|
||
|
||
```bash
|
||
sudo mkdir -p /etc/docker
|
||
sudo tee /etc/docker/daemon.json <<'EOF'
|
||
{
|
||
"registry-mirrors": [
|
||
"https://docker.1ms.run",
|
||
"https://docker.m.daocloud.io",
|
||
"https://hub-mirror.c.163.com"
|
||
]
|
||
}
|
||
EOF
|
||
sudo systemctl daemon-reload
|
||
sudo systemctl restart docker
|
||
```
|
||
|
||
### 1. 准备代码与数据
|
||
|
||
代码与 Docker 环境镜像**分开下载**：
|
||
|
||
```bash
|
||
pip install modelscope
|
||
modelscope login --token ms-3d554a39-6e07-496d-8022-0b0ee64a6389
|
||
|
||
# 下载项目代码
|
||
python3 -c "
|
||
from modelscope.hub.snapshot_download import snapshot_download
|
||
snapshot_download(
|
||
'SoraAmami/evalscope-code',
|
||
repo_type='dataset',
|
||
cache_dir='/data1/sora/evalscope',
|
||
local_dir='/data1/sora/evalscope'
|
||
)
|
||
"
|
||
|
||
# 下载数据集
|
||
python3 -c "
|
||
from modelscope.hub.snapshot_download import snapshot_download
|
||
snapshot_download(
|
||
'SoraAmami/evalscope-datasets',
|
||
repo_type='dataset',
|
||
cache_dir='/data1/sora/evalscope',
|
||
local_dir='/data1/sora/evalscope'
|
||
)
|
||
"
|
||
```
|
||
|
||
### 2. 加载 Docker 环境镜像
|
||
|
||
```bash
|
||
python3 -c "
|
||
from modelscope.hub.file_download import model_file_download
|
||
model_file_download(
|
||
model_id='SoraAmami/evalscope-docker',
|
||
file_path='evalscope-complete-py312.tar.gz',
|
||
local_dir='/data1/sora/evalscope/docker'
|
||
)
|
||
"
|
||
|
||
docker load -i /data1/sora/evalscope/docker/evalscope-complete-py312.tar.gz
|
||
```
|
||
|
||
### 3. 构建 bigcodebench sandbox 镜像
|
||
|
||
```bash
|
||
cd /data1/sora/evalscope
|
||
|
||
# 如果 Docker Hub 慢，先通过国内镜像拉取上游镜像
|
||
docker pull docker.1ms.run/bigcodebench/bigcodebench-evaluate:latest
|
||
docker tag docker.1ms.run/bigcodebench/bigcodebench-evaluate:latest bigcodebench/bigcodebench-evaluate:latest
|
||
|
||
# 构建去掉 entrypoint 的 sandbox 镜像
|
||
docker build -t bigcodebench-sandbox:latest docker/bigcodebench_sandbox/
|
||
```
|
||
|
||
### 4. 跑 Standard 版
|
||
|
||
```bash
|
||
mkdir -p /data1/sora/evalscope/output_standard
|
||
docker run -it --rm \
|
||
--network host \
|
||
-v /data1/sora/evalscope:/opt/evalscope \
|
||
-v /data1/sora/evalscope/datasets:/opt/evalscope/datasets \
|
||
-v /data1/sora/evalscope/output_standard:/opt/evalscope/output_standard \
|
||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||
evalscope-complete-py312:latest \
|
||
bash -c "
|
||
cd /opt/evalscope && \
|
||
python bash/run_standard.py \
|
||
--model DeepSeek-V4-Flash-Int8 \
|
||
--api-url http://localhost:30000/v1 \
|
||
--dataset-dir /opt/evalscope \
|
||
--output-dir /opt/evalscope/output_standard
|
||
"
|
||
```
|
||
|
||
## 更多内容
|
||
|
||
| 需求 | 查看文档 |
|
||
|------|----------|
|
||
| 完整 Docker 评测流程 | `work.md` 第 3 章 |
|
||
| Docker 镜像构建过程 | `DOCKER_BUILD.md` |
|
||
| 评测新模型 | `HOWTO_EVAL_NEW_MODEL.md` |
|
||
| 三个版本（Lite/Standard/Full）说明 | `work.md` 3.7 节 |
|