9acf9fdfdb
- sskj.deploy runtime 支持 NODE_HOSTS 多节点编排(ssh 分发/本地 rank/LOCAL_NODE_RANK)
与 ENGINE=vllm 启动(SERVER_CMD)，容器名按 rank 自动唯一
- scripts/common/deploy_cli.sh 新增 deploy_stop/status/multinode helper 与 node-rank 透传
- src/sskj/common/env.py 修复嵌套 ${VAR:-${OTHER}/path} 展开(平衡花括号扫描)
- deploy/profiles/pro6000/ 新增 6 个 profile: tp16/tp16_eagle/glm52(多节点)、
sglang/vllm tp_dp_matrix、qwen3(单节点)
- 6 个实验 start/stop 脚本改为 deploy 薄包装，run_bench/adaptive 的 server 启停走
deploy_render_args/deploy_start/deploy_stop，tp16 新增 matrix.json
- 首次入库 glm52_pro6000_sglang_multinode_tp16 实验目录；ops/README.md 补 pro6000 章节
- 实测通过: 单节点 dsv4 sglang/vllm 链路 + tp16 双节点启动/bench/清理
35 lines
1.5 KiB
Bash
35 lines
1.5 KiB
Bash
# DeepSeek-V4-Pro SGLang multi-node TP=16 EP=2 deployment profile (2x RTX 6000D).
|
|
# Nodes: 10.101.0.11 (rank 0) + 10.101.0.13 (rank 1), 16x RTX 6000D (85GB).
|
|
# Key: SGLANG_SHARED_EXPERT_TP1=1 + --ep-size 2 -> TP=16 works with FP8 block=128.
|
|
# Model-team only. Ops only run `python -m sskj.bench` against the served URL.
|
|
|
|
PLATFORM=pro6000
|
|
EXPERIMENT=dsv4_pro6000_sglang_tp16
|
|
MODEL_NAME=DeepSeek-V4-Pro
|
|
ENGINE=sglang
|
|
RUNTIME=docker
|
|
DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260720-b3570a45
|
|
CONTAINER_NAME=${EXPERIMENT}_node${NODE_RANK}
|
|
MODEL_PATH=/data/hf_models/DeepSeek-V4-Pro
|
|
SERVED_MODEL_NAME=default
|
|
PORT=30000
|
|
HEALTH_PATH=/health
|
|
HEALTH_HOST=10.101.0.11
|
|
HEALTH_WAIT_S=2400
|
|
CONTAINER_PYTHON=python3
|
|
|
|
# ---- Multi-node topology (rank order; rank 0 exposes the HTTP API) ----
|
|
NNODES=2
|
|
NODE_HOSTS="10.101.0.11 10.101.0.13"
|
|
NODE_SSH_USER=root
|
|
MASTER_IP=10.101.0.11
|
|
DIST_PORT=20000
|
|
|
|
DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
|
|
ENGINE_ENV="NCCL_SOCKET_IFNAME=eth0 NCCL_DEBUG=WARN SGLANG_SHARED_EXPERT_TP1=1 PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1"
|
|
|
|
DOCKER_FLAGS="--gpus all --network host --ipc=host --ulimit memlock=-1 --shm-size=20g"
|
|
VOLUMES="${MODEL_PATH}:${MODEL_PATH}:ro"
|
|
|
|
LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --ep-size 2 --nnodes ${NNODES} --node-rank ${NODE_RANK} --dist-init-addr ${MASTER_IP}:${DIST_PORT} --trust-remote-code --host 0.0.0.0 --port ${PORT} --mem-fraction-static 0.9 --cuda-graph-max-bs-decode 64 --max-running-requests 256"
|