46e79d63e7
- platforms/ascend_910c.env: 8-card 910C config (16 dies, 64GB HBM/die), Ascend Docker Runtime, ASCEND_VISIBLE_DEVICES device selection - scripts/common/platform.sh: auto-detect 910C via npu-smi + Huawei PCI IDs - scripts/common/npu_smi_sampler.py: standalone npu-smi -> nvidia-smi CSV sampler so parse_backend.py needs no changes - experiments/910c/glm52_910c_vllm_tp_dp_matrix/: GLM-5.2 (w4a8c8) experiment, model present on host, ready for smoke after image load - experiments/910c/dsv4_910c_vllm_tp_dp_matrix/: DSV4-Flash experiment (placeholder MODEL_PATH, weights not yet downloaded) - envs/ASCEND_910C_ENV_SETUP.md: full onboarding guide (permissions, image load, Ascend Docker Runtime, NPU monitor, known pitfalls) - Both experiments: TP2/DP4 + TP4/DP2 + TP8/DP1, matrix.json capped at 128K context per 64GB HBM/die
43 lines
2.0 KiB
Bash
43 lines
2.0 KiB
Bash
# Platform configuration for Huawei Ascend 910C NPU
|
|
# Source this file via scripts/common/platform.sh
|
|
#
|
|
# Host reference (910c.1 / NPU-NODE61):
|
|
# - openEuler 22.03 SP4, aarch64, 8x Ascend910 (16 dies, 64GB HBM/die)
|
|
# - CANN 9.0.0 + ascend-toolkit, driver 25.5.2
|
|
# - Ascend Docker Runtime is the default docker runtime (see /etc/docker/daemon.json)
|
|
# - User must be in HwHiAiUser + docker groups to access NPU devices and docker.
|
|
|
|
CHIP="ascend_910c"
|
|
ACCELERATOR="Huawei Ascend 910C"
|
|
HARDWARE="8x Ascend910 (16 dies, 64GB HBM/die)"
|
|
ENGINE="vllm-ascend"
|
|
|
|
# Container runtime. vllm-ascend images ship as tarballs on this host and must
|
|
# be `docker load`-ed first (see envs/ASCEND_910C_ENV_SETUP.md). The exact tag
|
|
# is overridden per-experiment in config.env; this is the default for new ones.
|
|
# Known-good tarballs on /mnt/models:
|
|
# vllm-ascend-v0.23.0rc1-a3-openeuler.tar -> general vllm-ascend
|
|
# vllm-ascend-glm5.2-a3-openeuler.tar -> GLM5.2-tuned variant
|
|
DOCKER_IMAGE="${DOCKER_IMAGE:-quay.io/ascend/vllm-ascend:latest}"
|
|
CONTAINER_NAME="${CONTAINER_NAME:-vllm-ascend-910c}"
|
|
|
|
# Device selection. Ascend Docker Runtime maps NPU dies via ASCEND_VISIBLE_DEVICES
|
|
# (0-based NPU card id). 8 cards => 0..7. Each card exposes 2 dies; vllm-ascend
|
|
# addresses dies individually so TP can go up to 16.
|
|
DEVICE_SELECT_ENV="ASCEND_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
|
|
ASCEND_VISIBLE_DEVICES="${ASCEND_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}"
|
|
|
|
# Default serving port and model root on the host.
|
|
# /mnt/models holds GLM-5.2-{w4a8c8,w8a8} and vllm-ascend image tarballs.
|
|
DEFAULT_PORT="30050"
|
|
MODEL_ROOT="/mnt/models"
|
|
|
|
# Python interpreter inside the vllm-ascend container (used to run the
|
|
# benchmark client via `docker exec`). Standard path in the official image.
|
|
CONTAINER_PYTHON="${CONTAINER_PYTHON:-/usr/local/bin/python}"
|
|
|
|
# CANN / toolkit paths on the host (informational; the container bundles its
|
|
# own CANN). Referenced by env setup docs, not by run scripts directly.
|
|
HOST_CANN_ROOT="/usr/local/Ascend/ascend-toolkit/latest"
|
|
HOST_DRIVER_VERSION="25.5.2"
|