qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(910c/glm52): 并行配置改为只测 TP=8 和 TP=16,并按TP区分服务参数

Browse Source 
config.env: PARALLEL_CONFIGS 默认值从 "2 4"/"4 2"/"8 1" 改为 "8 1"/"16 1"
run_adaptive_concurrency_add16.sh: engine_build_server_args 按 TP 覆盖参数
- TP=8:  gpu_mem_util=0.95, max_model_len=16384,  max_num_seqs=64  (64GB/die KV cache 紧张)
- TP=16: gpu_mem_util=0.92, max_model_len=131072, max_num_seqs=256 (16 die 全用)
可通过 TP8_*/TP16_* 环境变量进一步覆盖
This commit is contained in:
shishi 2026-07-28 16:51:14 +08:00
parent 98cdb67b66
commit 6c81183fd7
2 changed files with 25 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
experiments/910c/glm52_910c_vllm_tp_dp_matrix/config.env
View File

@ -47,9 +47,9 @@ export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}"
RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}" RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}"
# Parallel configurations to test. Format: "TP DP" # Parallel configurations to test. Format: "TP DP"
# Each Ascend910 card has 2 dies; TP addresses dies, so TP=8 uses 8 dies across # Each Ascend910 card has 2 dies; TP addresses dies. TP=8 uses 8 dies (4 cards),
# 4 cards and leaves room for DP. TP=2/DP=4 and TP=4/DP=2 and TP=8/DP=1 all fit # TP=16 uses all 16 dies (8 cards). Only these two configs are tested.
# within 8 cards (16 dies). Override via PARALLEL_CONFIGS_STR="8,1". # Override via PARALLEL_CONFIGS_STR="8,1 16,1" (space-separated).
if [[ -n "${PARALLEL_CONFIGS_STR:-}" ]]; then if [[ -n "${PARALLEL_CONFIGS_STR:-}" ]]; then
declare -a PARALLEL_CONFIGS=() declare -a PARALLEL_CONFIGS=()
for pair in $PARALLEL_CONFIGS_STR; do for pair in $PARALLEL_CONFIGS_STR; do
@ -57,9 +57,8 @@ if [[ -n "${PARALLEL_CONFIGS_STR:-}" ]]; then
done done
else else
declare -a PARALLEL_CONFIGS=( declare -a PARALLEL_CONFIGS=(
"2 4"
"4 2"
"8 1" "8 1"
"16 1"
) )
fi fi

24
experiments/910c/glm52_910c_vllm_tp_dp_matrix/run_adaptive_concurrency_add16.sh
View File

@ -74,6 +74,24 @@ engine_stop_server() {
engine_build_server_args() { engine_build_server_args() {
local tp="$1" local tp="$1"
local dp="$2" local dp="$2"
# Per-TP overrides: TP=8 has tight KV cache on 64GB/die (cap context/seqs),
# TP=16 uses all 16 dies and can serve the full 128K context.
local mem_util="$GPU_MEMORY_UTILIZATION"
local max_len="$MAX_MODEL_LEN"
local max_seqs="$MAX_NUM_SEQS"
case "$tp" in
8)
mem_util="${TP8_GPU_MEMORY_UTILIZATION:-0.95}"
max_len="${TP8_MAX_MODEL_LEN:-16384}"
max_seqs="${TP8_MAX_NUM_SEQS:-64}"
;;
16)
mem_util="${TP16_GPU_MEMORY_UTILIZATION:-0.92}"
max_len="${TP16_MAX_MODEL_LEN:-131072}"
max_seqs="${TP16_MAX_NUM_SEQS:-256}"
;;
esac
log "server args tp=${tp} dp=${dp} gpu_mem_util=${mem_util} max_model_len=${max_len} max_num_seqs=${max_seqs}" >&2
local -a args=( local -a args=(
vllm serve "$MODEL_PATH" vllm serve "$MODEL_PATH"
--served-model-name "$SERVED_MODEL_NAME" --served-model-name "$SERVED_MODEL_NAME"
@ -81,9 +99,9 @@ engine_build_server_args() {
--kv-cache-dtype "$KV_CACHE_DTYPE" --kv-cache-dtype "$KV_CACHE_DTYPE"
--block-size "$BLOCK_SIZE" --block-size "$BLOCK_SIZE"
--tensor-parallel-size "$tp" --tensor-parallel-size "$tp"
--gpu-memory-utilization "$GPU_MEMORY_UTILIZATION" --gpu-memory-utilization "$mem_util"
--max-model-len "$MAX_MODEL_LEN" --max-model-len "$max_len"
--max-num-seqs "$MAX_NUM_SEQS" --max-num-seqs "$max_seqs"
--host 0.0.0.0 --host 0.0.0.0
--port "$ENGINE_PORT" --port "$ENGINE_PORT"
) )