9acf9fdfdb
- sskj.deploy runtime 支持 NODE_HOSTS 多节点编排(ssh 分发/本地 rank/LOCAL_NODE_RANK)
与 ENGINE=vllm 启动(SERVER_CMD)，容器名按 rank 自动唯一
- scripts/common/deploy_cli.sh 新增 deploy_stop/status/multinode helper 与 node-rank 透传
- src/sskj/common/env.py 修复嵌套 ${VAR:-${OTHER}/path} 展开(平衡花括号扫描)
- deploy/profiles/pro6000/ 新增 6 个 profile: tp16/tp16_eagle/glm52(多节点)、
sglang/vllm tp_dp_matrix、qwen3(单节点)
- 6 个实验 start/stop 脚本改为 deploy 薄包装，run_bench/adaptive 的 server 启停走
deploy_render_args/deploy_start/deploy_stop，tp16 新增 matrix.json
- 首次入库 glm52_pro6000_sglang_multinode_tp16 实验目录；ops/README.md 补 pro6000 章节
- 实测通过: 单节点 dsv4 sglang/vllm 链路 + tp16 双节点启动/bench/清理
57 lines
2.5 KiB
Bash
57 lines
2.5 KiB
Bash
# Adaptive concurrency search settings.
|
|
#
|
|
# For each fixed (TP, DP, ISL, OSL), probe:
|
|
# C = start, start * multiplier, ... up to max
|
|
# and stop after Total TPS has less than TPS_MIN_GAIN_PCT meaningful growth for
|
|
# PLATEAU_PATIENCE consecutive points.
|
|
|
|
SEARCH_START_CONCURRENCY="${SEARCH_START_CONCURRENCY:-1}"
|
|
SEARCH_MAX_CONCURRENCY="${SEARCH_MAX_CONCURRENCY:-64}"
|
|
|
|
# At the add16 initial probe, restart and retry C=8 then C=1 after an OOM.
|
|
ENABLE_INITIAL_OOM_BACKOFF="${ENABLE_INITIAL_OOM_BACKOFF:-1}"
|
|
SEARCH_MULTIPLIER="${SEARCH_MULTIPLIER:-2}"
|
|
NUM_PROMPTS_MULTIPLIER="${NUM_PROMPTS_MULTIPLIER:-5}"
|
|
|
|
# A gain below 2% is treated as throughput saturation. Two consecutive
|
|
# low-gain points prevent one noisy measurement from stopping the search.
|
|
TPS_MIN_GAIN_PCT="${TPS_MIN_GAIN_PCT:-2.0}"
|
|
PLATEAU_PATIENCE="${PLATEAU_PATIENCE:-2}"
|
|
|
|
# Stop a shape when p95 TTFT exceeds the SLO; keep group skipping disabled.
|
|
TTFT_SLO_MS="${TTFT_SLO_MS:-4000}"
|
|
ENABLE_TTFT_SLO_STOP="${ENABLE_TTFT_SLO_STOP:-1}"
|
|
|
|
# Keep the same random workload semantics as the fixed matrix baseline.
|
|
# DATASET_PATH must contain at least SEARCH_MAX_CONCURRENCY times
|
|
# NUM_PROMPTS_MULTIPLIER valid two-turn conversations. Set this explicitly to
|
|
# random-ids to use generated token IDs without a ShareGPT seed dataset.
|
|
BENCH_DATASET_NAME="${BENCH_DATASET_NAME:-random}"
|
|
# SGLang interprets 0.0 as Uniform[1, requested_len]. Use 1.0 for fixed
|
|
# ISL/OSL points; lower values intentionally benchmark a length distribution.
|
|
RANDOM_RANGE_RATIO="${RANDOM_RANGE_RATIO:-1.0}"
|
|
# Before each measured point, warm up with the same concurrency so lazy kernel
|
|
# compilation and CUDA graph capture are excluded from TTFT/TPS. 0 means no
|
|
# cap; set a positive cap only when very high-concurrency warmup is impractical.
|
|
BENCH_WARMUP_MAX_REQUESTS="${BENCH_WARMUP_MAX_REQUESTS:-0}"
|
|
|
|
# Reject a point if the completed request count or actual token lengths do not
|
|
# match the requested workload.
|
|
INPUT_LENGTH_TOLERANCE_PCT="${INPUT_LENGTH_TOLERANCE_PCT:-5.0}"
|
|
OUTPUT_LENGTH_TOLERANCE_PCT="${OUTPUT_LENGTH_TOLERANCE_PCT:-10.0}"
|
|
|
|
MAX_POINT_RETRIES="${MAX_POINT_RETRIES:-1}"
|
|
SERVER_RESTART_COOLDOWN_S="${SERVER_RESTART_COOLDOWN_S:-10}"
|
|
SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-1800}"
|
|
GPU_MEM_SAMPLE_INTERVAL_S="${GPU_MEM_SAMPLE_INTERVAL_S:-1}"
|
|
|
|
# Optional space-separated filters, useful for smoke tests:
|
|
# TP_LIST="8" ISL_LIST="1024" OSL_LIST="128"
|
|
TP_LIST="${TP_LIST:-}"
|
|
ISL_LIST="${ISL_LIST:-}"
|
|
OSL_LIST="${OSL_LIST:-}"
|
|
|
|
DRY_RUN="${DRY_RUN:-0}"
|
|
# Counts ISL/OSL shapes per TP/DP config, not individual concurrency probes.
|
|
GRID_LIMIT="${GRID_LIMIT:-0}"
|