ad1853f49b
- 冠军 A6 TP4PP2-nomtp@0.90（池 647,040）：hit90 out cc1-4=28.8/47.5/63.9/76.2、cc8/16=106.4/125.7（vs TP2PP4 基线 cc4+14%/cc8+15%），并发独立 128k 文档 4 条、512k 单条 ✓、质量门 7/7；60.8 在役（实测实例切 unless-stopped） - 判决：DP attention 容量负收益判死（非 MoE 权重按 attn 组复制致 dp4 池 96,448/rank + EAGLE×DP 两层崩溃）；DCP 对 DSA 静默算错禁用（dsa_backend 零引用无 guard）；MTP@128k accept 2.07 判负（vs EAGLE 3.46 存活但池 276k 过不了容量门槛） - 交付：60.5 v3 脚本（只交未执行）+ 补丁束（60.8:/root，md5 6922e534）+ profile tp4pp2_hicache.env + CURRENT.md 更新 - 归档：70 文件全量日志（含 DP 臂失败记录、A3 容器日志）+ 7 脚本 + README（双口径协议：hit90 主扫 + distinct-doc 容量探针）
48 lines
3.0 KiB
Bash
48 lines
3.0 KiB
Bash
# GLM-5.3-NVFP4 SGLang TP=4 PP=2 + hicache deployment profile (single RTX 6000D node, 8 GPUs).
|
||
# 2026-09-09 hit90 场景实验优胜配置（60.8 现役；60.5 交付 deploy_glm53_605_v3.sh 待执行）。
|
||
# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/scripts/deploy_glm53_605_v3.sh
|
||
# 60.8 启动口径：bash /root/deploy_ppmtp_r37.sh '--tp 4 --pp-size 2 --disable-overlap-schedule
|
||
# --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length 1048576' nomtp 8192 0.90 1 1 0 0
|
||
#
|
||
# 与 TP2PP4-hicache（glm53_nvfp4_pro6000_sglang_tp2pp4_hicache.env）的关键差异（勿混淆）：
|
||
# - tp2/pp4 → tp4/pp2，memfrac 0.85 → 0.90；其余（cu13+9 挂载、chunk 8192、MRR 16、
|
||
# hicache 3、ctx 1048576、双 parser）逐项同构
|
||
# - 实测 KV 池 647,040（fp8 KV 18.52GB/rank，PP0 初始化后剩 10.94GB）
|
||
# - 取胜依据（hit90=90% 命中 i128k/o512 主场景）：out cc1/2/3/4 = 28.8/47.5/63.9/76.2、
|
||
# cc8/16 = 106.4/125.7（vs TP2PP4：cc4 +14%/cc8 +15%/cc16 +1%）；cap cc2/cc4 = 62.9/98.5
|
||
# 零排队；质量门 7/7
|
||
# - 让步项（知情选择）：并发独立 128k 文档 4 条（TP2PP4 为 6）；512k 单条 151.9s
|
||
# （TP2PP4 为 114.4s，PP4 单条巨请求 prefill 流水更优）；无投机解码
|
||
# - 判决背景：DP attention 对本模型容量负收益、DCP 对 DSA 静默算错、MTP@128k accept 2.07 判负
|
||
# （见 experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/README.md）
|
||
PLATFORM=pro6000
|
||
EXPERIMENT=glm53_nvfp4_hit90_dp_dcp_bench
|
||
MODEL_NAME=GLM-5.3-NVFP4
|
||
ENGINE=sglang
|
||
RUNTIME=docker
|
||
DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729
|
||
CONTAINER_NAME=glm53-nvfp4
|
||
MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
|
||
SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
|
||
PORT=30000
|
||
HEALTH_PATH=/health
|
||
HEALTH_WAIT_S=1800
|
||
CONTAINER_PYTHON=python3
|
||
|
||
TP=4
|
||
PP=2
|
||
MEM_FRACTION_STATIC=0.90
|
||
MAX_RUNNING_REQUESTS=16
|
||
CHUNKED_PREFILL_SIZE=8192
|
||
CONTEXT_LENGTH=1048576
|
||
|
||
DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
|
||
ENGINE_ENV="PYTHONUNBUFFERED=1 SGLANG_PP_DEGLOO=1 SGLANG_PP_SPEC_FORCE_EAGER_DRAFT=1 SGLANG_PP_FORCE_EAGER_VERIFY=0 SGLANG_PP_SPEC_DEBUG=0"
|
||
|
||
DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host --cap-add SYS_PTRACE -p ${PORT}:${PORT}"
|
||
VOLUMES="/data/hf_models:/data/hf_models + 9 patch ro-mounts (full list in scripts/deploy_glm53_605_v3.sh; files live in /root + /root/sglang_patch2 on the host, bundle md5 6922e53439991bc13feee72f3760704f)"
|
||
|
||
BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
|
||
|
||
LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --reasoning-parser glm45 --tool-call-parser glm47 --enable-hierarchical-cache --hicache-ratio 3 --disable-overlap-schedule --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length ${CONTEXT_LENGTH} --host 0.0.0.0 --port ${PORT}"
|