1e8c36b7d1
- Scenarios: A prefill 16k/1 cc16, B decode 16k/512 cc16, D 128k/90% prefix-hit cc8, E NCCL_PROTO=Simple protocol control
- Key findings: PP bubble disproven (96% both-stage busy, chunked prefill = natural microbatch pipeline);
prefill wall = PP0 chain compute 22.8s(65%) + TP4 AllReduce 10.4s(30%);
AR 96MB/3.7ms => busbw ~39GB/s ~ 75% of practical PCIe Gen5 (near-optimal, RING_LL kernel name is NCCL 2.28 mixed-plan naming);
NCCL_PROTO forcing refuted: docker -e never reaches scheduler ranks (sglang worker spawn drops NCCL env);
decode step 21-26ms (MoE GEMM 35%, SendRecv 18%, PP1 AllGather 273 launches outside cuda graph);
128k/90%: DSA indexer sm120_fp8_mqa_logits 28%/23% = 1.6x the sparse attention it serves;
with_stack pathological on this nightly (2 repros) - flush wedges server
- Full report: D:\sskj\reports\GLM53_TP4PP2_Profile与优化方向_2026-09-06.md
- Raw traces remain on 60.5:/data/hf_models/pp2tp4_profiles/{A_prefill,B_decode,D_128k,E_nccl_simple}/
20 lines
1.2 KiB
Bash
20 lines
1.2 KiB
Bash
# GLM-5.3-NVFP4 TP4PP2 profile 实验配置（174.1.60.5, 2026-09-06）
|
||
HOST=174.1.60.5
|
||
MODEL=/data/hf_models/GLM-5.3-NVFP4
|
||
IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
|
||
CONTAINER=glm53-nvfp4
|
||
PORT=30000
|
||
|
||
# 部署（与 09-05 获胜配置逐字一致）
|
||
DEPLOY="bash /root/deploy_par.sh --tp 4 --pp-size 2 --disable-overlap-schedule --max-prefill-tokens 16384 nomtp"
|
||
EXTRA_SERVER_DEFAULTS="kv fp8_e4m3 auto; --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune; hicache-ratio 3; mem-frac 0.88; chunked-prefill 8192"
|
||
|
||
# 采集
|
||
PROFILER=sglang /start_profile (CPU+GPU, no stack; with_stack 在本 nightly 病态禁用)
|
||
TRACE_BASE=/data/hf_models/pp2tp4_profiles
|
||
SCENARIOS=A_prefill(16x16384->1 cc16), B_decode(16x16384->512 cc16, 60 steps), D_128k(8x131072 cc8, 90% shared, 16 steps), E_nccl_simple(16k probe x2 fwd)
|
||
|
||
# 基准（干净无 profiler，seed 每轮更换）
|
||
BENCH_CC16="python3 -m sglang.bench_serving --backend sglang --dataset-name random-ids --num-prompts 16 --random-input-len 16384 --random-output-len 512 --max-concurrency 16 --temperature 0.0 --flush-cache --warmup-requests 1"
|
||
BASELINE_CLEAN="input 3350 tok/s; output 104.7 tok/s; TTFT 31.36s; TPOT 50.87ms"
|