qqtang/sskj - sskj - 是石科技代码管理系统

qqtang/sskj

Fork 0

Commit Graph

Author	SHA1	Message	Date
yy-fighting	5ed30006a5	[Experiment] GLM-5.3-NVFP4 TP4PP2 round-2 optimization: all config-level quick wins refuted (2026-09-07) - Clean 128k/90% baseline filled (TTFT p50 14.70s, TPOT 51.7ms, hit 0.8999) - prior D-scenario bench was profiler-polluted - chunk 16384 refuted: flashinfer cutlass MoE workspace scales with chunk -> CUDA OOM (3.08GB needed vs 2.11GB free at mem-frac 0.88); physics kills the upside (AR is bandwidth-bound so call-count halving saves ~nothing; indexer q*k work is chunk-invariant) - AR overlap/quant switch scan: no usable path on TP4+PP2+SM120+PCIe (quant-communications is NPU-only via hard ValueError; flashinfer AR fusion auto-enable gated SM90/SM100; symm-mem/NVLS NVLink-oriented) - indexer direction closed: 28% at 128k is DSA's inherent cost (per-request suffix queries are unique -> no cross-request reuse; SM120 has only the deepgemm backend; model already shares topk across layers via index_topk_freq=4; ~219 TFLOPS/rank is reasonable for paged-gather) - Conclusion: current config (chunk 8192 / mem-frac 0.88 / default NCCL) is config-optimal on this stack; remaining gains are dev work: PP+MTP upstreaming (decode) > AR-chunk-overlap/quantized-AR kernel dev (prefill) - Replaced archived deploy_par_605.sh with the actual /root/deploy_par.sh from 60.5 (md5 cf405176...) - includes sglang_patch mounts	2026-09-08 11:23:59 +08:00
yy-fighting	1e8c36b7d1	[Experiment] GLM-5.3-NVFP4 TP4PP2 torch-profiler profile on 174.1.60.5 (2026-09-06) - Scenarios: A prefill 16k/1 cc16, B decode 16k/512 cc16, D 128k/90% prefix-hit cc8, E NCCL_PROTO=Simple protocol control - Key findings: PP bubble disproven (96% both-stage busy, chunked prefill = natural microbatch pipeline); prefill wall = PP0 chain compute 22.8s(65%) + TP4 AllReduce 10.4s(30%); AR 96MB/3.7ms => busbw ~39GB/s ~ 75% of practical PCIe Gen5 (near-optimal, RING_LL kernel name is NCCL 2.28 mixed-plan naming); NCCL_PROTO forcing refuted: docker -e never reaches scheduler ranks (sglang worker spawn drops NCCL env); decode step 21-26ms (MoE GEMM 35%, SendRecv 18%, PP1 AllGather 273 launches outside cuda graph); 128k/90%: DSA indexer sm120_fp8_mqa_logits 28%/23% = 1.6x the sparse attention it serves; with_stack pathological on this nightly (2 repros) - flush wedges server - Full report: D:\sskj\reports\GLM53_TP4PP2_Profile与优化方向_2026-09-06.md - Raw traces remain on 60.5:/data/hf_models/pp2tp4_profiles/{A_prefill,B_decode,D_128k,E_nccl_simple}/	2026-09-08 11:23:59 +08:00

Author

SHA1

Message

Date

yy-fighting

5ed30006a5

[Experiment] GLM-5.3-NVFP4 TP4PP2 round-2 optimization: all config-level quick wins refuted (2026-09-07)

- Clean 128k/90% baseline filled (TTFT p50 14.70s, TPOT 51.7ms, hit 0.8999) - prior D-scenario bench was profiler-polluted
- chunk 16384 refuted: flashinfer cutlass MoE workspace scales with chunk -> CUDA OOM (3.08GB needed vs 2.11GB free at mem-frac 0.88);
  physics kills the upside (AR is bandwidth-bound so call-count halving saves ~nothing; indexer q*k work is chunk-invariant)
- AR overlap/quant switch scan: no usable path on TP4+PP2+SM120+PCIe
  (quant-communications is NPU-only via hard ValueError; flashinfer AR fusion auto-enable gated SM90/SM100; symm-mem/NVLS NVLink-oriented)
- indexer direction closed: 28% at 128k is DSA's inherent cost (per-request suffix queries are unique -> no cross-request reuse;
  SM120 has only the deepgemm backend; model already shares topk across layers via index_topk_freq=4; ~219 TFLOPS/rank is reasonable for paged-gather)
- Conclusion: current config (chunk 8192 / mem-frac 0.88 / default NCCL) is config-optimal on this stack;
  remaining gains are dev work: PP+MTP upstreaming (decode) > AR-chunk-overlap/quantized-AR kernel dev (prefill)
- Replaced archived deploy_par_605.sh with the actual /root/deploy_par.sh from 60.5 (md5 cf405176...) - includes sglang_patch mounts

2026-09-08 11:23:59 +08:00

yy-fighting

1e8c36b7d1

[Experiment] GLM-5.3-NVFP4 TP4PP2 torch-profiler profile on 174.1.60.5 (2026-09-06)

- Scenarios: A prefill 16k/1 cc16, B decode 16k/512 cc16, D 128k/90% prefix-hit cc8, E NCCL_PROTO=Simple protocol control
- Key findings: PP bubble disproven (96% both-stage busy, chunked prefill = natural microbatch pipeline);
  prefill wall = PP0 chain compute 22.8s(65%) + TP4 AllReduce 10.4s(30%);
  AR 96MB/3.7ms => busbw ~39GB/s ~ 75% of practical PCIe Gen5 (near-optimal, RING_LL kernel name is NCCL 2.28 mixed-plan naming);
  NCCL_PROTO forcing refuted: docker -e never reaches scheduler ranks (sglang worker spawn drops NCCL env);
  decode step 21-26ms (MoE GEMM 35%, SendRecv 18%, PP1 AllGather 273 launches outside cuda graph);
  128k/90%: DSA indexer sm120_fp8_mqa_logits 28%/23% = 1.6x the sparse attention it serves;
  with_stack pathological on this nightly (2 repros) - flush wedges server
- Full report: D:\sskj\reports\GLM53_TP4PP2_Profile与优化方向_2026-09-06.md
- Raw traces remain on 60.5:/data/hf_models/pp2tp4_profiles/{A_prefill,B_decode,D_128k,E_nccl_simple}/

2026-09-08 11:23:59 +08:00

2 Commits