qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
228 Commits 4 Branches 0 Tags
Commit Graph

228 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhiyi Hong
9652bfdb9d [Artifacts] update B300 matrix through 15:23 2026-09-10 15:37:49 +08:00
Zhiyi Hong
7984c25586 [Artifacts] archive B300 DSV4 and GLM-5.3 matrix snapshot 2026-09-10 11:01:29 +08:00
yy-fighting
9e56401384 PP+MTP deepdive r3: race bisect (mask tooling, r34 candidate), decode-round quantification (gloo rendezvous stalls + AR spin dominate; 3-source hypothesis refuted), bench-profile crash forensics 2026-09-08 11:24:00 +08:00
yy-fighting
5ed30006a5 [Experiment] GLM-5.3-NVFP4 TP4PP2 round-2 optimization: all config-level quick wins refuted (2026-09-07) 
- Clean 128k/90% baseline filled (TTFT p50 14.70s, TPOT 51.7ms, hit 0.8999) - prior D-scenario bench was profiler-polluted
- chunk 16384 refuted: flashinfer cutlass MoE workspace scales with chunk -> CUDA OOM (3.08GB needed vs 2.11GB free at mem-frac 0.88);
  physics kills the upside (AR is bandwidth-bound so call-count halving saves ~nothing; indexer q*k work is chunk-invariant)
- AR overlap/quant switch scan: no usable path on TP4+PP2+SM120+PCIe
  (quant-communications is NPU-only via hard ValueError; flashinfer AR fusion auto-enable gated SM90/SM100; symm-mem/NVLS NVLink-oriented)
- indexer direction closed: 28% at 128k is DSA's inherent cost (per-request suffix queries are unique -> no cross-request reuse;
  SM120 has only the deepgemm backend; model already shares topk across layers via index_topk_freq=4; ~219 TFLOPS/rank is reasonable for paged-gather)
- Conclusion: current config (chunk 8192 / mem-frac 0.88 / default NCCL) is config-optimal on this stack;
  remaining gains are dev work: PP+MTP upstreaming (decode) > AR-chunk-overlap/quantized-AR kernel dev (prefill)
- Replaced archived deploy_par_605.sh with the actual /root/deploy_par.sh from 60.5 (md5 cf405176...) - includes sglang_patch mounts
 2026-09-08 11:23:59 +08:00
yy-fighting
1e8c36b7d1 [Experiment] GLM-5.3-NVFP4 TP4PP2 torch-profiler profile on 174.1.60.5 (2026-09-06) 
- Scenarios: A prefill 16k/1 cc16, B decode 16k/512 cc16, D 128k/90% prefix-hit cc8, E NCCL_PROTO=Simple protocol control
- Key findings: PP bubble disproven (96% both-stage busy, chunked prefill = natural microbatch pipeline);
  prefill wall = PP0 chain compute 22.8s(65%) + TP4 AllReduce 10.4s(30%);
  AR 96MB/3.7ms => busbw ~39GB/s ~ 75% of practical PCIe Gen5 (near-optimal, RING_LL kernel name is NCCL 2.28 mixed-plan naming);
  NCCL_PROTO forcing refuted: docker -e never reaches scheduler ranks (sglang worker spawn drops NCCL env);
  decode step 21-26ms (MoE GEMM 35%, SendRecv 18%, PP1 AllGather 273 launches outside cuda graph);
  128k/90%: DSA indexer sm120_fp8_mqa_logits 28%/23% = 1.6x the sparse attention it serves;
  with_stack pathological on this nightly (2 repros) - flush wedges server
- Full report: D:\sskj\reports\GLM53_TP4PP2_Profile与优化方向_2026-09-06.md
- Raw traces remain on 60.5:/data/hf_models/pp2tp4_profiles/{A_prefill,B_decode,D_128k,E_nccl_simple}/
 2026-09-08 11:23:59 +08:00
Zhiyi Hong
8189942353 [Feature] Add Kimi-K3 standard PD deployment 2026-08-27 14:25:34 +08:00
Zhiyi Hong
1aa6c0f911 [Docs] narrow Phase 3 artifact scope 2026-08-03 10:12:44 +08:00
Zhiyi Hong
783c9325ae [Artifacts] archive Phase 3 Nsight reports 2026-08-03 10:08:26 +08:00
Zhiyi Hong
56d286bb4e [Docs] define Phase 3 raw artifact archive 2026-08-03 10:03:35 +08:00
Zhiyi Hong
41e2b000a4 [Docs] finalize Phase 3 timeline profiling results 2026-08-02 01:16:49 +08:00
Zhiyi Hong
82b7d91ac0 [BugFix] capture mixed trace after prefill admission 2026-08-02 00:30:46 +08:00
Zhiyi Hong
bc491eeeed [BugFix] align mixed Phase 3 capture with prefill injection 2026-08-02 00:05:54 +08:00
Zhiyi Hong
4628d49755 [Feat] finalize Phase 3 SGLang timeline capture 2026-08-01 19:38:36 +08:00
Zhiyi Hong
e1719bd575 [Docs] finalize Phase 2.5 RDMA demand model 2026-08-01 15:29:26 +08:00
Zhiyi Hong
c5fa700c50 [Feat] add Phase 2.5 RDMA demand modeling 2026-08-01 02:43:06 +08:00
Zhiyi Hong
119701a417 [BugFix] align Phase 3 captures with active decode 2026-08-01 00:50:45 +08:00
Zhiyi Hong
405608ad23 [BugFix] make Phase 3 artifact collection fail closed 2026-07-31 18:53:19 +08:00
Zhiyi Hong
771b868757 [Docs] link Phase 2 results to server evidence 2026-07-31 18:43:30 +08:00
Zhiyi Hong
3b7291e8a0 [Feat] add Phase 3 SGLang timeline profiling 2026-07-31 18:37:12 +08:00
Zhiyi Hong
1dc5612e3a [Docs] finalize Phase 2 hardware attribution 2026-07-31 17:38:36 +08:00
Zhiyi Hong
5096661ce3 [Docs] gate phase archives on completed results 2026-07-31 17:05:19 +08:00
Zhiyi Hong
7be3062c51 [Docs] unify phase experiment archive naming 2026-07-31 17:00:22 +08:00
Zhiyi Hong
5f24b7d22f [Docs] record Phase 2 worker staging smoke test 2026-07-31 16:29:03 +08:00
Zhiyi Hong
e9c5f85500 [Docs] document Phase 2 worker staging fix 2026-07-31 16:20:53 +08:00
Zhiyi Hong
39fc2ba565 [BugFix] stage Phase 2 communication tool on workers 2026-07-31 16:19:23 +08:00
Zhiyi Hong
66d1db8581 [Docs] link Phase 2 code walkthrough 2026-07-31 16:01:57 +08:00
Zhiyi Hong
4892c0b14d [Docs] document final Phase 2 attribution workflow 2026-07-31 15:47:10 +08:00
Zhiyi Hong
30664faa41 [Feat] finalize Phase 2 hardware attribution pipeline 2026-07-31 15:40:20 +08:00
Zhiyi Hong
a583c337ba [Docs] restore Phase 2 metric command guide 2026-07-31 14:22:35 +08:00
Zhiyi Hong
72bae06576 [Docs] clarify Phase 2 worker preparation 2026-07-31 14:00:58 +08:00
Zhiyi Hong
337195254a [Docs] summarize Phase 2 hardware attribution 2026-07-31 13:50:27 +08:00
Zhiyi Hong
3964b3d210 [Docs] add Phase 1 and Phase 2 code walkthroughs 2026-07-31 13:13:57 +08:00
Zhiyi Hong
ca1f2f6337 [Fix] ignore Phase 2 runtime artifacts 2026-07-31 12:24:00 +08:00
Zhiyi Hong
daa4221128 [Feat] add DSV4-Pro two-node SGLang hardware attribution 2026-07-31 12:18:34 +08:00
Zhiyi Hong
451782457d [Docs] record two-node sysstat monitoring setup 2026-07-31 10:48:48 +08:00
Zhiyi Hong
ae85217225 [Docs] record DSV4-Pro long-decode results 2026-07-31 00:15:43 +08:00
Zhiyi Hong
06b017483c [Feat] add DSV4-Pro long-decode cases 2026-07-30 23:41:08 +08:00
Zhiyi Hong
25418ec174 [Docs] record completed DSV4-Pro Phase 1 quick map 2026-07-30 23:13:48 +08:00
Zhiyi Hong
75182c6ede [Feat] add Phase 1 sanity gate 2026-07-30 18:59:14 +08:00
Zhiyi Hong
0d3dd86519 [BugFix] enforce RDMA transport for DSV4-Pro TP16 quick map 2026-07-30 18:17:33 +08:00
Zhiyi Hong
595bdde5d7 [Docs] audit DSV4-Pro TP16 TTFT benchmark semantics 2026-07-30 16:41:11 +08:00
Zhiyi Hong
d5d96bd7e4 [Feat] add DSV4-Pro two-node SGLang quick map 2026-07-30 14:40:16 +08:00
shishi
e885fd0dc2 feat(adaptive): support tiered per-ISL TTFT SLO via get_ttft_slo_ms() 
- adaptive_bench_lib.sh: add default get_ttft_slo_ms() fallback (flat TTFT_SLO_MS),
  use it instead of hardcoded TTFT_SLO_MS in SLO comparison and logs,
  add ttft_slo_tiers_desc to run_manifest.json
- glm52_910c config.env: define tiered SLO for GLM-5.2:
  ≤2048:5000ms, ≤8192:8000ms, ≤32768:12000ms, ≤131072:20000ms, >131072:30000ms
  (~70-80% of DSv4-Pro values, since GLM-5.2 has simpler architecture)
- glm52_910c adaptive_config.env: update TTFT_SLO_MS comment noting tiered override

Backward compatible: experiments without get_ttft_slo_ms() keep flat 4000ms behavior.
 2026-07-30 10:47:56 +08:00
shishi
ad4fd2b878 fix: INVALID_WORKLOAD 只跳过当前shape而不中止所有配置 
原来 ADAPTIVE_FATAL_WORKLOAD=1 会设 abort_all=1,导致所有剩余TP/DP配置
全部被跳过。改为 continue 只跳过当前 shape,让后续 shape 和配置正常
继续执行。

这修复了 TP=8/DP=2 在 isl=4096 osl=128 因 KV cache 不足导致 workload
验证失败后,TP=16/DP=1 被跳过的 bug。
 2026-07-30 10:05:28 +08:00
Zhiyi Hong
e53b2c7e4c fix: JSONL parsing in run_batch, add results/ to gitignore 2026-07-29 17:51:47 +08:00
Zhiyi Hong
c839230c0b feat: add EAGLE speculative decoding experiment (dsv4_pro6000_sglang_tp16_eagle) 2026-07-29 17:51:40 +08:00
shishi
4914ff4041 fix(dsv4): disable MTP speculative decoding for fair H20 comparison 
MTP (multi-token prediction) speculative decoding was enabled, giving the
910C an unfair decode throughput advantage over the H20 baseline (which
has no MTP). Disable it so the benchmark measures pure model throughput.

- config.env: add DSV4_ENABLE_MTP=0 (default off)
- start_vllm_docker.sh: only add --speculative-config when DSV4_ENABLE_MTP=1
 2026-07-29 16:50:13 +08:00
shishi
455a78161b fix(910c/glm52): 对齐官方A3教程参数(修DP die分配问题,同dsv4 99a22f0) 
dsv4实验发现DP副本绑定到同一组die的问题(99a22f0),glm52存在相同问题:
缺--max-num-batched-tokens和--api-server-count导致DP worker设备分配异常
(不加api-server-count时vllm为N个DP rank启动N个API server)。

对齐docs.vllm.ai GLM5.2 A3官方教程:
- 加 --max-num-batched-tokens 8192 (官方值,影响DP调度)
- 加 --api-server-count 1 (官方值,避免多API server干扰设备分配)
- 去掉 --kv-cache-dtype fp8 (官方不指定,用默认bfloat16;且此镜像fp8本就未生效)
- 保留 --trust-remote-code / --enable-expert-parallel / enable_dsa_cp (官方有)
 2026-07-29 15:28:54 +08:00
shishi
99a22f05b8 fix(dsv4): align launch params with official A3 tutorial (fixes DP die allocation) 
The previous params caused all 4 DP replicas to bind to the same 4 dies
(die 0-3), leaving 12 dies idle. Root cause was a combination of missing
official params + extra non-official params that interfered with DP
worker device placement.

Verified: with aligned params, TP=4 DP=4 correctly distributes 16 workers
across all 16 dies (8 cards x 2 dies), each at ~57 GB HBM (91% util).

Changes (align to docs.vllm.ai A3 tutorial):
- Add --max-num-batched-tokens 10240 (was missing; affects DP scheduling)
- Add --api-server-count 1 (was missing; without it vllm spawns N API
  servers for N DP ranks, disturbing device assignment)
- Remove --kv-cache-dtype fp8 (official uses default bfloat16)
- Remove --trust-remote-code (official doesn't use it for DSV4)
- Remove enable_dsa_cp from additional-config (official doesn't have it)
- max-model-len: per-TP caps (32768/65536/131072) -> 1048576 for all TPs
  (official uses full 1M; the caps were over-cautious)
- max-num-seqs: per-TP (128/256/256) -> 64 for all (official value)
 2026-07-29 15:09:27 +08:00
shishi
c222ed98b2 feat(910c/glm52): 并行配置改为 4 4 / 8 2 / 16 1 对标H20的 2 4 / 4 2 / 8 1 
A3 910C 有16 dies(8卡x2die),H20有8卡。为公平对比,TP按卡数等效:
  A3 TP=4  DP=4 (4die/副本x4) == H20 TP=2 DP=4 (2卡/副本x4)
  A3 TP=8  DP=2 (8die/副本x2) == H20 TP=4 DP=2 (4卡/副本x2)
  A3 TP=16 DP=1 (16die/副本x1) == H20 TP=8 DP=1 (8卡/副本x1)

与dsv4实验(a65849b)保持一致的配置思路。

新增TP=4 per-TP参数覆盖(expert-parallel下专家分4份+dense复制,
KV cache极紧): gpu_mem=0.97, max_model_len=4096, max_num_seqs=32
TP=4可能OOM,若发生会自动记录并跳过。

config.env: PARALLEL_CONFIGS 8 1/16 1 -> 4 4/8 2/16 1; 加TP4_变量
start_vllm_docker.sh: case $TP 加 TP=4 分支
run_adaptive_concurrency_add16.sh: case $tp 加 TP=4 分支
 2026-07-29 13:44:40 +08:00