qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
230 Commits 4 Branches 0 Tags
Commit Graph

47 Commits

Author SHA1 Message Date
yy-fighting
9e56401384 PP+MTP deepdive r3: race bisect (mask tooling, r34 candidate), decode-round quantification (gloo rendezvous stalls + AR spin dominate; 3-source hypothesis refuted), bench-profile crash forensics 2026-09-08 11:24:00 +08:00
yy-fighting
5ed30006a5 [Experiment] GLM-5.3-NVFP4 TP4PP2 round-2 optimization: all config-level quick wins refuted (2026-09-07) 
- Clean 128k/90% baseline filled (TTFT p50 14.70s, TPOT 51.7ms, hit 0.8999) - prior D-scenario bench was profiler-polluted
- chunk 16384 refuted: flashinfer cutlass MoE workspace scales with chunk -> CUDA OOM (3.08GB needed vs 2.11GB free at mem-frac 0.88);
  physics kills the upside (AR is bandwidth-bound so call-count halving saves ~nothing; indexer q*k work is chunk-invariant)
- AR overlap/quant switch scan: no usable path on TP4+PP2+SM120+PCIe
  (quant-communications is NPU-only via hard ValueError; flashinfer AR fusion auto-enable gated SM90/SM100; symm-mem/NVLS NVLink-oriented)
- indexer direction closed: 28% at 128k is DSA's inherent cost (per-request suffix queries are unique -> no cross-request reuse;
  SM120 has only the deepgemm backend; model already shares topk across layers via index_topk_freq=4; ~219 TFLOPS/rank is reasonable for paged-gather)
- Conclusion: current config (chunk 8192 / mem-frac 0.88 / default NCCL) is config-optimal on this stack;
  remaining gains are dev work: PP+MTP upstreaming (decode) > AR-chunk-overlap/quantized-AR kernel dev (prefill)
- Replaced archived deploy_par_605.sh with the actual /root/deploy_par.sh from 60.5 (md5 cf405176...) - includes sglang_patch mounts
 2026-09-08 11:23:59 +08:00
yy-fighting
1e8c36b7d1 [Experiment] GLM-5.3-NVFP4 TP4PP2 torch-profiler profile on 174.1.60.5 (2026-09-06) 
- Scenarios: A prefill 16k/1 cc16, B decode 16k/512 cc16, D 128k/90% prefix-hit cc8, E NCCL_PROTO=Simple protocol control
- Key findings: PP bubble disproven (96% both-stage busy, chunked prefill = natural microbatch pipeline);
  prefill wall = PP0 chain compute 22.8s(65%) + TP4 AllReduce 10.4s(30%);
  AR 96MB/3.7ms => busbw ~39GB/s ~ 75% of practical PCIe Gen5 (near-optimal, RING_LL kernel name is NCCL 2.28 mixed-plan naming);
  NCCL_PROTO forcing refuted: docker -e never reaches scheduler ranks (sglang worker spawn drops NCCL env);
  decode step 21-26ms (MoE GEMM 35%, SendRecv 18%, PP1 AllGather 273 launches outside cuda graph);
  128k/90%: DSA indexer sm120_fp8_mqa_logits 28%/23% = 1.6x the sparse attention it serves;
  with_stack pathological on this nightly (2 repros) - flush wedges server
- Full report: D:\sskj\reports\GLM53_TP4PP2_Profile与优化方向_2026-09-06.md
- Raw traces remain on 60.5:/data/hf_models/pp2tp4_profiles/{A_prefill,B_decode,D_128k,E_nccl_simple}/
 2026-09-08 11:23:59 +08:00
Zhiyi Hong
8189942353 [Feature] Add Kimi-K3 standard PD deployment 2026-08-27 14:25:34 +08:00
Zhiyi Hong
1aa6c0f911 [Docs] narrow Phase 3 artifact scope 2026-08-03 10:12:44 +08:00
Zhiyi Hong
783c9325ae [Artifacts] archive Phase 3 Nsight reports 2026-08-03 10:08:26 +08:00
Zhiyi Hong
56d286bb4e [Docs] define Phase 3 raw artifact archive 2026-08-03 10:03:35 +08:00
Zhiyi Hong
41e2b000a4 [Docs] finalize Phase 3 timeline profiling results 2026-08-02 01:16:49 +08:00
Zhiyi Hong
82b7d91ac0 [BugFix] capture mixed trace after prefill admission 2026-08-02 00:30:46 +08:00
Zhiyi Hong
bc491eeeed [BugFix] align mixed Phase 3 capture with prefill injection 2026-08-02 00:05:54 +08:00
Zhiyi Hong
4628d49755 [Feat] finalize Phase 3 SGLang timeline capture 2026-08-01 19:38:36 +08:00
Zhiyi Hong
c5fa700c50 [Feat] add Phase 2.5 RDMA demand modeling 2026-08-01 02:43:06 +08:00
Zhiyi Hong
119701a417 [BugFix] align Phase 3 captures with active decode 2026-08-01 00:50:45 +08:00
Zhiyi Hong
405608ad23 [BugFix] make Phase 3 artifact collection fail closed 2026-07-31 18:53:19 +08:00
Zhiyi Hong
3b7291e8a0 [Feat] add Phase 3 SGLang timeline profiling 2026-07-31 18:37:12 +08:00
Zhiyi Hong
39fc2ba565 [BugFix] stage Phase 2 communication tool on workers 2026-07-31 16:19:23 +08:00
Zhiyi Hong
30664faa41 [Feat] finalize Phase 2 hardware attribution pipeline 2026-07-31 15:40:20 +08:00
Zhiyi Hong
ca1f2f6337 [Fix] ignore Phase 2 runtime artifacts 2026-07-31 12:24:00 +08:00
Zhiyi Hong
daa4221128 [Feat] add DSV4-Pro two-node SGLang hardware attribution 2026-07-31 12:18:34 +08:00
Zhiyi Hong
ae85217225 [Docs] record DSV4-Pro long-decode results 2026-07-31 00:15:43 +08:00
Zhiyi Hong
06b017483c [Feat] add DSV4-Pro long-decode cases 2026-07-30 23:41:08 +08:00
Zhiyi Hong
75182c6ede [Feat] add Phase 1 sanity gate 2026-07-30 18:59:14 +08:00
Zhiyi Hong
0d3dd86519 [BugFix] enforce RDMA transport for DSV4-Pro TP16 quick map 2026-07-30 18:17:33 +08:00
Zhiyi Hong
d5d96bd7e4 [Feat] add DSV4-Pro two-node SGLang quick map 2026-07-30 14:40:16 +08:00
Zhiyi Hong
e53b2c7e4c fix: JSONL parsing in run_batch, add results/ to gitignore 2026-07-29 17:51:47 +08:00
Zhiyi Hong
c839230c0b feat: add EAGLE speculative decoding experiment (dsv4_pro6000_sglang_tp16_eagle) 2026-07-29 17:51:40 +08:00
Zhiyi Hong
6d3338244b fix: cuda-graph-max-bs=64 (256 OOMs during capture) 2026-07-28 17:31:52 +08:00
Zhiyi Hong
cd362c5ca0 fix: max_running_requests=256, cuda_graph_max_bs=256 to match max concurrency 128 2026-07-28 17:20:25 +08:00
Zhiyi Hong
da1d4d3d78 add tp = 16 deepseek v4 pro sglang bench 2026-07-28 17:09:26 +08:00
yy-fighting
3b0297516e fix: replace 5 buggy parse_results.py (raw_requests bloat) with shared parse_backend.py wrapper 2026-07-22 03:47:22 +00:00
Quantong Qiu
aee25d4088 feat(pro6000): add qwen3_235b_pro6000_sglang_tp8 experiment (code + README + report.md) 2026-07-21 23:54:19 +08:00
Quantong Qiu
8c737b840b [Feat] use nightly SGLang FlashInfer MoE 2026-07-21 15:22:47 +08:00
Quantong Qiu
83f5ea2197 [Fix] skip longer contexts after C1 OOM 2026-07-21 14:19:38 +08:00
Quantong Qiu
f263695e1b Merge branch 'main' of gitee.com:yy-fighting/sskj into auto/main/11665927/9c484989-1 
Signed-off-by: Quantong Qiu <q_qtang@163.com>
 2026-07-21 05:30:44 +00:00
Quantong Qiu
5e864d2393 !13 [Feat] apply validated deployment capacity caps 
Merge pull request !13 from Zhiyi Hong/auto/main/11665927/494ca4a8-1
 2026-07-21 05:26:41 +00:00
Quantong Qiu
0c2256ca1c [Feat] add 6000D DSV4 tiny adaptive benchmarks 2026-07-21 11:54:04 +08:00
Quantong Qiu
db16057962 [Feat] apply validated deployment capacity caps 2026-07-21 11:50:23 +08:00
Quantong Qiu
db25bc7e5f 6000D Results 2026-07-21 11:14:14 +08:00
Quantong Qiu
359df6a7b6 refactor(paths): derive repository files from root 2026-07-20 17:33:51 +08:00
Quantong Qiu
8562c26524 fix(sglang): cap SM120 CUDA graph prefill 2026-07-20 16:06:09 +08:00
Quantong Qiu
5ffbd0a21a fix(bench): use default limits and back off after OOM 2026-07-20 11:48:10 +08:00
Quantong Qiu
958c5778f0 Add new kernels and autotune configurations for DeepseekV4 model 
- Introduced a new CUDA kernel for transposing and packing FP32 into UE8.
- Added binary file for the new kernel.
- Created multiple autotune configuration files for version 0.6.14 of FlashInfer, supporting various input shapes and configurations.
- Added model information JSON for DeepseekV4ForCausalLM, detailing its architecture and capabilities.
 2026-07-18 10:31:24 +08:00
Quantong Qiu
8beed2411f update 8k and 32k 2026-07-17 16:47:25 +08:00
yy-fighting
8652a685e6 rewrite README, add new platform onboarding guide, fix broken scripts/common paths 
- rewrite README with project purpose, standard workflow, corrected index
- add docs/NEW_PLATFORM_GUIDE.md (new GPU onboarding SOP, GLM5.2 reuse)
- fix ../../scripts/common -> ../../../scripts/common in 42 experiment scripts
- refresh stale docs (EXPERIMENT_GUIDE, H200_QUICKSTART, ADAPTIVE_CONCURRENCY_USAGE, BENCHMARK_WORKFLOW)
- remove dead code (dp_proxy.py) and .bak leftovers
- add p800 adaptive results (tp4_dp2/tp8_dp1 metrics + summary)
- gitignore envs/charts and .tmp_charts
 2026-07-17 06:18:05 +00:00
SSKJ Dev
ff218bdeb3 enable TTFT group skip for all add16 benchmarks 2026-07-17 02:04:05 +00:00
SSKJ Dev
589d3b079d enable initial backoff for all add16 benchmarks 2026-07-16 09:05:37 +00:00
SSKJ Dev
a4e38b9e33 Reorganize experiments into hardware-specific subdirectories 
Move all experiments under hardware-specific folders:
- experiments/h200/     : H200 GPU experiments (15 dirs)
- experiments/h20/      : H20 GPU experiments (2 dirs)
- experiments/p800/     : Kunlun P800 experiments (3 dirs)
- experiments/pro6000/    : RTX 6000D experiments (2 dirs)

This improves discoverability and keeps hardware-specific configs
isolated from each other.
 2026-07-16 04:11:07 +00:00