qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/experiments/pro6000/kimi3_pro6000_pd_pp8_standard/README.md
Zhiyi Hong 8189942353 [Feature] Add Kimi-K3 standard PD deployment
2026-08-27 14:25:34 +08:00

2.8 KiB
Raw Blame History

Kimi-K3 Standard PD on 8x RTX 6000D Nodes

This experiment validates the minimum standard PD path before enabling any speculative decoding:

  • Prefill: nodes 601-604, PP8/TP4/EP4, FlashInfer MXFP4, chunk size 8K.
  • Decode: nodes 605-608, PP1/TP32/EP32, Marlin, speculative decoding disabled.
  • Transfer: Mooncake 0.3.12.post1 over mlx5_0..3 RDMA.
  • Router: SMG MiniLB on node 605, port 31000.

The PD image adds only upstream SGLang PRs #31869 and #32797 plus Mooncake 0.3.12.post1 on top of the validated phase5 image.

Files

  • config.env: topology and fixed P/D parameters.
  • Dockerfile.pd: reproducible PD image delta.
  • deploy_pd.sh: the only service lifecycle entrypoint.
  • run_pd_validation.sh: standard four-case suite plus 16K-to-512 C1/C8 modes.
  • patches/: unmodified upstream SGLang patches.

P and D must use the same KV page size. This experiment fixes both sides to page_size=64; a mismatch is rejected by the decode server before transfer.

Run

cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_pp8_standard

bash deploy_pd.sh build-image
bash deploy_pd.sh preflight

RUN_ID=kimi3-pd-standard-$(date +%Y%m%d-%H%M%S) \
  bash deploy_pd.sh start

RUN_ID=kimi3-pd-validation-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh

RUN_ID=kimi3-pd-decode512-c1-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh decode512

RUN_ID=kimi3-pd-decode512-c8-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh decode512-c8

bash deploy_pd.sh status
bash deploy_pd.sh stop

The sudo password is read from /data/hzy/.sudo_password by default and is never stored in Git.

Validated Result

Validated on 2026-08-27 with 601-604 as P and 605-608 as D. All 91 requests completed with the requested token counts, and the final service-log scans contained no fatal pattern.

Case Requests Input TPS TTFT P50 / P95 TPOT P50 / P95
64 -> 8, C1 1 42.18 1.217 / 1.217 s 33.27 / 33.27 ms
1K -> 16, C1, non-streaming smoke 1 508.09 1.956 / 1.956 s N/A
16K -> 1, C1 8 2161.23 7.565 / 7.614 s N/A
16K -> 1, C8 40 6364.31 20.555 / 21.345 s N/A
16K -> 512, C1 1 595.27 7.702 / 7.702 s 38.66 / 38.66 ms
16K -> 512, C8 40 2544.27 18.755 / 23.182 s 63.20 / 66.55 ms

Raw results:

  • results/kimi3-pd-validation-final2-20260827-1310/
  • results/kimi3-pd-decode512-20260827-1325/
  • results/kimi3-pd-decode512-c8-20260827-1340/

The first request after service startup spent 4.775 seconds in TTFT while Mooncake established RDMA endpoints. The next two 64-token smoke runs were stable at 1.193 and 1.217 seconds. P logs record TP4-to-TP32 Mamba state-slice transfer, and D logs record RDMA-ready acknowledgements, confirming the real P-to-D path. PP8 completed without the fixed-delay/deadlock symptom, so the optional PP1 control was not required.