# Kimi-K3 Standard PD on 8x RTX 6000D Nodes

This experiment validates the minimum standard PD path before enabling any
speculative decoding:

- Prefill: nodes 601-604, PP8/TP4/EP4, FlashInfer MXFP4, chunk size 8K.
- Decode: nodes 605-608, PP1/TP32/EP32, Marlin, speculative decoding disabled.
- Transfer: Mooncake 0.3.12.post1 over mlx5_0..3 RDMA.
- Router: SMG MiniLB on node 605, port 31000.

The PD image adds only upstream SGLang PRs #31869 and #32797 plus Mooncake
0.3.12.post1 on top of the validated phase5 image.

## Files

- `config.env`: topology and fixed P/D parameters.
- `Dockerfile.pd`: reproducible PD image delta.
- `deploy_pd.sh`: the only service lifecycle entrypoint.
- `run_pd_validation.sh`: standard four-case suite plus 16K-to-512 C1/C8 modes.
- `patches/`: unmodified upstream SGLang patches.

P and D must use the same KV page size. This experiment fixes both sides to
`page_size=64`; a mismatch is rejected by the decode server before transfer.

## Run

```bash
cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_pp8_standard

bash deploy_pd.sh build-image
bash deploy_pd.sh preflight

RUN_ID=kimi3-pd-standard-$(date +%Y%m%d-%H%M%S) \
  bash deploy_pd.sh start

RUN_ID=kimi3-pd-validation-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh

RUN_ID=kimi3-pd-decode512-c1-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh decode512

RUN_ID=kimi3-pd-decode512-c8-$(date +%Y%m%d-%H%M%S) \
  bash run_pd_validation.sh decode512-c8

bash deploy_pd.sh status
bash deploy_pd.sh stop
```

The sudo password is read from `/data/hzy/.sudo_password` by default and is
never stored in Git.

## Validated Result

Validated on 2026-08-27 with 601-604 as P and 605-608 as D. All 91 requests
completed with the requested token counts, and the final service-log scans
contained no fatal pattern.

| Case | Requests | Input TPS | TTFT P50 / P95 | TPOT P50 / P95 |
|---|---:|---:|---:|---:|
| 64 -> 8, C1 | 1 | 42.18 | 1.217 / 1.217 s | 33.27 / 33.27 ms |
| 1K -> 16, C1, non-streaming smoke | 1 | 508.09 | 1.956 / 1.956 s | N/A |
| 16K -> 1, C1 | 8 | 2161.23 | 7.565 / 7.614 s | N/A |
| 16K -> 1, C8 | 40 | 6364.31 | 20.555 / 21.345 s | N/A |
| 16K -> 512, C1 | 1 | 595.27 | 7.702 / 7.702 s | 38.66 / 38.66 ms |
| 16K -> 512, C8 | 40 | 2544.27 | 18.755 / 23.182 s | 63.20 / 66.55 ms |

Raw results:

- `results/kimi3-pd-validation-final2-20260827-1310/`
- `results/kimi3-pd-decode512-20260827-1325/`
- `results/kimi3-pd-decode512-c8-20260827-1340/`

The first request after service startup spent 4.775 seconds in TTFT while
Mooncake established RDMA endpoints. The next two 64-token smoke runs were
stable at 1.193 and 1.217 seconds. P logs record TP4-to-TP32 Mamba state-slice
transfer, and D logs record RDMA-ready acknowledgements, confirming the real
P-to-D path. PP8 completed without the fixed-delay/deadlock symptom, so the
optional PP1 control was not required.