This experiment is the real-serving acceptance gate for Kimi-K3 FlashInfer MXFP4 MoE on NVIDIA SM120. It does not modify the existing baseline experiment.

Fixed matrix

Dimension	Values
Nodes	601-604, 8 GPUs per node
Parallelism	TP32, EP32, DP1
MoE runner	`marlin`, `flashinfer_mxfp4`
Input / output	16K / 1 token
Concurrency	8, 16
Chunked prefill	8K, 16K
Requests	40 per repeat
Repeats	3

The eight configurations are identical except for MoE runner and chunk size. The client uses SGLang random-ids, which produces the exact synthetic token length without depending on a ShareGPT file or text truncation. Radix cache is disabled. OSL=1 makes TTFT, E2E latency and input TPS the primary metrics; TPOT is not meaningful for this matrix.

Unique entrypoint

Run on 601 only. The sudo password is supplied at runtime and is never written to Git or result files.

cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill

read -rsp 'sudo password: ' SUDO_PASSWORD; echo
export SUDO_PASSWORD
export RUN_ID=kimi3-moe-prefill-$(date +%Y%m%d-%H%M%S)

bash run_moe_backend_prefill.sh build-image
bash run_moe_backend_prefill.sh run

For an unattended run, use the same entrypoint under tmux:

umask 077
printf '%s\n' "$SUDO_PASSWORD" > /tmp/kimi3-phase5-sudo
unset SUDO_PASSWORD

tmux new-session -d -s kimi3-moe-prefill \
  "cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill && \
   SUDO_PASSWORD_FILE=/tmp/kimi3-phase5-sudo RUN_ID=\"${RUN_ID}\" \
   bash run_moe_backend_prefill.sh all 2>&1 | tee /data/hzy/${RUN_ID}.log"

Delete /tmp/kimi3-phase5-sudo after the run. The file is mode 0600 and is never copied to result directories.

Outputs

results/<RUN_ID>/
  build/            per-node image build and smoke evidence
  service/          exact docker commands and all four server logs
  raw/              one benchmark JSONL per repeat
  bench/            human-readable benchmark logs
  gpu/              per-node GPU snapshots
  results.csv       flat repeat-level metrics
  summary.json      median metrics and FlashInfer-vs-Marlin deltas
  orchestrator.log  lifecycle and progress

The run aborts on a failed service start, any failed request, a missing result, or an image capability smoke failure. Containers are removed on exit.

The candidate image preserves the Kimi base image's dependency set. It installs the patched FlashInfer wheel/AOT libraries and applies only the two production source files from SGLang commit fb929bb; it does not replace the complete SGLang Python tree or require a newer sglang-kernel package.