diff --git a/README.md b/README.md
index 3184752..dcd3e92 100644
--- a/README.md
+++ b/README.md
@@ -44,6 +44,7 @@
 | `experiments/p800/dsv4_p800_sglang_tp_dp_matrix/` | P800 + SGLang（INT8；TP2/DP4 启动 OOM 无数据，见 config.env 注释） |
 | `experiments/pro6000/dsv4_pro6000_vllm_tp_dp_matrix/` | RTX 6000D + vLLM |
 | `experiments/pro6000/dsv4_pro6000_sglang_tp_dp_matrix/` | RTX 6000D + SGLang |
+| `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/` | RTX 6000D + SGLang，Kimi-K3（TP32×EP32，部署手册见 docs/KIMI_K3_DEPLOY.md） |
 
 每个目录内：`run_bench.sh` 跑固定并发矩阵；`run_adaptive_concurrency.sh` 从 C=1 指数倍增搜饱和点；`run_adaptive_concurrency_add16.sh` 从 C=16 线性 +16 步进、带 TTFT SLO 停止与回退（当前主力用法，见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`）。
 
diff --git a/deploy/profiles/pro6000/kimi3_pro6000_sglang_tp32ep32.env b/deploy/profiles/pro6000/kimi3_pro6000_sglang_tp32ep32.env
new file mode 100644
index 0000000..addc203
--- /dev/null
+++ b/deploy/profiles/pro6000/kimi3_pro6000_sglang_tp32ep32.env
@@ -0,0 +1,54 @@
+# Kimi-K3 SGLang multi-node TP=32 EP=32 deployment profile (4x RTX 6000D).
+# Nodes: 174.1.60.5~8 (rank 0~3), 32x NVIDIA RTX 6000D (85GB, sm_120).
+#
+# 关键点（实测踩坑，勿随意改）:
+#   - MoE 后端必须 marlin（K3 的 MXFP4 缩放因子为 uint8，DeepGEMM 只接受 fp32/UE8M0）
+#   - RoCE: NCCL_IB_HCA=mlx5_0..3（4 张独立卡, 10.100.21-24/24, RoCEv2 GID index 3）
+#     实测 32-rank 117MB allreduce 2.4ms；勿用 mlx5_bond_0（仅 4.5GB/s）
+#   - 容器必须 --ulimit memlock=-1（否则 ibv_create_cq 报 Cannot allocate memory）
+#   - 不要设 NCCL_ALGO=TREE（CUDA graph 捕获报 "NCCL error: invalid usage"）
+#   - 首次请求含 ~110s Triton JIT 编译，属正常现象，预热一次后回落
+#   - flashkda 可选后端（与 triton prefill 性能等价）见 docs/KIMI_K3_DEPLOY.md 附录
+#
+# Model-team only. Ops only run `python -m sskj.bench` against the served URL.
+
+PLATFORM=pro6000
+EXPERIMENT=kimi3_pro6000_sglang_tp32ep32
+MODEL_NAME=Kimi-K3
+ENGINE=sglang
+RUNTIME=docker
+DOCKER_IMAGE=lmsysorg/sglang:kimi-k3
+CONTAINER_NAME=${EXPERIMENT}_node${NODE_RANK}
+MODEL_PATH=/data/hf_models/Kimi-K3
+SERVED_MODEL_NAME=kimi-k3
+PORT=30000
+HEALTH_PATH=/health
+HEALTH_HOST=174.1.60.5
+HEALTH_WAIT_S=2400
+CONTAINER_PYTHON=python3
+
+# ---- 并行度（固定，勿改）----
+TP=32
+DP=1
+EP_SIZE=32
+
+# ---- Multi-node topology (rank order; rank 0 exposes the HTTP API) ----
+NNODES=4
+NODE_HOSTS="174.1.60.5 174.1.60.6 174.1.60.7 174.1.60.8"
+NODE_SSH_USER=root
+LOCAL_NODE_RANK=0
+MASTER_IP=174.1.60.5
+DIST_PORT=20000
+
+DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
+ENGINE_ENV="NCCL_SOCKET_IFNAME=bond0 GLOO_SOCKET_IFNAME=bond0 NCCL_IB_HCA=mlx5_0,mlx5_1,mlx5_2,mlx5_3 NCCL_IB_GID_INDEX=3 NCCL_IB_TIMEOUT=22 NCCL_IB_RETRY_CNT=7 NCCL_CUMEM_ENABLE=1 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 SGLANG_MOE_FUSED_GATE_RADIX=1"
+
+DOCKER_FLAGS="--gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband --shm-size 32g --entrypoint ''"
+VOLUMES="${MODEL_PATH}:${MODEL_PATH}:ro"
+PATCH_MOUNTS="/tmp/patch_k3_sm120.py:/tmp/patch_k3_sm120.py:ro"
+
+# BOOTSTRAP 在容器内执行: 打 sm_120 补丁 → 按节点 rank 计算 SGLANG_HOST_IP → 启动 sglang。
+# SGLANG_HOST_IP 必须为本节点实际 IP（174.1.60.5~8 = 5 + NODE_RANK）。
+BOOTSTRAP="python3 /tmp/patch_k3_sm120.py && export SGLANG_HOST_IP=\"174.1.60.$((5 + ${NODE_RANK}))\" && exec python3 -m sglang.launch_server ${LAUNCH_ARGS}"
+
+LAUNCH_ARGS="--model-path ${MODEL_PATH} --served-model-name ${SERVED_MODEL_NAME} --tp-size ${TP} --ep-size 32 --nnodes ${NNODES} --node-rank ${NODE_RANK} --dist-init-addr ${MASTER_IP}:${DIST_PORT} --trust-remote-code --moe-runner-backend marlin --mem-fraction-static 0.88 --cuda-graph-max-bs-decode 16 --mamba-radix-cache-strategy extra_buffer_lazy --dist-timeout 3600 --mamba-full-memory-ratio 0.36 --host 0.0.0.0 --port ${PORT}"
diff --git a/docs/KIMI_K3_DEPLOY.md b/docs/KIMI_K3_DEPLOY.md
new file mode 100644
index 0000000..4b24b89
--- /dev/null
+++ b/docs/KIMI_K3_DEPLOY.md
@@ -0,0 +1,194 @@
+# Kimi-K3 部署手册（运维版）
+
+> 面向运维的一键部署手册：按顺序复制粘贴命令即可完成 Kimi-K3 在 4 节点
+> RTX 6000D 集群上的部署。**不需要理解 SGLang 引擎参数。**
+> 部署配置统一在 `deploy/profiles/pro6000/kimi3_pro6000_sglang_tp32ep32.env`，
+> 压测统一走 `python -m sskj.bench`（见 `ops/README.md`）。
+
+## 0. 概述
+
+| 项 | 值 |
+|---|---|
+| 模型 | Kimi-K3（Moonshot AI，2.8T 参数 MXFP4 量化，~1.5TB） |
+| 节点 | 6000D-5~8 = 174.1.60.5~8（每节点 8× RTX 6000D 85GB，共 32 卡） |
+| 镜像 | `lmsysorg/sglang:kimi-k3`（专用镜像，普通 sglang 镜像不识别该模型架构） |
+| 并行 | TP=32 × EP=32（跨 4 节点） |
+| 服务地址 | `http://174.1.60.5:30000`（OpenAI 兼容） |
+| 模型 ID | `kimi-k3` |
+| 部署方式 | `python -m sskj.deploy start --profile pro6000/kimi3_pro6000_sglang_tp32ep32` |
+
+## 1. 前置检查（每步都过再继续）
+
+```bash
+# 1.1 4 节点 8 卡全部空闲（应全部显示 0 MiB / 0%）
+for i in 5 6 7 8; do echo "== 174.1.60.$i"; ssh 174.1.60.$i 'nvidia-smi --query-gpu=index,memory.used --format=csv,noheader'; done
+# 注意：如显示被占用，联系模型团队确认（其他用户的作业需先让出）
+
+# 1.2 磁盘空间（模型 1.5TB + 运行余量，/data 需 ≥ 2TB 可用）
+ssh 174.1.60.5 'df -h /data'
+
+# 1.3 RoCE 网卡就绪（应输出 4 个 ACTIVE 且 link_layer=Ethernet）
+ssh 174.1.60.5 'cat /sys/class/infiniband/mlx5_*/ports/1/link_layer; cat /sys/class/infiniband/mlx5_*/ports/1/state'
+
+# 1.4 节点互信（174.1.60.5 免密登录其余节点；若失败执行步骤 1.5）
+ssh 174.1.60.5 'for i in 6 7 8; do ssh -o StrictHostKeyChecking=no 174.1.60.$i hostname; done'
+
+# 1.5 若 1.4 失败，在 174.1.60.5 上执行（把 .5 的 root 公钥发到各节点）
+ssh 174.1.60.5 'for i in 6 7 8; do ssh-copy-id -o StrictHostKeyChecking=no root@174.1.60.$i; done'
+```
+
+## 2. 模型下载（仅首次，~1.5TB，需数小时）
+
+模型权重放 `/data/hf_models/Kimi-K3`（4 节点同一路径，本集群已就绪则跳过本节）。
+
+```bash
+# 在 174.1.60.5 上执行（国内走 ModelScope 最快；modelscope CLI 需先 pip install modelscope）
+pip3 install -q modelscope
+mkdir -p /data/hf_models
+nohup modelscope download --model moonshotai/Kimi-K3 \
+  --local_dir /data/hf_models/Kimi-K3 > /data/hf_models/download_k3.log 2>&1 &
+# 查看进度
+tail -f /data/hf_models/download_k3.log
+```
+
+下载完成后校验（96 个分片，共约 1.5TB）：
+
+```bash
+ssh 174.1.60.5 'ls /data/hf_models/Kimi-K3/*.safetensors | wc -l; du -sh /data/hf_models/Kimi-K3'
+# 期望：96（safetensors 数量）、1.5T（总大小）
+```
+
+**分发到其余 3 台**（.5 上执行；源盘 NVMe 读是瓶颈，聚合约 3.5GB/s）：
+
+```bash
+ssh 174.1.60.5 'for i in 6 7 8; do rsync -aH --partial /data/hf_models/Kimi-K3/ 174.1.60.$i:/data/hf_models/Kimi-K3/ & done; wait'
+```
+
+> 每节点需约 1.5T 空闲磁盘（`df -h /data` 确认）。
+
+## 3. 镜像准备（已拉取则跳过）
+
+```bash
+# 4 节点并行拉取（约 9.6GB；如 Docker Hub 直连失败，各机 daemon.json 需配镜像加速：
+#   "registry-mirrors": ["https://docker.m.daocloud.io", "https://docker.xuanyuan.me"]）
+for i in 5 6 7 8; do ssh 174.1.60.$i 'docker pull lmsysorg/sglang:kimi-k3' & done; wait
+
+# 验证镜像存在
+ssh 174.1.60.5 'docker images lmsysorg/sglang:kimi-k3 --format "{{.Repository}}:{{.Tag}} {{.Size}}"'
+```
+
+## 4. 补丁与依赖分发
+
+Kimi-K3 镜像在消费级 Blackwell（sm_120，RTX 6000D）上有一个内核不兼容点，
+需要打补丁（补丁源文件在本仓库 `platforms/patches/pro6000/kimi_k3/patch_k3_sm120.py`，
+修复 attn_res 融合内核误用数据中心 Blackwell 专属的 tcgen05 指令问题）。
+
+```bash
+# 在 174.1.60.5 上，把补丁分发到 4 节点 /tmp（部署时容器自动挂载并执行）
+ssh 174.1.60.5 '
+  for i in 5 6 7 8; do
+    scp /data/yy/sskj/platforms/patches/pro6000/kimi_k3/patch_k3_sm120.py 174.1.60.$i:/tmp/patch_k3_sm120.py
+  done
+  for i in 5 6 7 8; do ssh 174.1.60.$i "sha256sum /tmp/patch_k3_sm120.py"; done
+'
+# 4 台输出的 sha256 必须一致（内容校验）
+```
+
+## 5. 一键部署
+
+```bash
+cd /data/yy/sskj
+
+# 5.1 先预览要执行的命令（不实际启动）
+PYTHONPATH=src python3 -m sskj.deploy start \
+  --profile pro6000/kimi3_pro6000_sglang_tp32ep32 --dry-run
+
+# 5.2 正式启动（4 节点同时拉起，耗时 8~12 分钟：模型加载 + CUDA graph 捕获）
+PYTHONPATH=src python3 -m sskj.deploy start \
+  --profile pro6000/kimi3_pro6000_sglang_tp32ep32
+
+# 5.3 查看状态
+PYTHONPATH=src python3 -m sskj.deploy status \
+  --profile pro6000/kimi3_pro6000_sglang_tp32ep32
+```
+
+就绪判定（满足其一）：
+- 步骤 5.3 `status` 显示容器 Up 且健康检查通过
+- 或手动验证：`curl -s http://174.1.60.5:30000/health` 返回 `{"status":"ok"}`
+
+## 6. 验证服务
+
+```bash
+# 6.1 模型列表
+curl -s http://174.1.60.5:30000/v1/models | head -c 300
+# 应包含 "id": "kimi-k3"
+
+# 6.2 推理冒烟（正确性：23×47 应算得 1081，输出可能带思考标签属正常）
+curl -s -m 300 http://174.1.60.5:30000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model": "kimi-k3", "messages": [{"role": "user", "content": "What is 23*47? Answer briefly."}], "max_tokens": 128, "temperature": 0.1}'
+
+# 6.3 服务日志（确认无 ERROR）
+ssh 174.1.60.5 'docker logs kimi3_pro6000_sglang_tp32ep32_node0 2>&1 | tail -20'
+```
+
+> ⚠️ **首次请求很慢（TTFT 可达 100+ 秒）是正常现象**：Triton/FlashKDA 内核首次
+> JIT 编译。发一个任意请求预热后，后续请求恢复正常（16K 输入 TTFT ~6s）。
+
+## 7. 停止 / 重启
+
+```bash
+cd /data/yy/sskj
+
+# 停止（4 节点容器全部移除）
+PYTHONPATH=src python3 -m sskj.deploy stop \
+  --profile pro6000/kimi3_pro6000_sglang_tp32ep32
+
+# 重启 = 再次执行 5.2 的 start 命令（幂等，会先清旧容器）
+
+# 手动兜底（start/stop 异常时）：4 节点各执行
+for i in 5 6 7 8; do ssh 174.1.60.$i 'docker rm -f kimi3_pro6000_sglang_tp32ep32_node*'; done
+```
+
+## 8. 性能自检（可选）
+
+```bash
+# 稳态基准：16K 输入 TTFT / prefill 吞吐 / TPOT（先跑一次丢弃 JIT 冷启动）
+cd /data/yy/sskj
+scp -o ConnectTimeout=10 /data/flashkda_deploy/bench_16k_steady.py 174.1.60.5:/tmp/ 2>/dev/null || true
+ssh 174.1.60.5 'python3 /tmp/bench_16k_steady.py 3'
+# 参考值（稳态）：TTFT ~6s / prefill ~2700 tok/s / TPOT ~39ms / decode ~26 tok/s
+```
+
+正式压测（矩阵 + 自适应并发搜索）走 bench 层：
+
+```bash
+cd /data/yy/sskj/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32
+DRY_RUN=1 bash run_adaptive_concurrency_add16.sh        # 先看计划
+bash run_adaptive_concurrency_add16.sh                  # 正式跑（放 tmux）
+```
+
+## 9. 故障排查速查
+
+| 现象 | 原因 | 处理 |
+|---|---|---|
+| 首次请求 TTFT 100+ 秒 | Triton/FlashKDA 内核首次 JIT 编译 | 正常现象，预热一次即可 |
+| `Not enough GPU memory for hybrid mamba state cache` | 某节点显存被其他作业占用 | 检查 4 节点 `nvidia-smi`，确认 8 卡全空闲后重启部署 |
+| `ibv_create_cq failed: Cannot allocate memory` | 容器 memlock 限制 | 部署配置已带 `--ulimit memlock=-1`，检查 profile 未被改动 |
+| `NCCL error: invalid usage`（graph 捕获时） | 误设了 `NCCL_ALGO=TREE` | 确认环境变量里没有 NCCL_ALGO |
+| 启动后 `/health` 一直不通 | 分布式初始化失败/节点未就绪 | 看 master 日志；确认 4 节点 GPU 全空闲后 `stop` + 重新 `start` |
+| 推理结果带 `<\|open\|>think` 标签 | K3 思考型模型正常输出格式 | 非故障；如需精简可调 chat template（联系模型团队） |
+| 并发压测时部分请求失败 | flashkda 后端已知问题（未修复） | 当前默认 triton 后端无此问题；如误用 flashkda 见附录 |
+
+## 附录 A：flashkda 可选后端（默认不启用）
+
+FlashKDA（MoonshotAI CUTLASS KDA 内核，支持 sm_120）可作为 KDA prefill 的备选后端。
+实测与默认 triton prefill **性能等价**（16K prefill 2693 vs 2695 tok/s），仅作备份用途。
+
+启用方式（需模型团队协助）：
+1. 构建 wheel（脚本见 `/data/flashkda_deploy/01_build_flashkda.sh`，产物
+   `flash_kda-0.0.1-cp312-cp312-linux_x86_64.whl` 分发到 4 节点 `/tmp/`）
+2. profile 增加 wheel 挂载与安装步骤，并在 LAUNCH_ARGS 加
+   `--linear-attn-prefill-backend flashkda`
+
+已知问题：flashkda 后端在 4/8 并发压测下服务异常（纯 triton 无此问题），修复前不建议生产使用。
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/adaptive_config.env b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/adaptive_config.env
new file mode 100644
index 0000000..3cf7dd7
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/adaptive_config.env
@@ -0,0 +1,56 @@
+# Adaptive concurrency search settings.
+#
+# For each fixed (TP, DP, ISL, OSL), probe:
+#   C = start, start * multiplier, ... up to max
+# and stop after Total TPS has less than TPS_MIN_GAIN_PCT meaningful growth for
+# PLATEAU_PATIENCE consecutive points.
+
+SEARCH_START_CONCURRENCY="${SEARCH_START_CONCURRENCY:-1}"
+SEARCH_MAX_CONCURRENCY="${SEARCH_MAX_CONCURRENCY:-64}"
+
+# At the add16 initial probe, restart and retry C=8 then C=1 after an OOM.
+ENABLE_INITIAL_OOM_BACKOFF="${ENABLE_INITIAL_OOM_BACKOFF:-1}"
+SEARCH_MULTIPLIER="${SEARCH_MULTIPLIER:-2}"
+NUM_PROMPTS_MULTIPLIER="${NUM_PROMPTS_MULTIPLIER:-5}"
+
+# A gain below 2% is treated as throughput saturation. Two consecutive
+# low-gain points prevent one noisy measurement from stopping the search.
+TPS_MIN_GAIN_PCT="${TPS_MIN_GAIN_PCT:-2.0}"
+PLATEAU_PATIENCE="${PLATEAU_PATIENCE:-2}"
+
+# Stop a shape when p95 TTFT exceeds the SLO; keep group skipping disabled.
+TTFT_SLO_MS="${TTFT_SLO_MS:-4000}"
+ENABLE_TTFT_SLO_STOP="${ENABLE_TTFT_SLO_STOP:-1}"
+
+# Keep the same random workload semantics as the fixed matrix baseline.
+# DATASET_PATH must contain at least SEARCH_MAX_CONCURRENCY times
+# NUM_PROMPTS_MULTIPLIER valid two-turn conversations. Set this explicitly to
+# random-ids to use generated token IDs without a ShareGPT seed dataset.
+BENCH_DATASET_NAME="${BENCH_DATASET_NAME:-random}"
+# SGLang interprets 0.0 as Uniform[1, requested_len]. Use 1.0 for fixed
+# ISL/OSL points; lower values intentionally benchmark a length distribution.
+RANDOM_RANGE_RATIO="${RANDOM_RANGE_RATIO:-1.0}"
+# Before each measured point, warm up with the same concurrency so lazy kernel
+# compilation and CUDA graph capture are excluded from TTFT/TPS. 0 means no
+# cap; set a positive cap only when very high-concurrency warmup is impractical.
+BENCH_WARMUP_MAX_REQUESTS="${BENCH_WARMUP_MAX_REQUESTS:-0}"
+
+# Reject a point if the completed request count or actual token lengths do not
+# match the requested workload.
+INPUT_LENGTH_TOLERANCE_PCT="${INPUT_LENGTH_TOLERANCE_PCT:-5.0}"
+OUTPUT_LENGTH_TOLERANCE_PCT="${OUTPUT_LENGTH_TOLERANCE_PCT:-10.0}"
+
+MAX_POINT_RETRIES="${MAX_POINT_RETRIES:-1}"
+SERVER_RESTART_COOLDOWN_S="${SERVER_RESTART_COOLDOWN_S:-10}"
+SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-1800}"
+GPU_MEM_SAMPLE_INTERVAL_S="${GPU_MEM_SAMPLE_INTERVAL_S:-1}"
+
+# Optional space-separated filters, useful for smoke tests:
+#   TP_LIST="8" ISL_LIST="1024" OSL_LIST="128"
+TP_LIST="${TP_LIST:-}"
+ISL_LIST="${ISL_LIST:-}"
+OSL_LIST="${OSL_LIST:-}"
+
+DRY_RUN="${DRY_RUN:-0}"
+# Counts ISL/OSL shapes per TP/DP config, not individual concurrency probes.
+GRID_LIMIT="${GRID_LIMIT:-0}"
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/config.env b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/config.env
new file mode 100644
index 0000000..4ebb7e6
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/config.env
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# Kimi-K3 TP=32 EP=32 benchmark experiment on 4x RTX 6000D (174.1.60.5~8).
+# 固定单配置（TP32×EP32，跨 4 节点）；服务器生命周期走 deploy profile
+# `pro6000/kimi3_pro6000_sglang_tp32ep32`（部署参数以 profile 为准，勿在此重复）。
+
+EXPERIMENT="kimi3_pro6000_sglang_tp32ep32"
+MODEL_NAME="Kimi-K3"
+MODEL_PATH="/data/hf_models/Kimi-K3"
+SERVED_MODEL_NAME="kimi-k3"
+
+SGLANG_PORT="${SGLANG_PORT:-30000}"
+
+# Python interpreter for orchestration scripts (parse_backend.py, compare.py, etc.)
+# and the benchmark client. Defaults to the system python3 if the sglang venv
+# does not exist on the host.
+VENV_CLIENT="${VENV_CLIENT:-/root/.miniconda3/envs/sglang}"
+
+# Run the benchmark client natively (0) or inside Docker (1).
+USE_DOCKER_CLIENT="${USE_DOCKER_CLIENT:-1}"
+
+export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}"
+
+# Runtime working directory for logs, pid files, and tmp.
+RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}"
+
+# 单配置：TP=32 DP=1（EP=32 与 RoCE 等参数固定在 deploy profile 内）
+declare -a PARALLEL_CONFIGS=(
+  "32 1"
+)
+
+# K3 服务器参数（与 deploy profile 保持一致，供本地/诊断脚本引用）。
+MEM_FRACTION_STATIC="${MEM_FRACTION_STATIC:-0.88}"
+MOE_RUNNER_BACKEND="${MOE_RUNNER_BACKEND:-marlin}"
+EP_SIZE="${EP_SIZE:-32}"
+MAX_RUNNING_REQUESTS="${MAX_RUNNING_REQUESTS:-64}"
+
+# Deployment switch. 1 = Docker（走 deploy profile），0 = 本地 venv（仅单节点调试用）。
+USE_DOCKER="${USE_DOCKER:-1}"
+DOCKER_IMAGE="${DOCKER_IMAGE:-lmsysorg/sglang:kimi-k3}"
+
+# Deploy profile used by start/stop scripts and the adaptive search loop.
+DEPLOY_PROFILE="${DEPLOY_PROFILE:-pro6000/kimi3_pro6000_sglang_tp32ep32}"
+
+# To use ShareGPT, set BENCH_DATASET_NAME=random and DATASET_PATH explicitly.
+BENCH_DATASET_NAME="${BENCH_DATASET_NAME:-random}"
+DATASET_PATH="${DATASET_PATH:-${ROOT_DIR}/dataset/ShareGPT_V3_unfiltered_cleaned_split.json}"
+SGLANG_BENCH_MODULE="${SGLANG_BENCH_MODULE:-sglang.benchmark.serving}"
+
+# Matrix and concurrency rules are defined in matrix.json by default.
+MATRIX_FILE="${MATRIX_FILE:-${SCRIPT_DIR:-.}/matrix.json}"
+MATRIX_MODE="${MATRIX_MODE:-Y}"
+
+# Sampling density for concurrency.
+export CONCURRENCY_SAMPLES="${CONCURRENCY_SAMPLES:-2}"
+
+# Per-scenario timeout to avoid hangs (seconds). K3 首请求含 JIT 编译（~110s），
+# 预热由 adaptive 框架处理；超时给足。
+SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-3600}"
+
+# GPU memory sampling interval (seconds).
+GPU_MEM_SAMPLE_INTERVAL_S="${GPU_MEM_SAMPLE_INTERVAL_S:-1}"
+
+# Dry-run mode: if 1, only log the server args and scenario plan without starting
+# any server or sending requests.
+DRY_RUN="${DRY_RUN:-0}"
+
+# Per-config scenario limit for quick smoke tests. 0 = run all generated scenarios.
+GRID_LIMIT="${GRID_LIMIT:-0}"
+
+# PyTorch CUDA allocator setting for the SGLang server.
+PYTORCH_CUDA_ALLOC_CONF="${PYTORCH_CUDA_ALLOC_CONF:-expandable_segments:True}"
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/matrix.json b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/matrix.json
new file mode 100644
index 0000000..3b2c73d
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/matrix.json
@@ -0,0 +1,38 @@
+{
+  "comment": "ISL/OSL matrix for kimi3_pro6000_sglang_tp32ep32 (multi-node TP32 EP32). Y=must test, P=optional, N=skip. K3 最大上下文 ~364K tokens（max_total_num_tokens），ISL 上限 16384 保守可测。",
+  "mode": "Y",
+  "matrix": {
+    "1024": {
+      "128": "Y",
+      "256": "Y",
+      "512": "Y",
+      "1024": "Y",
+      "2048": "Y",
+      "4096": "Y"
+    },
+    "4096": {
+      "128": "Y",
+      "256": "Y",
+      "512": "Y",
+      "1024": "Y",
+      "2048": "Y",
+      "4096": "Y"
+    },
+    "8192": {
+      "128": "Y",
+      "256": "Y",
+      "512": "Y",
+      "1024": "Y",
+      "2048": "Y",
+      "4096": "Y"
+    },
+    "16384": {
+      "128": "Y",
+      "256": "Y",
+      "512": "Y",
+      "1024": "Y",
+      "2048": "Y",
+      "4096": "Y"
+    }
+  }
+}
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_adaptive_concurrency_add16.sh b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_adaptive_concurrency_add16.sh
new file mode 100755
index 0000000..6651355
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_adaptive_concurrency_add16.sh
@@ -0,0 +1,163 @@
+#!/usr/bin/env bash
+# Find the Total-TPS saturation concurrency for each SGLang TP/DP/ISL/OSL shape.
+set -Eeuo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
+
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/adaptive_config.env"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/deploy_cli.sh"
+
+DEPLOY_PROFILE="${DEPLOY_PROFILE:-pro6000/kimi3_pro6000_sglang_tp32ep32}"
+
+ENGINE="sglang"
+ENGINE_PORT="$SGLANG_PORT"
+RESULT_BASE="${RESULT_BASE:-${SCRIPT_DIR}/adaptive_results}"
+ACTIVE_ENGINE_SERVER_LOG=""
+
+if [[ -x "${VENV_CLIENT}/bin/python" ]]; then
+  PYTHON="${VENV_CLIENT}/bin/python"
+else
+  PYTHON="$(command -v python3)"
+fi
+
+DOCKER_IMAGE="${DOCKER_IMAGE:-lmsysorg/sglang:nightly-dev-cu13-20260720-b3570a45}"
+
+engine_is_healthy() {
+  curl --fail --silent --show-error --max-time 5 \
+    "http://127.0.0.1:${ENGINE_PORT}/health" >/dev/null 2>&1
+}
+
+engine_stop_server() {
+  local tp="$1"
+  local dp="$2"
+
+  log "stopping sglang server tp=${tp} dp=${dp} via deploy profile"
+  bash "${SCRIPT_DIR}/stop_sglang_docker.sh" "$tp" "$dp" >> "${ADAPTIVE_LOG_DIR}/sglang_tp${tp}_dp${dp}.server.outer.log" 2>&1 || true
+  ACTIVE_ENGINE_SERVER_LOG=""
+  sleep 2
+}
+
+engine_build_server_args() {
+  local tp="$1"
+  local dp="$2"
+  deploy_render_args "$DEPLOY_PROFILE" "$tp" "$dp" "$ENGINE_PORT" "$MODEL_PATH"
+}
+
+engine_start_server() {
+  local tp="$1"
+  local dp="$2"
+  local outer_log="${ADAPTIVE_LOG_DIR}/sglang_tp${tp}_dp${dp}.server.outer.log"
+  log "starting sglang server tp=${tp} dp=${dp}"
+  if [[ -n "${CONTAINER_NAME:-}" ]]; then
+    bash "${SCRIPT_DIR}/run_sglang_in_container.sh" "$tp" "$dp" >> "$outer_log" 2>&1
+  else
+    bash "${SCRIPT_DIR}/start_sglang_dp.sh" "$tp" "$dp" >> "$outer_log" 2>&1
+  fi
+  if ! engine_is_healthy; then
+    log "ERROR: sglang health check failed tp=${tp} dp=${dp}"
+    return 1
+  fi
+  if [[ -z "${CONTAINER_NAME:-}" ]]; then
+    ACTIVE_ENGINE_SERVER_LOG="$(
+      find "${RUNTIME_BASE}/logs" -maxdepth 1 -type f \
+        -name "${EXPERIMENT}_sglang*tp${tp}_dp${dp}_*.log" \
+        -printf '%T@ %p\n' 2>/dev/null | sort -nr | head -n 1 | cut -d' ' -f2-
+    )"
+  fi
+  log "sglang server healthy tp=${tp} dp=${dp} log=${ACTIVE_ENGINE_SERVER_LOG:-container:/tmp/sglang_server.log}"
+}
+
+engine_detect_oom() {
+  local detail_log="$1"
+  local tp="$2"
+  local dp="$3"
+  local pattern='CUDA out of memory|torch\.OutOfMemoryError|OutOfMemory|out of memory|OOM|RESOURCE_EXHAUSTED|Failed to allocate memory'
+  local outer_log="${ADAPTIVE_LOG_DIR}/sglang_tp${tp}_dp${dp}.server.outer.log"
+  local -a logs=("$detail_log" "$outer_log")
+  if [[ -n "$ACTIVE_ENGINE_SERVER_LOG" ]]; then
+    logs+=("$ACTIVE_ENGINE_SERVER_LOG")
+  fi
+  if grep -Eiq "$pattern" "${logs[@]}" 2>/dev/null; then
+    return 0
+  fi
+  if [[ -n "${CONTAINER_NAME:-}" ]]; then
+    docker exec "$CONTAINER_NAME" grep -Eiq "$pattern" /tmp/sglang_server.log 2>/dev/null
+    return $?
+  fi
+  return 1
+}
+
+engine_run_bench() {
+  local isl="$1"
+  local osl="$2"
+  local concurrency="$3"
+  local num_prompts="$4"
+  local output_file="$5"
+  local warmup_requests
+  warmup_requests="$(adaptive_warmup_request_count "$concurrency")"
+  local -a bench_args=(
+    --backend sglang
+    --host 127.0.0.1
+    --port "$ENGINE_PORT"
+    --dataset-name "$BENCH_DATASET_NAME"
+    --random-input-len "$isl"
+    --random-output-len "$osl"
+    --random-range-ratio "$RANDOM_RANGE_RATIO"
+    --num-prompts "$num_prompts"
+    --max-concurrency "$concurrency"
+    --request-rate 10000
+    --warmup-requests "$warmup_requests"
+    --output-file "$output_file"
+    --output-details
+    --disable-tqdm
+  )
+  if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
+    bench_args+=(--dataset-path "$DATASET_PATH")
+  elif [[ "$BENCH_DATASET_NAME" == "random-ids" ]]; then
+    : # random-ids does not need --tokenize-prompt
+  else
+    bench_args+=(--tokenize-prompt)
+  fi
+
+  if [[ "$USE_DOCKER_CLIENT" == "1" ]]; then
+    local -a volume_args=(-v "${MODEL_PATH}:${MODEL_PATH}:ro" -v "${RESULT_BASE}:${RESULT_BASE}")
+    if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
+      volume_args+=(-v "${DATASET_PATH}:${DATASET_PATH}:ro")
+    fi
+    docker run --rm \
+      --network host \
+      "${volume_args[@]}" \
+      -e PYTHONUNBUFFERED=1 \
+      --entrypoint python3 \
+      "$DOCKER_IMAGE" \
+      -m "$SGLANG_BENCH_MODULE" "${bench_args[@]}"
+  else
+    "$PYTHON" -m "$SGLANG_BENCH_MODULE" "${bench_args[@]}"
+  fi
+}
+
+export -f engine_run_bench
+export ENGINE_PORT MODEL_PATH RESULT_BASE DOCKER_IMAGE USE_DOCKER_CLIENT
+export BENCH_DATASET_NAME DATASET_PATH RANDOM_RANGE_RATIO BENCH_WARMUP_MAX_REQUESTS PYTHON SGLANG_BENCH_MODULE
+
+adaptive_main "$@"
+
+export SEARCH_START_CONCURRENCY=16
+export SEARCH_ADDEND=16
+# If the initial concurrency violates the TTFT SLO, search downward. Stop at
+# the first acceptable value (16 -> 8; only try 1 when 8 still violates it).
+export SEARCH_INITIAL_BACKOFF_CONCURRENCIES="8 1"
+# When concurrency 1 still has a severely excessive TTFT, stop the remaining
+# shapes in this TP/DP group. Zero disables this rule.
+export TTFT_GROUP_SKIP_MS="${TTFT_GROUP_SKIP_MS:-8000}"
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_bench.sh b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_bench.sh
new file mode 100755
index 0000000..e1180bd
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/run_bench.sh
@@ -0,0 +1,528 @@
+#!/usr/bin/env bash
+# Kimi-K3 TP32×EP32 benchmark (multi-node SGLang).
+set -Eeuo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
+
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/deploy_cli.sh"
+
+RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')}"
+RESULT_BASE="${SCRIPT_DIR}/results"
+MATRIX_FILE="${MATRIX_FILE:-${SCRIPT_DIR}/matrix.json}"
+MATRIX_MODE="${MATRIX_MODE:-Y}"
+SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-1800}"
+GPU_MEM_SAMPLE_INTERVAL_S="${GPU_MEM_SAMPLE_INTERVAL_S:-1}"
+DRY_RUN="${DRY_RUN:-0}"
+GRID_LIMIT="${GRID_LIMIT:-0}"
+
+# Export variables used inside functions that are called via bash -c subshells.
+export BENCH_DATASET_NAME DATASET_PATH MODEL_PATH RESULT_BASE DOCKER_IMAGE USE_DOCKER_CLIENT SGLANG_BENCH_MODULE
+
+if [[ -x "${VENV_CLIENT}/bin/python" ]]; then
+  PYTHON="${VENV_CLIENT}/bin/python"
+else
+  PYTHON="$(command -v python3)"
+fi
+
+DOCKER_IMAGE="${DOCKER_IMAGE:-lmsysorg/sglang:nightly-dev-cu13-20260720-b3570a45}"
+
+log_dir_global="${RESULT_BASE}/${RUN_ID}/logs"
+mkdir -p "$log_dir_global"
+log_init "${log_dir_global}/orchestrator.log"
+
+log "experiment=${EXPERIMENT_NAME} run_id=${RUN_ID} platform=${PLATFORM} hardware=${HARDWARE}"
+log "matrix_mode=${MATRIX_MODE} matrix_file=${MATRIX_FILE} dry_run=${DRY_RUN} grid_limit=${GRID_LIMIT}"
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+is_server_healthy() {
+  curl --fail --silent --show-error --max-time 5 "http://127.0.0.1:${SGLANG_PORT}/health" >/dev/null 2>&1
+}
+
+stop_server() {
+  local tp="$1"
+  local dp="$2"
+
+  log "stopping sglang server tp=${tp} dp=${dp} via deploy profile"
+  bash "${SCRIPT_DIR}/stop_sglang_docker.sh" "$tp" "$dp" >> "${log_dir_global}/sglang_tp${tp}_dp${dp}.server.outer.log" 2>&1 || true
+
+  # Fallback: kill any SGLang launch processes for this model.
+  pkill -9 -f "sglang.launch_server.*${MODEL_NAME}" 2>/dev/null || true
+  pkill -9 -f "sglang.launch_server.*${MODEL_PATH}" 2>/dev/null || true
+  sleep 2
+}
+
+build_server_args() {
+  local tp="$1"
+  local dp="$2"
+
+  deploy_render_args \
+    "${DEPLOY_PROFILE:-pro6000/kimi3_pro6000_sglang_tp32ep32}" \
+    "$tp" "$dp" \
+    "${SGLANG_PORT:-30000}" \
+    "$MODEL_PATH"
+}
+
+start_server() {
+  local tp="$1"
+  local dp="$2"
+
+  log "starting sglang server tp=${tp} dp=${dp}"
+  if [[ -n "${CONTAINER_NAME:-}" ]]; then
+    bash "${SCRIPT_DIR}/run_sglang_in_container.sh" "$tp" "$dp" \
+      >> "${log_dir_global}/sglang_tp${tp}_dp${dp}.server.outer.log" 2>&1
+  else
+    bash "${SCRIPT_DIR}/start_sglang_dp.sh" "$tp" "$dp" \
+      >> "${log_dir_global}/sglang_tp${tp}_dp${dp}.server.outer.log" 2>&1
+  fi
+
+  if ! is_server_healthy; then
+    log "error: sglang server tp=${tp} dp=${dp} failed health check on port ${SGLANG_PORT}"
+    return 1
+  fi
+  log "sglang server tp=${tp} dp=${dp} is healthy on port ${SGLANG_PORT}"
+}
+
+restart_server() {
+  local tp="$1"
+  local dp="$2"
+  log "restarting sglang server tp=${tp} dp=${dp} after non-OOM failure"
+  stop_server "$tp" "$dp"
+  sleep 10
+  start_server "$tp" "$dp"
+}
+
+run_bench_serving() {
+  # Inject the offline workload choice consistently for warmup and main runs.
+  local -a dataset_args=(--dataset-name "$BENCH_DATASET_NAME")
+  if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
+    dataset_args+=(--dataset-path "$DATASET_PATH")
+  elif [[ "$BENCH_DATASET_NAME" == "random-ids" ]]; then
+    : # random-ids does not need --tokenize-prompt
+  else
+    dataset_args+=(--tokenize-prompt)
+  fi
+  if [[ "${USE_DOCKER_CLIENT:-1}" == "1" ]]; then
+    local vol_args=()
+    vol_args+=("-v" "${MODEL_PATH}:${MODEL_PATH}:ro")
+    if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
+      vol_args+=("-v" "${DATASET_PATH}:${DATASET_PATH}:ro")
+    fi
+    vol_args+=("-v" "${RESULT_BASE}:${RESULT_BASE}")
+    docker run --rm \
+      --network host \
+      "${vol_args[@]}" \
+      -e PYTHONUNBUFFERED=1 \
+      --entrypoint python3 \
+      "${DOCKER_IMAGE}" \
+      -m "$SGLANG_BENCH_MODULE" "${dataset_args[@]}" "$@"
+  else
+    "$PYTHON" -m "$SGLANG_BENCH_MODULE" "${dataset_args[@]}" "$@"
+  fi
+}
+export -f run_bench_serving
+
+run_warmup() {
+  local input_len="$1"
+  local output_len="$2"
+
+  log "warming up (input=${input_len}, output=${output_len}, num=1)"
+  bash -c '
+    run_bench_serving \
+      --backend sglang \
+      --host 127.0.0.1 \
+      --port "'"$SGLANG_PORT"'" \
+      --random-input-len "'"$input_len"'" \
+      --random-output-len "'"$output_len"'" \
+      --num-prompts 1 \
+      --max-concurrency 1 \
+      --request-rate 10000 \
+      --output-file /dev/null \
+      --output-details \
+      >> "'"${log_dir_global}/warmup.log"'" 2>&1
+  '
+  log "warmup completed"
+}
+
+scenario_already_completed() {
+  local output_file="$1"
+  local expected="$2"
+  [[ -s "$output_file" ]] || return 1
+  local completed
+  completed="$("$PYTHON" -c "
+import json, sys
+path = sys.argv[1]
+try:
+    with open(path, 'r', encoding='utf-8') as f:
+        for line in f:
+            line = line.strip()
+            if line:
+                data = json.loads(line)
+                print(data.get('completed', 0))
+                break
+except Exception:
+    print(0)
+" "$output_file")"
+  [[ "${completed:-0}" -ge "$expected" ]]
+}
+
+scenario_already_processed() {
+  local result_root="$1"
+  local scenario_name="$2"
+  local json_path="${result_root}/results.json"
+  [[ -f "$json_path" ]] || return 1
+  "$PYTHON" -c "
+import json, sys
+path, name = sys.argv[1], sys.argv[2]
+try:
+    with open(path, 'r', encoding='utf-8') as f:
+        data = json.load(f)
+    for s in data.get('scenarios', []):
+        if s.get('name') == name:
+            if s.get('status') or s.get('metrics', {}).get('success', 0) > 0:
+                sys.exit(0)
+except Exception:
+    pass
+sys.exit(1)
+" "$json_path" "$scenario_name"
+}
+
+detect_oom() {
+  local detail_log="$1"
+  local server_outer_log="$2"
+  local pattern='CUDA out of memory|torch\.OutOfMemoryError|OutOfMemory|out of memory|OOM|RESOURCE_EXHAUSTED|Failed to allocate memory'
+  if grep -Eiq "$pattern" "$detail_log" "$server_outer_log" 2>/dev/null; then
+    return 0
+  fi
+  return 1
+}
+
+start_gpu_monitor() {
+  local csv_path="$1"
+  mkdir -p "$(dirname "$csv_path")"
+  nvidia-smi \
+    --query-gpu=timestamp,index,memory.used,memory.total,utilization.gpu \
+    --format=csv \
+    -l "$GPU_MEM_SAMPLE_INTERVAL_S" \
+    > "$csv_path" 2>/dev/null &
+  echo $!
+}
+
+stop_gpu_monitor() {
+  local pid="$1"
+  if kill -0 "$pid" 2>/dev/null; then
+    kill "$pid" 2>/dev/null || true
+    sleep 1
+    kill -9 "$pid" 2>/dev/null || true
+  fi
+}
+
+append_scenario_record() {
+  local result_root="$1"
+  local json_path="$result_root/results.json"
+  shift
+  local scenario_json
+  scenario_json="$("$PYTHON" -c "
+import json, sys
+pairs = [a.split('=', 1) for a in sys.argv[1:]]
+d = {}
+for k, v in pairs:
+    try:
+        d[k] = json.loads(v)
+    except json.JSONDecodeError:
+        d[k] = v
+print(json.dumps(d, ensure_ascii=False))
+" "$@")"
+  PYTHON="$PYTHON" append_scenario_to_json "$json_path" "$scenario_json"
+}
+
+record_skipped_csv() {
+  local csv_path="$1"
+  shift
+  # Args: key=value
+  local row
+  row="$("$PYTHON" -c "
+import csv, json, sys, io
+pairs = [a.split('=', 1) for a in sys.argv[1:]]
+d = {}
+for k, v in pairs:
+    try:
+        d[k] = json.loads(v)
+    except json.JSONDecodeError:
+        d[k] = v
+buf = io.StringIO()
+writer = csv.DictWriter(buf, fieldnames=['engine','tp','dp','mark','isl','dsl','concurrency','status','reason','detail_log'], extrasaction='ignore')
+writer.writerow(d)
+print(buf.getvalue().strip())
+" "$@")"
+  echo "$row" >> "$csv_path"
+}
+
+skip_remaining_scenarios() {
+  local result_root="$1"
+  local scenario_tsv="$2"
+  local start_index="$3"
+  local status="$4"
+  local reason="$5"
+  local tp="$6"
+  local dp="$7"
+  local skipped_csv="${RESULT_BASE}/${RUN_ID}/skipped_after_oom.csv"
+
+  local i=0
+  tail -n +2 "$scenario_tsv" | while IFS=$'\t' read -r mark isl dsl conc num; do
+    if (( i < start_index )); then
+      i=$((i + 1))
+      continue
+    fi
+    i=$((i + 1))
+    local sname="c${conc}_i${isl}_o${dsl}"
+    if scenario_already_processed "$result_root" "$sname"; then
+      continue
+    fi
+    append_scenario_record "$result_root" \
+      "name=${sname}" \
+      "config=$(jq -n --arg phase main --argjson c "$conc" --argjson i "$isl" --argjson o "$dsl" --arg dataset "$BENCH_DATASET_NAME" --argjson n "$num" '{phase: $phase, concurrency: $c, input_len: $i, output_len: $o, dataset: $dataset, num_prompts: $n}')" \
+      "status=\"${status}\"" \
+      "note=\"${reason}\""
+    record_skipped_csv "$skipped_csv" \
+      "engine=sglang" "tp=${tp}" "dp=${dp}" "mark=${mark}" "isl=${isl}" "dsl=${dsl}" "concurrency=${conc}" "status=${status}" "reason=${reason}"
+  done
+}
+
+# ---------------------------------------------------------------------------
+# Per-configuration runner
+# ---------------------------------------------------------------------------
+
+run_parallel_config() {
+  local tp="$1"
+  local dp="$2"
+  local config_label="tp${tp}_dp${dp}"
+  local result_root="${RESULT_BASE}/${RUN_ID}/${config_label}"
+  local raw_dir="${result_root}/raw_outputs"
+  local gpu_log_dir="${result_root}/gpu_logs"
+  local phase_log_dir="${result_root}/logs"
+  mkdir -p "$raw_dir" "$gpu_log_dir" "$phase_log_dir"
+
+  log "===== ${config_label} START ====="
+
+  # Generate scenario list for this config.
+  local scenario_tsv="${result_root}/scenarios.tsv"
+  "$PYTHON" "${SCRIPT_DIR}/generate_scenarios.py" \
+    --matrix "$MATRIX_FILE" \
+    --mode "$MATRIX_MODE" \
+    > "$scenario_tsv"
+  local total_scenarios
+  total_scenarios="$(tail -n +2 "$scenario_tsv" | wc -l)"
+  log "generated ${total_scenarios} scenarios for ${config_label}"
+
+  # Write metadata.
+  ensure_result_root "$result_root"
+  write_metadata_json \
+    "${result_root}/results.json" \
+    "${EXPERIMENT_NAME}_${config_label}" \
+    "$RUN_ID" \
+    "$MODEL_PATH" \
+    "sglang" \
+    "sglang" \
+    "$HARDWARE" \
+    "$ACCELERATOR" \
+    "$CHIP" \
+    "experiments/${EXPERIMENT_NAME}/run_bench.sh" \
+    "$DOCKER_IMAGE" \
+    "Kimi-K3 TP32×EP32 matrix on 4x RTX 6000D"
+
+  local server_args_str
+  server_args_str="$(build_server_args "$tp" "$dp")"
+  jq --arg tp "$tp" --arg dp "$dp" --arg cuda "$CUDA_VISIBLE_DEVICES" --arg args "$server_args_str" \
+    '.config = {
+       "tp": ($tp | tonumber),
+       "dp": ($dp | tonumber),
+       "cuda_visible_devices": $cuda,
+       "backend": "sglang",
+       "server_start_script": "experiments/'${EXPERIMENT_NAME}'/start_sglang_dp.sh",
+       "server_args": $args
+     }' "${result_root}/results.json" > "${result_root}/results.json.tmp" && \
+    mv "${result_root}/results.json.tmp" "${result_root}/results.json"
+
+  if [[ "$DRY_RUN" == "1" ]]; then
+    log "DRY_RUN: would start server with args: ${server_args_str}"
+    local line
+    tail -n +2 "$scenario_tsv" | while IFS=$'\t' read -r mark isl dsl conc num; do
+      log "DRY_RUN: ${config_label} scenario mark=${mark} c=${conc} i=${isl} o=${dsl} n=${num}"
+    done
+    log "===== ${config_label} DONE (dry run) ====="
+    return 0
+  fi
+
+  # Initialize skipped_after_oom.csv for this run.
+  local skipped_csv="${RESULT_BASE}/${RUN_ID}/skipped_after_oom.csv"
+  if [[ ! -f "$skipped_csv" ]]; then
+    echo "engine,tp,dp,mark,isl,dsl,concurrency,status,reason,detail_log" > "$skipped_csv"
+  fi
+
+  # Start server once for this TP×DP config.
+  if ! start_server "$tp" "$dp"; then
+    log "ERROR: ${config_label} failed to start; skipping all scenarios"
+    skip_remaining_scenarios "$result_root" "$scenario_tsv" 0 "SKIPPED_SERVICE_START_FAILED" "service failed to start" "$tp" "$dp"
+    log "===== ${config_label} DONE ====="
+    return 0
+  fi
+
+  # Warmup with a small prompt before the first scenario.
+  run_warmup 1024 128 || true
+
+  # Read scenarios into an array so we can skip remaining entries on failure.
+  local -a scenarios=()
+  while IFS= read -r line; do
+    scenarios+=("$line")
+  done < <(tail -n +2 "$scenario_tsv")
+
+  local i mark isl dsl conc num
+  local output_file detail_log gpu_csv sname bench_rc
+  for (( i = 0; i < ${#scenarios[@]}; i++ )); do
+    IFS=$'\t' read -r mark isl dsl conc num <<< "${scenarios[$i]}"
+
+    if [[ "$GRID_LIMIT" -gt 0 && "$i" -ge "$GRID_LIMIT" ]]; then
+      log "GRID_LIMIT=${GRID_LIMIT} reached; skipping remaining scenarios"
+      skip_remaining_scenarios "$result_root" "$scenario_tsv" "$i" "SKIPPED_GRID_LIMIT" "GRID_LIMIT reached" "$tp" "$dp"
+      break
+    fi
+
+    sname="c${conc}_i${isl}_o${dsl}"
+    output_file="${raw_dir}/sglang_main_${conc}_${isl}_${dsl}.jsonl"
+    detail_log="${phase_log_dir}/sglang_${config_label}_${sname}.log"
+    gpu_csv="${gpu_log_dir}/gpu_mem_${conc}_${isl}_${dsl}.csv"
+
+    if scenario_already_completed "$output_file" "$num" || scenario_already_processed "$result_root" "$sname"; then
+      log "skipping already-processed ${config_label} scenario: ${sname}"
+      continue
+    fi
+
+    log "running ${config_label} scenario: mark=${mark} c=${conc} i=${isl} o=${dsl} n=${num}"
+
+    local gpu_pid
+    gpu_pid="$(start_gpu_monitor "$gpu_csv")"
+
+    bench_rc=0
+    timeout "$SCENARIO_TIMEOUT_S" bash -c '
+      run_bench_serving \
+        --backend sglang \
+        --host 127.0.0.1 \
+        --port "'"$SGLANG_PORT"'" \
+        --random-input-len "'"$isl"'" \
+        --random-output-len "'"$dsl"'" \
+        --num-prompts "'"$num"'" \
+        --max-concurrency "'"$conc"'" \
+        --request-rate 10000 \
+        --output-file "'"$output_file"'" \
+        --output-details \
+        > "'"$detail_log"'" 2>&1
+    ' || bench_rc=$?
+
+    stop_gpu_monitor "$gpu_pid"
+
+    if [[ "$bench_rc" -eq 0 ]]; then
+      log "finished ${config_label} scenario: output=${output_file}"
+      append_scenario_record "$result_root" \
+        "name=${sname}" \
+        "config=$(jq -n --arg phase main --argjson c "$conc" --argjson i "$isl" --argjson o "$dsl" --arg dataset "$BENCH_DATASET_NAME" --argjson n "$num" '{phase: $phase, concurrency: $c, input_len: $i, output_len: $o, dataset: $dataset, num_prompts: $n}')" \
+        "status=\"completed\"" \
+        "note=\"benchmark finished successfully\""
+      continue
+    fi
+
+    # Failure handling.
+    if detect_oom "$detail_log" "${log_dir_global}/sglang_tp${tp}_dp${dp}.server.outer.log"; then
+      log "ERROR: ${config_label} scenario ${sname} triggered OOM; stopping config"
+      append_scenario_record "$result_root" \
+        "name=${sname}" \
+        "config=$(jq -n --arg phase main --argjson c "$conc" --argjson i "$isl" --argjson o "$dsl" --arg dataset "$BENCH_DATASET_NAME" --argjson n "$num" '{phase: $phase, concurrency: $c, input_len: $i, output_len: $o, dataset: $dataset, num_prompts: $n}')" \
+        "status=\"OOM\"" \
+        "note=\"detected CUDA out-of-memory\""
+      record_skipped_csv "$skipped_csv" \
+        "engine=sglang" "tp=${tp}" "dp=${dp}" "mark=${mark}" "isl=${isl}" "dsl=${dsl}" "concurrency=${conc}" "status=OOM" "reason=detected CUDA out-of-memory" "detail_log=${detail_log}"
+      stop_server "$tp" "$dp"
+      skip_remaining_scenarios "$result_root" "$scenario_tsv" "$((i + 1))" "SKIPPED_AFTER_OOM" "previous case OOM" "$tp" "$dp"
+      break
+    fi
+
+    log "ERROR: ${config_label} scenario ${sname} failed (rc=${bench_rc}); see ${detail_log}"
+    if [[ "$mark" == "P" ]]; then
+      log "optional (P) scenario failed; recording as skipped and continuing"
+      append_scenario_record "$result_root" \
+        "name=${sname}" \
+        "config=$(jq -n --arg phase main --argjson c "$conc" --argjson i "$isl" --argjson o "$dsl" --arg dataset "$BENCH_DATASET_NAME" --argjson n "$num" '{phase: $phase, concurrency: $c, input_len: $i, output_len: $o, dataset: $dataset, num_prompts: $n}')" \
+        "status=\"skipped_optional\"" \
+        "note=\"optional scenario failed (rc=${bench_rc})\""
+      record_skipped_csv "$skipped_csv" \
+        "engine=sglang" "tp=${tp}" "dp=${dp}" "mark=${mark}" "isl=${isl}" "dsl=${dsl}" "concurrency=${conc}" "status=skipped_optional" "reason=optional scenario failed (rc=${bench_rc})" "detail_log=${detail_log}"
+      continue
+    fi
+
+    # Mandatory scenario failed but not OOM: try to restart the server.
+    if restart_server "$tp" "$dp"; then
+      run_warmup 1024 128 || true
+      log "resuming ${config_label} after server restart"
+      continue
+    fi
+
+    log "ERROR: ${config_label} server restart failed; skipping remaining scenarios"
+    append_scenario_record "$result_root" \
+      "name=${sname}" \
+      "config=$(jq -n --arg phase main --argjson c "$conc" --argjson i "$isl" --argjson o "$dsl" --arg dataset "$BENCH_DATASET_NAME" --argjson n "$num" '{phase: $phase, concurrency: $c, input_len: $i, output_len: $o, dataset: $dataset, num_prompts: $n}')" \
+      "status=\"FAILED\"" \
+      "note=\"scenario failed and server restart failed (rc=${bench_rc})\""
+    record_skipped_csv "$skipped_csv" \
+      "engine=sglang" "tp=${tp}" "dp=${dp}" "mark=${mark}" "isl=${isl}" "dsl=${dsl}" "concurrency=${conc}" "status=FAILED" "reason=scenario failed and server restart failed" "detail_log=${detail_log}"
+    skip_remaining_scenarios "$result_root" "$scenario_tsv" "$((i + 1))" "SKIPPED_RESTART_FAILED" "server restart failed" "$tp" "$dp"
+    break
+  done
+
+  stop_server "$tp" "$dp"
+
+  # Parse results.
+  log "parsing ${config_label} results"
+  "$PYTHON" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend sglang \
+    >> "${phase_log_dir}/parse.log" 2>&1 || {
+      log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
+    }
+
+  log "===== ${config_label} DONE ====="
+}
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+# Cleanup any leftovers.
+for cfg in "${PARALLEL_CONFIGS[@]}"; do
+  read -r tp dp <<< "$cfg"
+  stop_server "$tp" "$dp"
+done
+
+# Run each parallel configuration.
+for cfg in "${PARALLEL_CONFIGS[@]}"; do
+  read -r tp dp <<< "$cfg"
+  run_parallel_config "$tp" "$dp"
+done
+
+# Generate cross-configuration comparison.
+log "generating comparison report"
+"$PYTHON" "${SCRIPT_DIR}/compare.py" \
+  --run-root "${RESULT_BASE}/${RUN_ID}" \
+  --output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
+  >> "${log_dir_global}/compare.log" 2>&1 || {
+    log "WARNING: comparison script failed; see ${log_dir_global}/compare.log"
+  }
+
+log "all results saved to ${RESULT_BASE}/${RUN_ID}"
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_docker.sh b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_docker.sh
new file mode 100755
index 0000000..8a476b2
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_docker.sh
@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+# Start the SGLang TPxDP server through the shared deployment layer.
+# Usage: start_sglang_docker.sh <TP> <DP>
+set -Eeuo pipefail
+
+TP="${1:-}"
+DP="${2:-}"
+if [[ -z "$TP" || -z "$DP" ]]; then
+  echo "Usage: $0 <TP> <DP>"
+  exit 1
+fi
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/deploy_cli.sh"
+
+RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}"
+mkdir -p "${RUNTIME_BASE}/logs" "${RUNTIME_BASE}/tmp"
+
+log "starting SGLang server tp=${TP} dp=${DP} via deploy profile"
+deploy_start \
+  "${DEPLOY_PROFILE:-pro6000/kimi3_pro6000_sglang_tp32ep32}" \
+  "$TP" "$DP" \
+  "${RUNTIME_BASE}/logs" \
+  "${SGLANG_PORT:-30000}" \
+  "$MODEL_PATH" \
+  "${EXPERIMENT}_sglang_tp${TP}_dp${DP}"
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_dp.sh b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_dp.sh
new file mode 100755
index 0000000..b7e69e3
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/start_sglang_dp.sh
@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Start SGLang server for a given TP×DP configuration.
+# Usage: start_sglang_dp.sh <TP> <DP>
+#
+# By default this delegates to the Docker start script because the experiment
+# is intended to run SGLang inside a container. Set USE_DOCKER=0 to use the
+# local VENV_CLIENT environment instead.
+set -e
+
+TP="${1}"
+DP="${2}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+
+if [[ "${USE_DOCKER:-1}" == "1" ]]; then
+  exec "${SCRIPT_DIR}/start_sglang_docker.sh" "$@"
+fi
+
+RUNTIME_BASE="${RUNTIME_BASE:-${SCRIPT_DIR}/runtime}"
+mkdir -p "${RUNTIME_BASE}/logs" "${RUNTIME_BASE}/tmp"
+
+VENV="${VENV_CLIENT}"
+export PATH="$VENV/bin:$PATH"
+export PYTHONUNBUFFERED=1
+export PYTORCH_CUDA_ALLOC_CONF="${PYTORCH_CUDA_ALLOC_CONF:-expandable_segments:True}"
+export TMPDIR="${RUNTIME_BASE}/tmp"
+export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES}"
+
+LOG="${RUNTIME_BASE}/logs/${EXPERIMENT}_sglang_tp${TP}_dp${DP}_$(date +%Y%m%d_%H%M%S).log"
+PID_FILE="${RUNTIME_BASE}/${EXPERIMENT}_sglang_tp${TP}_dp${DP}.pid"
+
+rm -f "$PID_FILE"
+
+SERVER_ARGS=(
+  python3 -m sglang.launch_server
+  --model-path "$MODEL_PATH"
+  --trust-remote-code
+  --tp-size "$TP"
+  --moe-runner-backend "$MOE_RUNNER_BACKEND"
+  --mem-fraction-static "$MEM_FRACTION_STATIC"
+  --context-length "$CONTEXT_LENGTH"
+  --max-running-requests "$MAX_RUNNING_REQUESTS"
+  --host 0.0.0.0
+  --port "$SGLANG_PORT"
+)
+
+if [[ "$DP" -gt 1 ]]; then
+  SERVER_ARGS+=(
+    --dp-size "$DP"
+  )
+fi
+
+SERVER_ARGS_STR="${SERVER_ARGS[*]}"
+
+echo "=== Starting SGLang server (TP=${TP}, DP=${DP}) ==="
+echo "Model: $MODEL_PATH"
+echo "Port: $SGLANG_PORT"
+echo "Command: $SERVER_ARGS_STR"
+echo "Log: $LOG"
+
+nohup "${SERVER_ARGS[@]}" > "$LOG" 2>&1 &
+
+PID=$!
+echo $PID > "$PID_FILE"
+echo "PID: $PID"
+echo "Waiting for health on port ${SGLANG_PORT}..."
+
+for i in $(seq 1 240); do
+  if curl --fail --silent --show-error --max-time 5 "http://127.0.0.1:${SGLANG_PORT}/health" >/dev/null 2>&1; then
+    echo "SGLang server is ready at http://127.0.0.1:${SGLANG_PORT}"
+    echo "Log: $LOG"
+    exit 0
+  fi
+  if ! kill -0 $PID 2>/dev/null; then
+    echo "ERROR: SGLang server exited early"
+    tail -200 "$LOG"
+    exit 1
+  fi
+  echo "Waiting... ($i/240)"
+  sleep 5
+done
+
+echo "ERROR: SGLang server not healthy after 240 retries"
+tail -200 "$LOG"
+exit 1
diff --git a/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/stop_sglang_docker.sh b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/stop_sglang_docker.sh
new file mode 100644
index 0000000..8b7d117
--- /dev/null
+++ b/experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/stop_sglang_docker.sh
@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Stop the SGLang TPxDP server through the shared deployment layer.
+# Usage: stop_sglang_docker.sh <TP> <DP>
+set -Eeuo pipefail
+
+TP="${1:-}"
+DP="${2:-}"
+if [[ -z "$TP" || -z "$DP" ]]; then
+  echo "Usage: $0 <TP> <DP>"
+  exit 1
+fi
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/config.env"
+# shellcheck source=/dev/null
+source "${SCRIPT_DIR}/../../../scripts/common/deploy_cli.sh"
+
+log "stopping SGLang server tp=${TP} dp=${DP} via deploy profile"
+deploy_stop \
+  "${DEPLOY_PROFILE:-pro6000/kimi3_pro6000_sglang_tp32ep32}" \
+  "$TP" "$DP" \
+  "${SGLANG_PORT:-30000}" \
+  "$MODEL_PATH" \
+  "${EXPERIMENT}_sglang_tp${TP}_dp${DP}"
diff --git a/platforms/patches/pro6000/kimi_k3/patch_k3_sm120.py b/platforms/patches/pro6000/kimi_k3/patch_k3_sm120.py
new file mode 100644
index 0000000..ce861a2
--- /dev/null
+++ b/platforms/patches/pro6000/kimi_k3/patch_k3_sm120.py
@@ -0,0 +1,8 @@
+"""Patch sglang kimi-k3 image: fall back from tcgen05 fused-TMA attn_res on sm_120."""
+p = "/sgl-workspace/sglang/python/sglang/srt/layers/attn_residual.py"
+s = open(p).read()
+old = "        major, _ = torch.cuda.get_device_capability()\n        _FAST_SUPPORTED = major >= 10"
+new = "        major, _ = torch.cuda.get_device_capability()\n        # RTX 6000D is sm_120 (major 12): tcgen05 exists only on GB100/GB200/GB300\n        # (majors 10/11), so sm_120 must take the Triton score/combine pipeline.\n        _FAST_SUPPORTED = major in (10, 11)"
+assert old in s, "pattern not found in attn_residual.py"
+open(p, "w").write(s.replace(old, new))
+print("OK patched attn_residual._use_fast -> majors (10,11)")