qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: derive hybrid PD transfer bounds from full-attention layers

Browse Source
This commit is contained in:
Zhiyi Hong 2026-08-31 17:53:30 +08:00
parent 91ab3fe32d
commit 0e33209ae5
40 changed files with 4967 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -1,5 +1,7 @@
# sskj — 多平台大模型推理性能基准测试项目 # sskj — 多平台大模型推理性能基准测试项目
**更新（2026-08-31 17:50:51 CST）**：修复 Kimi 混合注意力缓存池在 PD 初始化时缺少 `end_layer` 的兼容问题，按完整注意力层偏移和本地层数确定传输范围，保留普通池与 layer-shard 路径。新增回归后 24 项 CPU 测试通过，八节点修复镜像摘要一致；P 组已完成预热和 Mooncake 注册，健康检查返回 200，D 组正在验证启动。当前 Run 为 `pd-dflash-kvbounds1-20260831-1740`，尚无 GSM8K 验收结果。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
**更新（2026-08-31 17:05:00 CST）**：新增 Kimi-K3 PP8 + DFlash PD 适配验证入口。基于 SGLang PR #33863 固定源码，接通 PP 分段 hidden 投影、P 侧 prompt draft KV 生成、D 侧输入生命周期与 TP4→TP32 的 draft GQA KV 传输，保留 Kimi SM120 FlashInfer MXFP4 接入。修复 P 普通预热误带 DFlash verify metadata 的启动问题，23 项 CPU 回归通过；修复镜像已同步 601–608，服务级验证正在进行，尚无 GSM8K 结果。配置为 P TP4/PP8/EP4、D TP32/PP1/EP4、BF16 KV、8K Chunk，计划固定 64 题 C1/C8。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。 **更新（2026-08-31 17:05:00 CST）**：新增 Kimi-K3 PP8 + DFlash PD 适配验证入口。基于 SGLang PR #33863 固定源码，接通 PP 分段 hidden 投影、P 侧 prompt draft KV 生成、D 侧输入生命周期与 TP4→TP32 的 draft GQA KV 传输，保留 Kimi SM120 FlashInfer MXFP4 接入。修复 P 普通预热误带 DFlash verify metadata 的启动问题，23 项 CPU 回归通过；修复镜像已同步 601–608，服务级验证正在进行，尚无 GSM8K 结果。配置为 P TP4/PP8/EP4、D TP32/PP1/EP4、BF16 KV、8K Chunk，计划固定 64 题 C1/C8。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
**更新（2026-08-27 13:53:26 CST）**：完成 Kimi-K3 八节点标准 PD 第一阶段。P 组 601-604 使用 PP8×TP4×EP4、FlashInfer MXFP4、Chunk 8K，D 组 605-608 使用 PP1×TP32×EP32、Marlin，通过 Mooncake 0.3.12.post1 和 4 Rail RDMA 传输；统一 P/D `page_size=64` 后，16K→1 与 16K→512 的 C1/C8 共 91/91 请求成功。代表结果：16K→1 C8 Input TPS 6364.31、TTFT P50/P95 20.555/21.345 秒；16K→512 C8 TPOT P50/P95 63.20/66.55 ms。详见 `experiments/pro6000/kimi3_pro6000_pd_pp8_standard/README.md`。 **更新（2026-08-27 13:53:26 CST）**：完成 Kimi-K3 八节点标准 PD 第一阶段。P 组 601-604 使用 PP8×TP4×EP4、FlashInfer MXFP4、Chunk 8K，D 组 605-608 使用 PP1×TP32×EP32、Marlin，通过 Mooncake 0.3.12.post1 和 4 Rail RDMA 传输；统一 P/D `page_size=64` 后，16K→1 与 16K→512 的 C1/C8 共 91/91 请求成功。代表结果：16K→1 C8 Input TPS 6364.31、TTFT P50/P95 20.555/21.345 秒；16K→512 C8 TPOT P50/P95 63.20/66.55 ms。详见 `experiments/pro6000/kimi3_pro6000_pd_pp8_standard/README.md`。

2
docs/Kimi-K3_PP与DFlash迁移审计.md
View File

@ -6,7 +6,7 @@
**优先推进 DFlash 的 PP + PD 适配。** #33863 的分段投影设计能复用于 DFlash，但需要补齐 DFlash worker、调度入口和异构 TP 下的 draft KV 传输。 **优先推进 DFlash 的 PP + PD 适配。** #33863 的分段投影设计能复用于 DFlash，但需要补齐 DFlash worker、调度入口和异构 TP 下的 draft KV 传输。
当前已实现 DFlash PP worker、Kimi 跨 stage capture、PD 输入衔接、异构 TP 的 draft KV 传输，以及 Kimi FlashInfer 布局/SiTU 接入。首轮 P 启动暴露普通预热误带 verify metadata 的问题，已修复并增加回归，23 项 CPU 测试通过。SM120 编译及 Kimi SiTU GPU 集成回归已通过。修复镜像已同步全部八节点，Run `pd-dflash-pwarm1-20260831-1652` 正在重新验证服务启动；尚无 PD 请求和 GSM8K 结果。详见 [实现进度与命令](evidence/kimi_k3_pd_dflash/README.md)。 当前已实现 DFlash PP worker、Kimi 跨 stage capture、PD 输入衔接、异构 TP 的 draft KV 传输，以及 Kimi FlashInfer 布局/SiTU 接入。P 端两次启动分别暴露普通预热误带 verify metadata、混合池缺少 `end_layer` 的问题，均已针对实际调用点修复，24 项 CPU 测试通过。SM120 编译及 Kimi SiTU GPU 集成回归已通过。Run `pd-dflash-kvbounds1-20260831-1740` 的 P 组健康检查已返回 200，D 组正在启动；尚无跨组请求和 GSM8K 结果。详见 [实现进度与命令](evidence/kimi_k3_pd_dflash/README.md)。
暂停 EAGLE3 baseline。KV cache 保持 BF16，chunk 保持 8192。FlashInfer 使用官方 0.6.18 加已合并 #4460 的显式 backport；原版 0.6.18 wheel 尚未包含所需 CUTLASS SiTU 接口。 暂停 EAGLE3 baseline。KV cache 保持 BF16，chunk 保持 8192。FlashInfer 使用官方 0.6.18 加已合并 #4460 的显式 backport；原版 0.6.18 wheel 尚未包含所需 CUTLASS SiTU 接口。

14
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md
View File

@ -1,6 +1,6 @@
# Kimi-K3 PP + DFlash 适配进度 # Kimi-K3 PP + DFlash 适配进度
2026-08-31：PP worker、PD 输入衔接、异构 TP 传输和 Kimi FlashInfer 接入已完成首版；SM120 编译及 Kimi SiTU GPU 集成回归通过。首轮八节点部署在 P 侧预热阶段失败，已修复普通预热误带 DFlash verify metadata 的问题；本地、601 构建镜像及 602 增量导入容器的 23 项 CPU 回归通过。正在重新验证服务启动，尚无 PD 请求或 GSM8K 结果。 2026-08-31：PP worker、PD 输入衔接、异构 TP 传输和 Kimi FlashInfer 接入已完成首版；SM120 编译及 Kimi SiTU GPU 集成回归通过。两项启动兼容修复及 24 项 CPU 回归已完成。Run `pd-dflash-kvbounds1-20260831-1740` 的 P 组已通过预热、Mooncake 初始化和健康检查（HTTP 200），D 组正在启动；尚无跨组请求或 GSM8K 结果。
## 源码和环境 ## 源码和环境
@ -46,16 +46,16 @@ python3 -m unittest discover -s test/registered/disaggregation -v
| 回归 | 数量 | 主要覆盖 | | 回归 | 数量 | 主要覆盖 |
|---|---:|---| |---|---:|---|
| 传输 | 7 | 144 组 KV-head/TP 组合、实际 P-TP4→D-TP32 全 32 rank、逐字节复制、越界/重复写拒绝、注册往返 | | 传输 | 7 | 144 组 KV-head/TP 组合、实际 P-TP4→D-TP32 全 32 rank、逐字节复制、越界/重复写拒绝、注册往返 |
| PP 与 PD 输入 | 12 | PP1/2/4/8/16 capture 归属、边界 residual、空 capture stage、BF16/FP32 分段投影、单次 norm、末 stage 写 KV、输入生命周期、P 普通预热不创建 verify metadata | | PP 与 PD 输入 | 13 | PP1/2/4/8/16 capture 归属、边界 residual、空 capture stage、BF16/FP32 分段投影、单次 norm、末 stage 写 KV、输入生命周期、P 普通预热不创建 verify metadata、混合池/普通池传输层范围 |
| Kimi MoE 合并 | 4 | 两类 gate/up 布局、SiTU 参数、activation 白名单、非连续输入、保留 SwigluStep、API 能力检查 | | Kimi MoE 合并 | 4 | 两类 gate/up 布局、SiTU 参数、activation 白名单、非连续输入、保留 SwigluStep、API 能力检查 |
23 项在本地、601 修复镜像和 602 导入容器中通过，并完成真实 DFlashWorkerV2、MooncakeKVManager 导入。8 条启动命令的 CLI 解析及两类服务的参数后处理已检查。CPU 测试对实际方法作 AST 提取，使用 CPU tensor 或记录型 engine，核验数学与调用契约，不替代服务级验证。 24 项在本地与 601 `kvbounds1` 镜像中通过。此前 `pwarm1` 镜像的 23 项也在 602 导入容器中通过，并完成真实 DFlashWorkerV2、MooncakeKVManager 导入。8 条启动命令的 CLI 解析及两类服务的参数后处理已检查。CPU 测试对实际方法作 AST 提取，使用 CPU tensor 或记录型 engine，核验数学与调用契约，不替代服务级验证。
601 GPU6 的 `test_kimi_k3_sm120_situ_layout_and_noncontiguous_input` 已通过：检查 Kimi gate/up 及 scale 布局、SiTU(4,25)、非连续输入，以及 SGLang adapter 与直接 FlashInfer 调用的输出一致性。这是小 shape 的集成回归；端到端正确性由后续 PD/GSM8K 检验。 601 GPU6 的 `test_kimi_k3_sm120_situ_layout_and_noncontiguous_input` 已通过：检查 Kimi gate/up 及 scale 布局、SiTU(4,25)、非连续输入，以及 SGLang adapter 与直接 FlashInfer 调用的输出一致性。这是小 shape 的集成回归；端到端正确性由后续 PD/GSM8K 检验。
修复前镜像 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460` 的独立 GPU 复测通过，Mooncake CUDA engine 导入成功，GPU 测试 1 passed、25.28 秒（复用 JIT 缓存）。 修复前镜像 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460` 的独立 GPU 复测通过，Mooncake CUDA engine 导入成功，GPU 测试 1 passed、25.28 秒（复用 JIT 缓存）。
当前镜像为 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1`，只追加 P 预热修复，不改变 GPU kernel。Linux/amd64 manifest 为 `sha256:281ccb2666a38acd539e6fbb5d55d3682eb2af9ac9089f67e9bf92a8ddd822eb`，image config 为 `sha256:8ab5eee9902556bcba2a2b439a9fa1b14d93e0554503754fb7ea74dad6c3cf79`；OCI index 为 `sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78`。区分这三个摘要，不将 Docker 不同模式显示的 ID 当成代码不一致。 当前镜像为 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1`，在 `pwarm1` 上只替换 Python 层范围处理及回归测试，不改变 FlashInfer 层或 GPU kernel。Linux/amd64 manifest 为 `sha256:e74c9b1e7fb6a9509db32576d2c9196b42ef329118909eeebd620cb9db2f91e9`，image config 为 `sha256:26584138eb1e097183ce167cbffe4198be0611e0cfde7422cbe692100d865982`；OCI index 为 `sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f`。区分这三个摘要，不将 Docker 不同模式显示的 ID 当成代码不一致。
601 证据均位于实验目录 `results/`： 601 证据均位于实验目录 `results/`：
@ -73,6 +73,8 @@ python3 -m unittest discover -s test/registered/disaggregation -v
- `pd-dflash-20260831-1617/logs/p_0.log` 至 `p_3.log`：首轮 P 预热报错原始证据；该 Run 未进入请求测试，退出码 1。 - `pd-dflash-20260831-1617/logs/p_0.log` 至 `p_3.log`：首轮 P 预热报错原始证据；该 Run 未进入请求测试，退出码 1。
- `image_build_pwarm1_20260831.log`：预热修复镜像构建及 23 项 CPU 回归。 - `image_build_pwarm1_20260831.log`：预热修复镜像构建及 23 项 CPU 回归。
- `pwarm1_602_cpu.log`、`pwarm1_602_inspect.json`：约 49 MiB 增量包导入后的容器回归与平台摘要。 - `pwarm1_602_cpu.log`、`pwarm1_602_inspect.json`：约 49 MiB 增量包导入后的容器回归与平台摘要。
- `pd-dflash-pwarm1-20260831-1652/logs/p_0.log`：P 预热完成后，在初始化传输管理器时发现 `HybridLinearKVPool.end_layer` 缺失；未进入 D 启动和请求测试。
- `image_build_kvbounds1_20260831.log`、`kvbounds_image_context/`：复用原 FlashInfer 层、只修改 Python 的构建配方及 24 项回归。
已通过 compileall、Black 和 Ruff 的未定义变量/语法检查。 已通过 compileall、Black 和 Ruff 的未定义变量/语法检查。
@ -86,7 +88,7 @@ cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation
DRY_RUN=1 RUN_ID=command-audit-20260831 bash deploy_pd_dflash.sh all DRY_RUN=1 RUN_ID=command-audit-20260831 bash deploy_pd_dflash.sh all
``` ```
首轮已执行 `all`，在 P 侧启动阶段退出。修复镜像同步完成后使用新 Run ID 重试。正式顺序为 `start` → `smoke` → `bench` → `logs` → `stop`，也可用 `all` 串行执行。所有操作使用同一个 `RUN_ID`。失败时保留本任务容器和日志，入口不会自动杀其他实验。 此前两轮均在 P 侧启动阶段退出，日志已保存、失败容器已清理。修复镜像同步完成后使用新 Run ID 重试。正式顺序为 `start` → `smoke` → `bench` → `logs` → `stop`，也可用 `all` 串行执行。所有操作使用同一个 `RUN_ID`。失败时保留本任务容器和日志，入口不会自动杀其他实验。
配置选择：P 为 TP4/PP8/EP4，D 为 TP32/PP1/EP4，两侧 FlashInfer；KV 为 BF16，chunk=8192，page=64。为 C1/C8 评测将活跃请求与 Decode Graph 上限设为 8；未额外改变模型 context 上限。 配置选择：P 为 TP4/PP8/EP4，D 为 TP32/PP1/EP4，两侧 FlashInfer；KV 为 BF16，chunk=8192，page=64。为 C1/C8 评测将活跃请求与 Decode Graph 上限设为 8；未额外改变模型 context 上限。
@ -102,4 +104,6 @@ GSM8K 为原数据集前 64 题，沿用历史客户端的 5-shot、temperature=
`base_runner.py::_dummy_run` 已将 PD Prefill target 设为普通 Decode 预热，但随后仍创建 `DFlashVerifyInput`。普通 Triton Attention 读取 `kv_indptr` 时因此报 `AttributeError`。修复让该分支的 `spec_info=None`，D 侧真正的 TARGET_VERIFY 路径不变。没有强行添加字段，也没有修改 DFlash 接受算法。 `base_runner.py::_dummy_run` 已将 PD Prefill target 设为普通 Decode 预热，但随后仍创建 `DFlashVerifyInput`。普通 Triton Attention 读取 `kv_indptr` 时因此报 `AttributeError`。修复让该分支的 `spec_info=None`，D 侧真正的 TARGET_VERIFY 路径不变。没有强行添加字段，也没有修改 DFlash 接受算法。
第二轮通过预热后，`PrefillBootstrapQueue._init_kv_manager` 直接访问混合池不存在的 `end_layer`。修复以现有的完整注意力层偏移加 `full_layer_nums` 得到传输范围，KDA 状态和 draft KV 仍分别使用明确的全局层 ID。新增回归覆盖 MLA/MHA 混合池、空完整注意力 stage、普通池和 layer-shard 分支，不将 K/V 两组条目误计为两倍层数。
镜像同步须检查 Docker 所在根盘，而不只检查模型盘 `/data`。本镜像层展开约 34.4GB，压缩内容约 15GB；共享层会减少增量占用。607 已按用户授权删除两个无容器引用的 vLLM 镜像，根盘恢复到约 107GB，原镜像元数据保存在 `results/607_vllm_images_before_delete_20260831.json`。用户清理 606 后其根盘恢复到约 91GB。601 使用 `ctr images export` 将既有 OCI 压缩层直接导出到 `/data`，其旧 vLLM 镜像尚未删除。 镜像同步须检查 Docker 所在根盘，而不只检查模型盘 `/data`。本镜像层展开约 34.4GB，压缩内容约 15GB；共享层会减少增量占用。607 已按用户授权删除两个无容器引用的 vLLM 镜像，根盘恢复到约 107GB，原镜像元数据保存在 `results/607_vllm_images_before_delete_20260831.json`。用户清理 606 后其根盘恢复到约 91GB。601 使用 `ctr images export` 将既有 OCI 压缩层直接导出到 `/data`，其旧 vLLM 镜像尚未删除。

2
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/config.env
View File

@ -1,5 +1,5 @@
EXPERIMENT=kimi3_pro6000_pd_dflash_validation EXPERIMENT=kimi3_pro6000_pd_dflash_validation
PD_IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1 PD_IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
MODEL_PATH=/data/hf_models/Kimi-K3 MODEL_PATH=/data/hf_models/Kimi-K3
DRAFT_MODEL_PATH=/data/hf_models/Kimi-K3-DFlash DRAFT_MODEL_PATH=/data/hf_models/Kimi-K3-DFlash
P_NODES=(174.1.60.1 174.1.60.2 174.1.60.3 174.1.60.4) P_NODES=(174.1.60.1 174.1.60.2 174.1.60.3 174.1.60.4)

4
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/pp_dflash_files.json
View File

@ -27,7 +27,7 @@
}, },
{ {
"path": "python/sglang/srt/disaggregation/prefill.py", "path": "python/sglang/srt/disaggregation/prefill.py",
"sha256": "d0175b3df8eddd58fbfa41805d0928d614e412cf74055e57aade82578b174bfd" "sha256": "d2e70dd6d8606e218da15a987a5e3a2e82f5ef77ab43f63f1feaf90ad5b8d286"
}, },
{ {
"path": "python/sglang/srt/disaggregation/utils.py", "path": "python/sglang/srt/disaggregation/utils.py",
@ -67,7 +67,7 @@
}, },
{ {
"path": "test/registered/disaggregation/test_dflash_pp_context.py", "path": "test/registered/disaggregation/test_dflash_pp_context.py",
"sha256": "3f3022afa4acd7f66e14caf6bbbfb88b1f2e9320d087300aa78c9d9e20d3a321" "sha256": "955e45f288b060371a26f7f5a1e7490e3bacc1c8941ab7f2dbca2e3bc98b4bc9"
}, },
{ {
"path": "test/registered/disaggregation/test_flashinfer_kimi_merge.py", "path": "test/registered/disaggregation/test_flashinfer_kimi_merge.py",

98
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/pp_dflash_integration.patch
View File

@ -424,7 +424,33 @@ diff --git a/python/sglang/srt/disaggregation/prefill.py b/python/sglang/srt/dis
build_kv_layer_ids, build_kv_layer_ids,
build_staging_slot_metadata, build_staging_slot_metadata,
get_dsv4_c128_state_indices, get_dsv4_c128_state_indices,
@@ -252,6 +253,18 @@ @@ -112,6 +113,13 @@
)
+def _transfer_end_layer(*, pool, start_layer: int) -> int:
+ if isinstance(pool, HybridLinearKVPool):
+ # Match the dense full-attention offset used by _transfer_start_layer.
+ return start_layer + pool.full_layer_nums
+ return pool.end_layer
+
+
def maybe_release_metadata_buffer(
req: Req, allocator: ReqToMetadataIdxAllocator
) -> None:
@@ -233,7 +241,10 @@
kv_args.prefill_end_layer = (
kv_args.prefill_start_layer + len(kv_data_ptrs)
if layer_shard_enabled
- else self.token_to_kv_pool.end_layer
+ else _transfer_end_layer(
+ pool=self.token_to_kv_pool,
+ start_layer=kv_args.prefill_start_layer,
+ )
)
draft_kv_pool = self.draft_token_to_kv_pool if transfer_draft_cache else None
@@ -252,6 +263,18 @@
kv_args.kv_data_ptrs = kv_data_ptrs kv_args.kv_data_ptrs = kv_data_ptrs
kv_args.kv_data_lens = kv_data_lens kv_args.kv_data_lens = kv_data_lens
kv_args.kv_item_lens = kv_item_lens kv_args.kv_item_lens = kv_item_lens
@ -1203,7 +1229,7 @@ diff --git a/test/registered/disaggregation/test_dflash_pp_context.py b/test/reg
new file mode 100644 new file mode 100644
--- /dev/null --- /dev/null
+++ b/test/registered/disaggregation/test_dflash_pp_context.py +++ b/test/registered/disaggregation/test_dflash_pp_context.py
@@ -0,0 +1,431 @@ @@ -0,0 +1,499 @@
+"""CPU contract tests of the production PP methods; no model/RDMA emulation claim.""" +"""CPU contract tests of the production PP methods; no model/RDMA emulation claim."""
+ +
+import ast +import ast
@ -1301,6 +1327,74 @@ new file mode 100644
+ +
+ +
+class TestPPContext(unittest.TestCase): +class TestPPContext(unittest.TestCase):
+ def test_prefill_transfer_bounds_for_hybrid_and_ordinary_pools(self):
+ class HybridPool:
+ def __init__(self, start, count):
+ self.start_layer = start
+ self.full_layer_nums = count
+
+ path = "disaggregation/prefill.py"
+ globals_dict = {"HybridLinearKVPool": HybridPool}
+ start_fn = load_function(path, "_transfer_start_layer", globals_dict)
+ end_fn = load_function(path, "_transfer_end_layer", globals_dict)
+ tree = ast.parse((SRT / path).read_text())
+ cls = next(
+ n
+ for n in tree.body
+ if isinstance(n, ast.ClassDef) and n.name == "PrefillBootstrapQueue"
+ )
+ method = next(
+ n
+ for n in cls.body
+ if isinstance(n, ast.FunctionDef) and n.name == "_init_kv_manager"
+ )
+ assignment = next(
+ n
+ for n in method.body
+ if isinstance(n, ast.Assign)
+ and any(
+ isinstance(t, ast.Attribute) and t.attr == "prefill_end_layer"
+ for t in n.targets
+ )
+ )
+ code = compile(
+ ast.fix_missing_locations(ast.Module(body=[assignment], type_ignores=[])),
+ str(SRT / path),
+ "exec",
+ )
+ config = SimpleNamespace(full_attention_layer_ids=[2, 6, 10, 14, 18])
+ cases = [
+ (HybridPool(0, 2), False, 2, 0, 2),
+ (HybridPool(8, 2), False, 2, 2, 4),
+ (
+ HybridPool(8, 2),
+ False,
+ 4,
+ 2,
+ 4,
+ ), # Separate K/V entries do not double layer count.
+ (HybridPool(20, 0), False, 0, 5, 5),
+ (SimpleNamespace(start_layer=8, end_layer=16), False, 3, 8, 16),
+ (SimpleNamespace(start_layer=8, end_layer=16), True, 3, 8, 11),
+ ]
+ for pool, sharded, entries, start, end in cases:
+ with self.subTest(
+ pool=type(pool).__name__, sharded=sharded, entries=entries
+ ):
+ self.assertEqual(start_fn(pool=pool, hf_text_config=config), start)
+ args = SimpleNamespace(prefill_start_layer=start)
+ exec(
+ code,
+ {
+ "kv_args": args,
+ "kv_data_ptrs": [0] * entries,
+ "layer_shard_enabled": sharded,
+ "self": SimpleNamespace(token_to_kv_pool=pool),
+ "_transfer_end_layer": end_fn,
+ },
+ )
+ self.assertEqual(args.prefill_end_layer, end)
+
+ def test_prefill_dummy_forward_has_no_verify_metadata(self): + def test_prefill_dummy_forward_has_no_verify_metadata(self):
+ path = SRT / "model_executor/runner/base_runner.py" + path = SRT / "model_executor/runner/base_runner.py"
+ tree = ast.parse(path.read_text()) + tree = ast.parse(path.read_text())

76
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/image_build_kvbounds1_20260831.log Normal file
View File

@ -0,0 +1,76 @@
#0 building with "default" instance using docker driver
#1 [internal] load build definition from kimi-dflash-kvbounds.Dockerfile
#1 transferring dockerfile: 458B done
#1 DONE 0.0s
#2 [internal] load metadata for docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1
#2 DONE 1.1s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [1/5] FROM docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1@sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
#4 resolve docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1@sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78 0.1s done
#4 ...
#5 [internal] load build context
#5 transferring context: 82.52kB done
#5 DONE 0.1s
#4 [1/5] FROM docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1@sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
#4 DONE 5.3s
#6 [2/5] COPY prefill.py /opt/kimi-dflash/python/sglang/srt/disaggregation/prefill.py
#6 DONE 0.0s
#7 [3/5] COPY test_dflash_pp_context.py /opt/kimi-dflash/test/registered/disaggregation/test_dflash_pp_context.py
#7 DONE 0.1s
#8 [4/5] COPY SOURCE_MANIFEST.json /opt/kimi-dflash/SOURCE_MANIFEST.json
#8 DONE 0.0s
#9 [5/5] RUN python3 -m unittest discover -s /opt/kimi-dflash/test/registered/disaggregation -v
#9 5.047 test_capture_ownership_including_boundaries (test_dflash_pp_context.TestPPContext.test_capture_ownership_including_boundaries) ... ok
#9 5.050 test_invalid_capture_configuration (test_dflash_pp_context.TestPPContext.test_invalid_capture_configuration) ... ok
#9 5.050 test_kimi_boundary_uses_next_stage_weights (test_dflash_pp_context.TestPPContext.test_kimi_boundary_uses_next_stage_weights) ... ok
#9 8.108 test_last_stage_without_capture_uses_incoming_context (test_dflash_pp_context.TestPPContext.test_last_stage_without_capture_uses_incoming_context) ... ok
#9 8.116 test_missing_incoming_context_fails_before_kv_write (test_dflash_pp_context.TestPPContext.test_missing_incoming_context_fails_before_kv_write) ... ok
#9 8.117 test_nonfinal_pool_is_minimal_without_mutating_target_config (test_dflash_pp_context.TestPPContext.test_nonfinal_pool_is_minimal_without_mutating_target_config) ... ok
#9 8.144 test_only_prefill_skips_draft_graph_initialization (test_dflash_pp_context.TestPPContext.test_only_prefill_skips_draft_graph_initialization) ... ok
#9 8.165 test_pd_input_builder_and_spec_dispatch (test_dflash_pp_context.TestPPContext.test_pd_input_builder_and_spec_dispatch) ... ok
#9 8.171 test_pp8_boundary_capture (test_dflash_pp_context.TestPPContext.test_pp8_boundary_capture) ... ok
#9 8.173 test_pp8_projection_empty_stages_float32_and_bf16 (test_dflash_pp_context.TestPPContext.test_pp8_projection_empty_stages_float32_and_bf16) ... ok
#9 8.181 test_pp_proxy_is_consumed_once_even_on_forward_failure (test_dflash_pp_context.TestPPContext.test_pp_proxy_is_consumed_once_even_on_forward_failure) ... ok
#9 8.207 test_prefill_dummy_forward_has_no_verify_metadata (test_dflash_pp_context.TestPPContext.test_prefill_dummy_forward_has_no_verify_metadata) ... ok
#9 8.213 test_prefill_transfer_bounds_for_hybrid_and_ordinary_pools (test_dflash_pp_context.TestPPContext.test_prefill_transfer_bounds_for_hybrid_and_ordinary_pools) ... ok
#9 8.248 test_activation_selection_accepts_situ_and_legacy_enum (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_activation_selection_accepts_situ_and_legacy_enum) ... ok
#9 8.260 test_capability_requires_public_parameters (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_capability_requires_public_parameters) ... ok
#9 8.270 test_runner_preserves_swiglu_step_and_old_api (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_runner_preserves_swiglu_step_and_old_api) ... ok
#9 8.275 test_weight_layout_and_situ_parameters (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_weight_layout_and_situ_parameters) ... ok
#9 8.297 test_all_gqa_shards_and_replicas_byte_exact (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_all_gqa_shards_and_replicas_byte_exact) ... ok
#9 8.375 test_flat_stride_mismatch_rejected (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_flat_stride_mismatch_rejected) ... ok
#9 8.375 test_invalid_metadata_rejected_before_copy (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_invalid_metadata_rejected_before_copy) ... ok
#9 8.375 test_mooncake_mixed_entries_match_layers_before_transfer (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_mooncake_mixed_entries_match_layers_before_transfer) ... ok
#9 8.410 test_real_kimi_dflash_page64_tp4_to_tp32 (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_real_kimi_dflash_page64_tp4_to_tp32) ... ok
#9 8.411 test_replicated_mla_is_flat_and_coalesced (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_replicated_mla_is_flat_and_coalesced) ... ok
#9 8.411 test_wire_roundtrip (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_wire_roundtrip) ... ok
#9 8.411
#9 8.411 ----------------------------------------------------------------------
#9 8.411 Ran 24 tests in 3.365s
#9 8.411
#9 8.411 OK
#9 DONE 9.2s
#10 exporting to image
#10 exporting layers
#10 exporting layers 0.2s done
#10 exporting manifest sha256:e74c9b1e7fb6a9509db32576d2c9196b42ef329118909eeebd620cb9db2f91e9 0.0s done
#10 exporting config sha256:26584138eb1e097183ce167cbffe4198be0611e0cfde7422cbe692100d865982 done
#10 exporting attestation manifest sha256:6d9e087e2e30bfb53545cdfe1a315bceda337ac57ef4cd07f0b209546a7e3caf done
#10 exporting manifest list sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f done
#10 naming to docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 done
#10 unpacking to docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
#10 unpacking to docker.io/local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 0.2s done
#10 DONE 0.7s

8
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/kvbounds1_delta_metadata.json Normal file
View File

@ -0,0 +1,8 @@
{
"archive": "sglang-kimi-pp-dflash-kvbounds1.delta.oci.tar",
"bytes": 133120,
"sha256": "800da9d85d186c21f41d1fe8681529ad3e658047786ae7e1bad92ec594939050",
"required_base_index": "d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78",
"image_index": "7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f",
"linux_amd64_manifest": "e74c9b1e7fb6a9509db32576d2c9196b42ef329118909eeebd620cb9db2f91e9"
}

10
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/kvbounds_image_context/SOURCE_MANIFEST.json Normal file
View File

@ -0,0 +1,10 @@
{
"repository": "https://github.com/sgl-project/sglang",
"base_commit": "6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5",
"base_pull_request": 33863,
"snapshot_scope": "python directory; all 4619 upstream blobs verified against git tree before edits",
"local_changes": "Mixed MLA/GQA transport; Kimi PP boundary capture; DFlash PP worker and PD input wiring; public FlashInfer SiTU adapter rebased from b95b534; PD-prefill warmup without verify metadata; dense hybrid prefill transfer bounds; 24 CPU regressions",
"status": "Experimental source snapshot; see external runtime logs for validation results",
"flashinfer_target": "official flashinfer-python 0.6.18 plus unchanged merged upstream SiTU commit b460bc00cb373541102d2155aec35bd626e522ce; old companion packages removed; standard JIT; version checks enabled",
"gpu_tested": false
}

5
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/kvbounds_image_context/kimi-dflash-kvbounds.Dockerfile Normal file
View File

@ -0,0 +1,5 @@
FROM local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1
COPY prefill.py /opt/kimi-dflash/python/sglang/srt/disaggregation/prefill.py
COPY test_dflash_pp_context.py /opt/kimi-dflash/test/registered/disaggregation/test_dflash_pp_context.py
COPY SOURCE_MANIFEST.json /opt/kimi-dflash/SOURCE_MANIFEST.json
RUN python3 -m unittest discover -s /opt/kimi-dflash/test/registered/disaggregation -v

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/commands/p_0.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/commands/p_1.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/commands/p_2.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/commands/p_3.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/exit_code Normal file
View File

@ -0,0 +1 @@
1

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/d_0.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_0

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/d_1.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_1

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/d_2.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_2

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/d_3.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_3

606
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/p_0.log Normal file
View File

File diff suppressed because one or more lines are too long

628
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/p_1.log Normal file
View File

File diff suppressed because one or more lines are too long

553
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/p_2.log Normal file
View File

File diff suppressed because one or more lines are too long

550
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/p_3.log Normal file
View File

File diff suppressed because one or more lines are too long

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/logs/router.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_router_0

8
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-20260831-1617/metadata/preflight.txt Normal file
View File

@ -0,0 +1,8 @@
174.1.60.1 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.2 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.3 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.4 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.5 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.6 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.7 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987
174.1.60.8 sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/commands/p_0.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/commands/p_1.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/commands/p_2.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/commands/p_3.cmd.txt Normal file
View File

@ -0,0 +1 @@
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-pwarm1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/exit_code Normal file
View File

@ -0,0 +1 @@
1

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/d_0.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_0

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/d_1.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_1

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/d_2.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_2

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/d_3.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_d_3

582
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/p_0.log Normal file
View File

File diff suppressed because one or more lines are too long

570
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/p_1.log Normal file
View File

File diff suppressed because one or more lines are too long

638
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/p_2.log Normal file
View File

File diff suppressed because one or more lines are too long

594
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/p_3.log Normal file
View File

File diff suppressed because one or more lines are too long

1
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/logs/router.log Normal file
View File

@ -0,0 +1 @@
Error response from daemon: No such container: kimi3_pro6000_pd_dflash_validation_router_0

8
experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-pwarm1-20260831-1652/metadata/preflight.txt Normal file
View File

@ -0,0 +1,8 @@
174.1.60.1 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.2 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.3 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.4 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.5 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.6 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.7 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78
174.1.60.8 sha256:d30d68757014b973c2edd5d061cc3fd55660a4c68cc3d0af9050fcb7e64c4a78