diff --git a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/COMPLETION_AUDIT.md b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/COMPLETION_AUDIT.md index c7b0492..c1e28f4 100644 --- a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/COMPLETION_AUDIT.md +++ b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/COMPLETION_AUDIT.md @@ -21,7 +21,7 @@ | 最终兼容回归 | 已通过 | Draft HEAD `300c87a`：`8 passed, 5 deselected, 17 warnings in 22.23s` | | 格式和 CI 注册 | 已通过 | 五个变更文件完整 pre-commit 全部通过；两个测试文件注册到 CUDA CI `base-b/1-gpu-small` | | 保留 EP4 服务级结果 | 已证明 | 601 原始目录有 24 个正式 JSONL 点；每组 3 repeats、每点 40/40 成功；错误扫描为空；未重跑 EP32 | -| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板；标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`；文档和最终 release 组合性能两项按事实保持未勾选 | +| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板；标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`；公开 PR 不宣称提速，历史 EP4 性能结果仅保留在内部实验档案 | | GitHub fork 交付 | 已完成 | `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`，Draft HEAD `300c87a` | | 内部仓库独立分支 | 本地和 601 已完成，远端待推送 | 601 分支 `hzy/feature-kimi-k3-sm120-flashinfer-mxfp4` 已到审计提交；`git.meta-stone.net` 尚未为本机或 601 key 授权，HTTPS 也无 Token | diff --git a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md index 65fce8f..7084740 100644 --- a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md +++ b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md @@ -80,41 +80,24 @@ pytest -q -s \ ## Speed Tests and Profiling -Four-node serving validation used 32 RTX 6000D GPUs with TP32/EP4/DP1, -16K input tokens, one output token, concurrency 8/16, and chunked Prefill -sizes 8K/16K. Each point was repeated three times; all 24 runs completed with -40/40 successful requests and no OOM, Traceback, NCCL error, or -`EngineDeadError`. - -| Chunk | C | Backend | Input TPS | TTFT P50 (s) | FlashInfer vs Marlin | -|---:|---:|---|---:|---:|---:| -| 8K | 8 | Marlin | 2935.02 | 43.51 | baseline | -| 8K | 8 | FlashInfer | 3257.96 | 39.19 | +11.00% TPS, -9.92% TTFT | -| 8K | 16 | Marlin | 2934.90 | 87.09 | baseline | -| 8K | 16 | FlashInfer | 3260.14 | 78.39 | +11.08% TPS, -9.99% TTFT | -| 16K | 8 | Marlin | 2812.91 | 45.47 | baseline | -| 16K | 8 | FlashInfer | 3027.50 | 42.27 | +7.63% TPS, -7.04% TTFT | -| 16K | 16 | Marlin | 2812.33 | 90.97 | baseline | -| 16K | 16 | FlashInfer | 3027.34 | 84.52 | +7.65% TPS, -7.09% TTFT | - -The service benchmark used the previously validated candidate integration -with the same Kimi layout, SiTU, non-contiguous-input, and SM120 guard fixes. -The final public #4460 API is covered by the accuracy test above; no claim is -made that an unreleased #4460 wheel was used for this four-node benchmark. +Not applicable as a merge claim. This PR adds missing model/hardware +compatibility and does not change the default backend or claim a speedup. +Correctness on the target SM120 path is the acceptance criterion. ## Checklist - [x] Format your code according to the [Format code with pre-commit](https://docs.sglang.io/developer_guide/contribution_guide.html#format-code-with-pre-commit). - [x] Add unit tests according to the [Run and add unit tests](https://docs.sglang.io/developer_guide/contribution_guide.html#run-and-add-unit-tests). - [ ] Update documentation according to [Write documentations](https://docs.sglang.io/developer_guide/contribution_guide.html#write-documentations). (Pending until #4460 has a released API; this Draft does not change defaults.) -- [ ] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is complete; the table above is supporting evidence from the equivalent candidate integration. This item remains open until the final SGLang head is benchmarked with an official #4460 RC or release.) +- [x] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is provided above; speed is N/A because this compatibility PR makes no speedup claim and does not change the default backend.) - [x] Follow the SGLang code style [guidance](https://docs.sglang.io/developer_guide/contribution_guide.html#code-style-guidance). Full pre-commit validation was run on all five changed files and every hook passed. The two test files are registered with CUDA CI in `base-b` using the `1-gpu-small` runner. -The unchecked items are intentional Draft gates, not known code failures. +The unchecked documentation item is an intentional Draft gate, not a known +code failure. ## Dependency and Merge Order diff --git a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/decision_log.md b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/decision_log.md index c25bf0f..6eb4d77 100644 --- a/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/decision_log.md +++ b/experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/decision_log.md @@ -35,9 +35,9 @@ `/data/hzy/phase5_sglang_pr4460_draft_cpu_final.log`。 - 公开 Draft PR 正文遵循 SGLang 官方模板，不暴露内网原始路径；原始日志路径、 镜像 SHA、wheel SHA 和去重证据继续保留在本仓库审计文件中。 -- Checklist 不全部预先勾选：文档等待 #4460 发布稳定 API；四机候选结果只作为 - 方向性 speed evidence，最终 SGLang HEAD 与官方 #4460 RC/release 的服务级复测 - 完成后再勾选 accuracy/speed 合并项。 +- Checklist 不全部预先勾选：文档等待 #4460 发布稳定 API。公开 PR 不做提速 + 声明，因此 Speed Tests 标为 N/A；四机候选结果继续保留在内部实验档案， + 不作为该兼容性 PR 的合并门槛。 ## 2026-08-18 Phase 5