[Docs] Scope SGLang draft to compatibility
This commit is contained in:
Zhiyi Hong
parent
39f692caae
commit
d28db48e4b
@ -21,7 +21,7 @@
|
||||
| 最终兼容回归 | 已通过 | Draft HEAD `300c87a`：`8 passed, 5 deselected, 17 warnings in 22.23s` |
|
||||
| 格式和 CI 注册 | 已通过 | 五个变更文件完整 pre-commit 全部通过；两个测试文件注册到 CUDA CI `base-b/1-gpu-small` |
|
||||
| 保留 EP4 服务级结果 | 已证明 | 601 原始目录有 24 个正式 JSONL 点；每组 3 repeats、每点 40/40 成功；错误扫描为空；未重跑 EP32 |
|
||||
| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板；标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`；文档和最终 release 组合性能两项按事实保持未勾选 |
|
||||
| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板；标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`；公开 PR 不宣称提速，历史 EP4 性能结果仅保留在内部实验档案 |
|
||||
| GitHub fork 交付 | 已完成 | `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`，Draft HEAD `300c87a` |
|
||||
| 内部仓库独立分支 | 本地和 601 已完成，远端待推送 | 601 分支 `hzy/feature-kimi-k3-sm120-flashinfer-mxfp4` 已到审计提交；`git.meta-stone.net` 尚未为本机或 601 key 授权，HTTPS 也无 Token |
|
||||
|
||||
|
||||
@ -80,41 +80,24 @@ pytest -q -s \
|
||||
|
||||
## Speed Tests and Profiling
|
||||
|
||||
Four-node serving validation used 32 RTX 6000D GPUs with TP32/EP4/DP1,
|
||||
16K input tokens, one output token, concurrency 8/16, and chunked Prefill
|
||||
sizes 8K/16K. Each point was repeated three times; all 24 runs completed with
|
||||
40/40 successful requests and no OOM, Traceback, NCCL error, or
|
||||
`EngineDeadError`.
|
||||
|
||||
| Chunk | C | Backend | Input TPS | TTFT P50 (s) | FlashInfer vs Marlin |
|
||||
|---:|---:|---|---:|---:|---:|
|
||||
| 8K | 8 | Marlin | 2935.02 | 43.51 | baseline |
|
||||
| 8K | 8 | FlashInfer | 3257.96 | 39.19 | +11.00% TPS, -9.92% TTFT |
|
||||
| 8K | 16 | Marlin | 2934.90 | 87.09 | baseline |
|
||||
| 8K | 16 | FlashInfer | 3260.14 | 78.39 | +11.08% TPS, -9.99% TTFT |
|
||||
| 16K | 8 | Marlin | 2812.91 | 45.47 | baseline |
|
||||
| 16K | 8 | FlashInfer | 3027.50 | 42.27 | +7.63% TPS, -7.04% TTFT |
|
||||
| 16K | 16 | Marlin | 2812.33 | 90.97 | baseline |
|
||||
| 16K | 16 | FlashInfer | 3027.34 | 84.52 | +7.65% TPS, -7.09% TTFT |
|
||||
|
||||
The service benchmark used the previously validated candidate integration
|
||||
with the same Kimi layout, SiTU, non-contiguous-input, and SM120 guard fixes.
|
||||
The final public #4460 API is covered by the accuracy test above; no claim is
|
||||
made that an unreleased #4460 wheel was used for this four-node benchmark.
|
||||
Not applicable as a merge claim. This PR adds missing model/hardware
|
||||
compatibility and does not change the default backend or claim a speedup.
|
||||
Correctness on the target SM120 path is the acceptance criterion.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] Format your code according to the [Format code with pre-commit](https://docs.sglang.io/developer_guide/contribution_guide.html#format-code-with-pre-commit).
|
||||
- [x] Add unit tests according to the [Run and add unit tests](https://docs.sglang.io/developer_guide/contribution_guide.html#run-and-add-unit-tests).
|
||||
- [ ] Update documentation according to [Write documentations](https://docs.sglang.io/developer_guide/contribution_guide.html#write-documentations). (Pending until #4460 has a released API; this Draft does not change defaults.)
|
||||
- [ ] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is complete; the table above is supporting evidence from the equivalent candidate integration. This item remains open until the final SGLang head is benchmarked with an official #4460 RC or release.)
|
||||
- [x] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is provided above; speed is N/A because this compatibility PR makes no speedup claim and does not change the default backend.)
|
||||
- [x] Follow the SGLang code style [guidance](https://docs.sglang.io/developer_guide/contribution_guide.html#code-style-guidance).
|
||||
|
||||
Full pre-commit validation was run on all five changed files and every hook
|
||||
passed. The two test files are registered with CUDA CI in `base-b` using the
|
||||
`1-gpu-small` runner.
|
||||
|
||||
The unchecked items are intentional Draft gates, not known code failures.
|
||||
The unchecked documentation item is an intentional Draft gate, not a known
|
||||
code failure.
|
||||
|
||||
## Dependency and Merge Order
|
||||
|
||||
|
||||
@ -35,9 +35,9 @@
|
||||
`/data/hzy/phase5_sglang_pr4460_draft_cpu_final.log`。
|
||||
- 公开 Draft PR 正文遵循 SGLang 官方模板，不暴露内网原始路径；原始日志路径、
|
||||
镜像 SHA、wheel SHA 和去重证据继续保留在本仓库审计文件中。
|
||||
- Checklist 不全部预先勾选：文档等待 #4460 发布稳定 API；四机候选结果只作为
|
||||
方向性 speed evidence，最终 SGLang HEAD 与官方 #4460 RC/release 的服务级复测
|
||||
完成后再勾选 accuracy/speed 合并项。
|
||||
- Checklist 不全部预先勾选：文档等待 #4460 发布稳定 API。公开 PR 不做提速
|
||||
声明，因此 Speed Tests 标为 N/A；四机候选结果继续保留在内部实验档案，
|
||||
不作为该兼容性 PR 的合并门槛。
|
||||
|
||||
## 2026-08-18 Phase 5
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user