[Docs] Scope SGLang draft to compatibility
This commit is contained in:
parent
39f692caae
commit
d28db48e4b
@ -21,7 +21,7 @@
|
|||||||
| 最终兼容回归 | 已通过 | Draft HEAD `300c87a`:`8 passed, 5 deselected, 17 warnings in 22.23s` |
|
| 最终兼容回归 | 已通过 | Draft HEAD `300c87a`:`8 passed, 5 deselected, 17 warnings in 22.23s` |
|
||||||
| 格式和 CI 注册 | 已通过 | 五个变更文件完整 pre-commit 全部通过;两个测试文件注册到 CUDA CI `base-b/1-gpu-small` |
|
| 格式和 CI 注册 | 已通过 | 五个变更文件完整 pre-commit 全部通过;两个测试文件注册到 CUDA CI `base-b/1-gpu-small` |
|
||||||
| 保留 EP4 服务级结果 | 已证明 | 601 原始目录有 24 个正式 JSONL 点;每组 3 repeats、每点 40/40 成功;错误扫描为空;未重跑 EP32 |
|
| 保留 EP4 服务级结果 | 已证明 | 601 原始目录有 24 个正式 JSONL 点;每组 3 repeats、每点 40/40 成功;错误扫描为空;未重跑 EP32 |
|
||||||
| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板;标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`;文档和最终 release 组合性能两项按事实保持未勾选 |
|
| 公开 PR 标题、正文、依赖和已知限制 | Draft 已完成 | `SGLANG_DRAFT_PR.md` 遵循 SGLang 官方模板;标题为 `[Feature] Support Kimi-K3 FlashInfer MXFP4 MoE on SM120`;公开 PR 不宣称提速,历史 EP4 性能结果仅保留在内部实验档案 |
|
||||||
| GitHub fork 交付 | 已完成 | `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`,Draft HEAD `300c87a` |
|
| GitHub fork 交付 | 已完成 | `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`,Draft HEAD `300c87a` |
|
||||||
| 内部仓库独立分支 | 本地和 601 已完成,远端待推送 | 601 分支 `hzy/feature-kimi-k3-sm120-flashinfer-mxfp4` 已到审计提交;`git.meta-stone.net` 尚未为本机或 601 key 授权,HTTPS 也无 Token |
|
| 内部仓库独立分支 | 本地和 601 已完成,远端待推送 | 601 分支 `hzy/feature-kimi-k3-sm120-flashinfer-mxfp4` 已到审计提交;`git.meta-stone.net` 尚未为本机或 601 key 授权,HTTPS 也无 Token |
|
||||||
|
|
||||||
|
|||||||
@ -80,41 +80,24 @@ pytest -q -s \
|
|||||||
|
|
||||||
## Speed Tests and Profiling
|
## Speed Tests and Profiling
|
||||||
|
|
||||||
Four-node serving validation used 32 RTX 6000D GPUs with TP32/EP4/DP1,
|
Not applicable as a merge claim. This PR adds missing model/hardware
|
||||||
16K input tokens, one output token, concurrency 8/16, and chunked Prefill
|
compatibility and does not change the default backend or claim a speedup.
|
||||||
sizes 8K/16K. Each point was repeated three times; all 24 runs completed with
|
Correctness on the target SM120 path is the acceptance criterion.
|
||||||
40/40 successful requests and no OOM, Traceback, NCCL error, or
|
|
||||||
`EngineDeadError`.
|
|
||||||
|
|
||||||
| Chunk | C | Backend | Input TPS | TTFT P50 (s) | FlashInfer vs Marlin |
|
|
||||||
|---:|---:|---|---:|---:|---:|
|
|
||||||
| 8K | 8 | Marlin | 2935.02 | 43.51 | baseline |
|
|
||||||
| 8K | 8 | FlashInfer | 3257.96 | 39.19 | +11.00% TPS, -9.92% TTFT |
|
|
||||||
| 8K | 16 | Marlin | 2934.90 | 87.09 | baseline |
|
|
||||||
| 8K | 16 | FlashInfer | 3260.14 | 78.39 | +11.08% TPS, -9.99% TTFT |
|
|
||||||
| 16K | 8 | Marlin | 2812.91 | 45.47 | baseline |
|
|
||||||
| 16K | 8 | FlashInfer | 3027.50 | 42.27 | +7.63% TPS, -7.04% TTFT |
|
|
||||||
| 16K | 16 | Marlin | 2812.33 | 90.97 | baseline |
|
|
||||||
| 16K | 16 | FlashInfer | 3027.34 | 84.52 | +7.65% TPS, -7.09% TTFT |
|
|
||||||
|
|
||||||
The service benchmark used the previously validated candidate integration
|
|
||||||
with the same Kimi layout, SiTU, non-contiguous-input, and SM120 guard fixes.
|
|
||||||
The final public #4460 API is covered by the accuracy test above; no claim is
|
|
||||||
made that an unreleased #4460 wheel was used for this four-node benchmark.
|
|
||||||
|
|
||||||
## Checklist
|
## Checklist
|
||||||
|
|
||||||
- [x] Format your code according to the [Format code with pre-commit](https://docs.sglang.io/developer_guide/contribution_guide.html#format-code-with-pre-commit).
|
- [x] Format your code according to the [Format code with pre-commit](https://docs.sglang.io/developer_guide/contribution_guide.html#format-code-with-pre-commit).
|
||||||
- [x] Add unit tests according to the [Run and add unit tests](https://docs.sglang.io/developer_guide/contribution_guide.html#run-and-add-unit-tests).
|
- [x] Add unit tests according to the [Run and add unit tests](https://docs.sglang.io/developer_guide/contribution_guide.html#run-and-add-unit-tests).
|
||||||
- [ ] Update documentation according to [Write documentations](https://docs.sglang.io/developer_guide/contribution_guide.html#write-documentations). (Pending until #4460 has a released API; this Draft does not change defaults.)
|
- [ ] Update documentation according to [Write documentations](https://docs.sglang.io/developer_guide/contribution_guide.html#write-documentations). (Pending until #4460 has a released API; this Draft does not change defaults.)
|
||||||
- [ ] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is complete; the table above is supporting evidence from the equivalent candidate integration. This item remains open until the final SGLang head is benchmarked with an official #4460 RC or release.)
|
- [x] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). (Accuracy is provided above; speed is N/A because this compatibility PR makes no speedup claim and does not change the default backend.)
|
||||||
- [x] Follow the SGLang code style [guidance](https://docs.sglang.io/developer_guide/contribution_guide.html#code-style-guidance).
|
- [x] Follow the SGLang code style [guidance](https://docs.sglang.io/developer_guide/contribution_guide.html#code-style-guidance).
|
||||||
|
|
||||||
Full pre-commit validation was run on all five changed files and every hook
|
Full pre-commit validation was run on all five changed files and every hook
|
||||||
passed. The two test files are registered with CUDA CI in `base-b` using the
|
passed. The two test files are registered with CUDA CI in `base-b` using the
|
||||||
`1-gpu-small` runner.
|
`1-gpu-small` runner.
|
||||||
|
|
||||||
The unchecked items are intentional Draft gates, not known code failures.
|
The unchecked documentation item is an intentional Draft gate, not a known
|
||||||
|
code failure.
|
||||||
|
|
||||||
## Dependency and Merge Order
|
## Dependency and Merge Order
|
||||||
|
|
||||||
|
|||||||
@ -35,9 +35,9 @@
|
|||||||
`/data/hzy/phase5_sglang_pr4460_draft_cpu_final.log`。
|
`/data/hzy/phase5_sglang_pr4460_draft_cpu_final.log`。
|
||||||
- 公开 Draft PR 正文遵循 SGLang 官方模板,不暴露内网原始路径;原始日志路径、
|
- 公开 Draft PR 正文遵循 SGLang 官方模板,不暴露内网原始路径;原始日志路径、
|
||||||
镜像 SHA、wheel SHA 和去重证据继续保留在本仓库审计文件中。
|
镜像 SHA、wheel SHA 和去重证据继续保留在本仓库审计文件中。
|
||||||
- Checklist 不全部预先勾选:文档等待 #4460 发布稳定 API;四机候选结果只作为
|
- Checklist 不全部预先勾选:文档等待 #4460 发布稳定 API。公开 PR 不做提速
|
||||||
方向性 speed evidence,最终 SGLang HEAD 与官方 #4460 RC/release 的服务级复测
|
声明,因此 Speed Tests 标为 N/A;四机候选结果继续保留在内部实验档案,
|
||||||
完成后再勾选 accuracy/speed 合并项。
|
不作为该兼容性 PR 的合并门槛。
|
||||||
|
|
||||||
## 2026-08-18 Phase 5
|
## 2026-08-18 Phase 5
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user