qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Fix] Materialize FlashInfer MXFP8 input layout

Browse Source
This commit is contained in:
Zhiyi Hong 2026-08-18 14:50:13 +08:00
parent d0863501ca
commit be9d6bfe3a
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill/patches/sglang-kimi-image-sm120-flashinfer-mxfp4.patch
View File

@ -36,7 +36,12 @@
x = dispatch_output.hidden_states x = dispatch_output.hidden_states
topk_output = dispatch_output.topk_output topk_output = dispatch_output.topk_output
@@ -384,7 +388,7 @@ @@ -343,3 +347,4 @@
+ x = x.contiguous()
x, input_sf = mxfp8_quantize(
x,
is_sf_swizzled_layout=True,
@@ -384,7 +389,7 @@
ep_rank=quant_info.moe_ep_rank, ep_rank=quant_info.moe_ep_rank,
use_w4_group_scaling=not use_mxfp8_act_scaling, use_w4_group_scaling=not use_mxfp8_act_scaling,
use_mxfp8_act_scaling=use_mxfp8_act_scaling, use_mxfp8_act_scaling=use_mxfp8_act_scaling,