[Profiling] Explain Kimi-K3 Deep PP scaling

README.md

@@ -1,5 +1,31 @@
 # sskj — 多平台大模型推理性能基准测试项目
 
+**更新（2026-08-20 10:13:01 CST）**：完成 Kimi-K3 Prefill TP Reduce Scatter 可行性审计并停止该方向。K3 的 MLA 输出门控仍依赖完整 7168 维 hidden，且 69/93 层为 KDA；保留或恢复 gate hidden 后，原 hidden All-Reduce 无法消除并新增 2112 维 latent All-Gather，估算通信量反增约 14.7%。研究原型仅保留为否决证据，不进入四节点实验或上游 PR；后续转向 MoE A2A 与 Pipeline Parallelism。详见 `experiments/pro6000/kimi3_pro6000_sglang_tp_reduce_scatter_prefill/README.md`。
+
+**更新（2026-08-19 16:51:43 CST）**：完成 Kimi-K3 / RTX 6000D / SGLang Prefill MoE backend 交付文档收敛与 Chunk=4K 补测。四节点 TP32×EP4、16K→1、C=8/16 下 12/12 重复、480/480 请求成功；FlashInfer 相对 Marlin 提升约 9.8% Input TPS、降低约 9.0% TTFT。结合 4K/8K/16K 全部结果，最终推荐保持 `flashinfer_mxfp4 + chunked_prefill_size=8192`。
+
+**更新（2026-08-19 14:23:52 CST）**：601 的 GitLab SSH key 已生效，内部 Kimi-K3 SM120 审计分支已推送至 `qqtang/sskj:hzy-kimi-k3-sm120-flashinfer-mxfp4`。远端已有普通分支 `hzy`，因此不能创建 `hzy/...` ref，最终采用无层级冲突的分支名；GitHub SGLang feature 分支保持不变。
+
+**更新（2026-08-19 13:58:50 CST）**：用户同步 GitHub fork 后，将 Kimi-K3 SM120 SGLang 的 7 个提交无冲突重放到最新 `main` `593b1a9`，`range-diff` 逐提交确认补丁等价；最终 Draft HEAD `b95b534` 已推送至 `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`。在 601 统一验证镜像中重跑最终 HEAD 的语法与定向回归，结果为 `10 passed, 3 deselected`；代表性四机性能数据保留，不重复消耗 GPU。
+
+**更新（2026-08-19 13:07:48 CST）**：完成 Kimi-K3 SM120 SGLang Draft PR 的最终代表点验收。基于 SGLang `300c87a` 与 FlashInfer #4460 `b525c51`，在 601-604 四节点 TP32×EP4 上完成 16K→1、C=8、Chunk=8K 的 Marlin/FlashInfer 各 3 次 A/B；240/240 请求成功。FlashInfer 的 Input TPS 中位数提升 10.96%，TTFT P50/P95 分别下降 9.91%/9.92%。公开 PR Checklist 已全部勾选，轻量汇总和原始证据索引已归档。
+
+**更新（2026-08-19 10:45:18 CST）**：新增 Kimi-K3 SM120 SGLang Draft PR 的单点代表性验收入口。使用精确 SGLang `300c87a`、FlashInfer #4460 实现 `b525c51` 和统一镜像，在 601-604 四节点 TP32×EP4 上仅复测 16K→1、C=8、Chunk=8K 的 Marlin/FlashInfer A/B，各 3 次重复；完整命令、镜像身份、原始日志和汇总统一落入单个 Run 目录。
+
+**更新（2026-08-18 23:00:42 CST）**：完成依赖 FlashInfer #4460 的 SGLang Draft PR 收敛。确认不提交任何 FlashInfer PR，也不 vendor 或 pin 未合并 kernel；在 601 GPU6 上从 #4460 源码构建 FlashInfer 0.6.18 wheel，SGLang 定向测试 8/8 通过。Draft 标题、依赖、兼容策略、测试和四机 EP4 数据见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md`。
+
+**更新（2026-08-18 22:21:40 CST）**：完成 Kimi-K3 SM120 FlashInfer MXFP4 去重审计。确认 CUTLASS SiTU kernel 已由未合并的 FlashInfer PR #4460 实现，不再提交平行 kernel；SGLang 贡献收敛为 Kimi gate/up 与 scale 布局、SiTU 4.0/25.0 参数映射、非连续输入和 SM120 attention-residual guard，并保留 601-604 EP4 的全部服务级结果。详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/UPSTREAM_DUPLICATION_AUDIT.md`。
+
+**更新（2026-08-18 18:38:17 CST）**：完成 Kimi-K3 四节点 TP32×EP4 的真实 Prefill MoE backend 验收；24/24 重复均 40/40 成功，FlashInfer MXFP4 在 8K Chunk、C=8/16 下相对 Marlin 提升约 11% Input TPS、降低约 10% TTFT，并确认 8K 优于 16K。实验默认 EP 改为 4，补齐非连续 Prefill 输入与 FlashInfer 官方 SiTU 回归，详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/README.md`。
+
+**更新（2026-08-18 13:55:30 CST）**：Kimi-K3 四节点 MoE backend 验收入口支持通过环境变量覆盖 EP size 及 backend/chunk/concurrency 子矩阵，用于先执行 TP32×EP4、16K Chunk、16K→1、C=16 的最大压力容量验证，同时保留默认 TP32×EP32 全矩阵。
+
+**更新（2026-08-18 13:03:33 CST）**：Phase 5 候选镜像改为针对 Kimi 基础镜像原生 SGLang 源码应用最小兼容补丁，仅补齐 SM120 FlashInfer MXFP4 backend 分发、权重布局与 SiTU 激活映射，避免整体替换新版 Python 包造成 `sglang-kernel` 版本错配。
+
+**更新（2026-08-18 12:24:19 CST）**：新增 Kimi-K3 / 601-604 / SGLang TP32×EP32 的真实 Prefill MoE backend 验收实验；固定比较 Marlin 与 FlashInfer MXFP4 在 16K→1、C=8/16、Chunk=8K/16K 下的 TTFT、E2E 与 Input TPS，并提供四节点一致候选镜像构建、原始证据和自动汇总，详见 `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill/README.md`。
+
+**更新（2026-08-17 11:56:47 CST）**：完成 Kimi-K3 / RTX PRO 6000D / SGLang FlashInfer MXFP4 MoE Phase 2 correctness harness；SwiGLU 控制矩阵 8/8 通过，覆盖 edge routing、EP global ID、NaN/Inf、确定性、CUDA Graph 和 Kimi 真尺寸，SiTU 目标矩阵 8/8 稳定复现同一 C++ 缺口，详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/README.md`。
+
 > 历史更新见 `git log`。项目目的与工作流见下方。
 > **项目目的**：当新显卡（GPU/NPU）到货时，用最短时间完成大模型在该平台上的推理性能评估与部署配置选型。
 > 当前模型：DeepSeek-V4-Flash（FP8 / INT8）；后续接入 GLM5.2，**完全复用**本项目的实验与报告流程。
@@ -45,6 +71,8 @@
 | `experiments/pro6000/dsv4_pro6000_vllm_tp_dp_matrix/` | RTX 6000D + vLLM |
 | `experiments/pro6000/dsv4_pro6000_sglang_tp_dp_matrix/` | RTX 6000D + SGLang |
 | `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/` | RTX 6000D + SGLang，Kimi-K3（TP32×EP32，部署手册见 docs/KIMI_K3_DEPLOY.md） |
+| `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill/` | RTX 6000D + SGLang，Kimi-K3 真实 Prefill 的 Marlin/FlashInfer MXFP4 与 Chunk 8K/16K 对照 |
+| `experiments/pro6000/kimi3_pro6000_sglang_tp_reduce_scatter_prefill/` | Kimi-K3 Prefill TP Reduce Scatter 可行性审计；因 MLA gate 与 KDA 依赖完整 hidden，结论为不推荐继续 |
 | `experiments/pro6000/kimi3_pro6000_pd_rdma/` | RTX 6000D + SGLang，Kimi-K3 PD 分离（MoonCake RDMA，8 节点，见 deploy_pd.sh + docs/KIMI_K3_DEPLOY.md 附录 B） |
 
 每个目录内：`run_bench.sh` 跑固定并发矩阵；`run_adaptive_concurrency.sh` 从 C=1 指数倍增搜饱和点；`run_adaptive_concurrency_add16.sh` 从 C=16 线性 +16 步进、带 TTFT SLO 停止与回退（当前主力用法，见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`）。
@@ -60,7 +88,6 @@
 | `experiments/h200/dsv4_h200_vllm_tp{2,4,8}_custom_bench/` | 自定义压测客户端（多服务负载均衡） |
 | `experiments/h200/dsv4_h200_max_context_length/` `…/dsv4_h200_long_context_matrix/` `…/dsv4_h200_256k_4k_probe/` `…/dsv4_h200_64k_sglang_vs_vllm/` | 长上下文专项 |
 | `experiments/p800/dsv4_p800_sglang/` `…/dsv4_p800_max_context_length/` `…/dsv4_p800_long_context_matrix/` `…/dsv4_p800_256k_4k_probe/` | P800 baseline 与长上下文专项 |
-| `experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/` | RTX 6000D + SGLang，GLM-5.3-NVFP4 双场景压测标准（16k/512 高并发 + 128k/64k 90% 命中低并发；真实语料 run-id 窗口纪律 + 三套部署配置 A/B/CAR，基线与口径见目录 README；部署 profile 见 `deploy/profiles/pro6000/glm53_nvfp4_*`） |
 | `experiments/TEMPLATE/` | 老式固定场景实验模板 |
 
 ## 快速复现

deploy/CURRENT.md

@@ -1,35 +0,0 @@
-# 现役部署状态页（live 核验于 2026-09-10，60.8 当日核验；其余机器 09-09 口径）
-
-> 本页记录 174.1.60.x 集群"现在跑的是什么"。改动机器前后先读这页；
-> **任何变动（停/起/改配置）完成后必须更新本页**。核验命令：`docker ps` + `nvidia-smi`。
-
-## 机器状态（2026-09-09 实测）
-
-| 机器 | 在役 | 口径 / 归属 | 对应 profile |
-|---|---|---|---|
-| 60.1 (6000D-1) | `glm53-pp4`（Up，8 卡满载，:30000） | **方案 D 生产**。09-09/10 两轮实验窗口停机（凌晨 D 基线战役 ~7h + 日间 A 基线战役 ~9.5h）均恢复核验（health+生成冒烟）。**D 配置升级路径已交付未执行**：latest 镜像+开 autotune+持久缓存，16k/512 cc8-64 五点 +1.2~3.2%（A/B/A 确认），profile `..._tp2pp4_latest_autotune.env`——**仅适用 D 配置**：09-10 A 基线（TP8+EAGLE）战役证明 latest 与 autotune 在 A 配置上均负收益（eager 路径对代差/ tactic 敏感，图内不敏感），A 配置维持 nightly+autotune off；**IPC@TP8 真增益 +4.6~9.1% 已验证**（vs 同镜像基线全点胜、cc8/cc64 分布不重叠，收益随 TP 度单调升：TP2 −3.5%→TP8 +9.1%）但补丁包仅兼容 latest（sglang 4 文件依赖新版 runtime_context API），net vs 生产≈0；落地=移植回 nightly 或等上游 PR #34528 合入，未执行。同轮 A 基线画像：生产 A 掉图陷阱主导场景二（cc16+ 全程 eager，TPOT 168-197ms，平台 ~70-73；已发表 95/96 是 A-s2 图覆盖 bs16 变体）；日内漂移=eager 点 8.5h 后 −6~8% 而图内点不动（CPU 侧签名）。实验全量 `experiments/pro6000/glm53_nvfp4_dsv4_migration_bench/`（REPORT.md + REPORT_A_BASELINE.md）。`dsv4_scan` 经授权保持停止，还原说明 `/root/dsv4_scan_restore_note.txt` | `profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4.env`（现役）；升级候选 `..._tp2pp4_latest_autotune.env`（仅 D 配置） |
-| 60.2 (6000D-2) | `glm53-nvfp4` 实验容器（09-08 晚 TP1PP8 phase，run_phase.sh 实验链进行中） | **他人实验进行中，勿动**（方案 F PD 链已拆除；GPU7 曾有外部裸金属任务）。动卡前仍先核实归属 | `profiles/pro6000/glm53_nvfp4_pro6000_pd_{prefill→decode 侧}.env` |
-| 60.3 | 无容器，但 8 卡被外部裸金属训练占用（`/data/mas/larm`，09-08 晚实测） | 外部任务，勿动（此前台账漏记） | — |
-| 60.4 | `glm53-nvfp4`（Up，:30000，restart=unless-stopped） | **TP8+EAGLE+custom-AR 1stage（E7b 配方）在役**（09-09 下午部署：EAGLE 4/1/5/mem0.90/MRR16/chunk8192/ctxlen270336/fp8KV+hicache3/decode 图桶 1-8/双 parser；CAR 补丁三处全注入、8 rank `SSKJ_CAR_PATCH_ACTIVE` 确认，health 200、生成冒烟、质量门 7/7 见 `/root/qg_604_car.log`）。启动 `/root/deploy_glm53_604_exp.sh`（=60.7 实验版逐字拷贝，`RESTART=yes CAR_PATCH=1`），补丁 `/root/patches/`（md5 与仓库 car_patch 归档一致）。当日早间曾短暂部署 TP2PP4 D 配方复刻（deploy_s2_test_604.sh 留盘可切回）后被本方案替换；同日经授权清退外部 vllm 评测流水线（tmux `mas` 的 run_multiseed.sh 链，--resume 可续跑） | `experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/`（deploy_glm53_604_exp.sh + car_patch/ 补丁快照） |
-| 60.5 | `glm53-nvfp4`（Up 2d，09-09 只读核验） | **NVFP4 团队生产**（deploy_glm53_605.sh，md5 fcd9109b）。生产机铁律：不实验、不重启、不覆盖脚本。**交付升级路径已更新为 v3（09-09 hit90 场景优胜 TP4PP2-hicache@0.90，池 647,040/c4 并发独立文档/hit90 cc8 out +34% vs 现役）：脚本 `experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/scripts/deploy_glm53_605_v3.sh` + 补丁束（60.8:/root/glm53_r37_patch_bundle_v3.tar.gz，md5 6922e534，需 scp 至 60.5）+ 需 docker pull cu13 镜像；未执行、未落 60.5 磁盘（60.5:/root 仅有原脚本，核验过）。此前 v2（TP2PP4-hicache，冷缓存口径优胜）被 v3 取代，仍留仓库可作"容量优先 6 条文档/512k 单条最快"备选；回滚=原脚本** | `profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp8eagle.env`（方案 A 口径）；待切 `glm53_nvfp4_pro6000_sglang_tp4pp2_hicache.env`（容量优先备选 `..._tp2pp4_hicache.env`） |
-| 60.6 | 无容器，但 8 卡被外部裸金属实验占用（`/data/hzy/sparse-opd-*`，09-08 晚实测） | 外部任务，勿动（此前台账漏记） | — |
-| 60.7 | 基本空（4 卡仍有 `/home/user/dirA_exp` 外部小任务，09-08 晚实测） | 09-08 已拆除清空（方案 F 前身单机实验 + 场景一深优资产留盘），不再恢复 | — |
-| 60.8 | `glm53-nvfp4`（Up，:30000，restart=unless-stopped） | **TP4PP2-nomtp@0.90 hit90 优胜口径在役**（09-09 晚接替同日早间的 TP2PP4@0.85：同 r37 栈 cu13+9 挂载+de-GLOO，仅改 tp4/pp2 + memfrac 0.90；KV 池 **647,040**；实测实例原样保留切 restart 策略，未重建容器。启动 `bash /root/deploy_ppmtp_r37.sh '--tp 4 --pp-size 2 --disable-overlap-schedule --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length 1048576' nomtp 8192 0.90 1 1 0 0`）。hit90（90% 命中 i128k/o512）out cc1/2/3/4/8/16 = **28.8/47.5/63.9/76.2/106.4/125.7**（vs TP2PP4 基线 cc4 +14%/cc8 +15%）；并发独立 128k 文档 **4 条**（cap cc4 98.5 零排队）、512k 单条 ✓ TTFT 135.8s、质量门 **7/7**；同轮判决：DP attention 容量负收益判死、DCP 对 DSA 静默算错禁用、MTP@128k accept 2.07 判负。让步项（知情）：容量 4 条（TP2PP4 为 6）、512k 单条慢 33%。回滚 TP2PP4 = 同命令改 `'--tp 2 --pp-size 4'` + memfrac 0.85。**09-10 B300 对标战役**：停役（容器 rename 保全 `glm53-nvfp4-insvc`）→ 双臂 B300 场景矩阵（TP2PP4-D 口径 24 点 + E7b 配方）→ E7b 高并发调参复测（初测 MRR16+图1-8 掉图断崖 → MRR64+图桶 1-64，`deploy_glm53_e7b_hicc.sh`，三场景 c8-c64 共 10 点全 OK；报告正文采用复测值：掉图断崖已修复、16K 仍 TP2PP4 占优（E7b 被 prefill 墙封 ~100 tok/s 平台）、decode 密集 1K→4K E7b 全档反超（c64 out 826.5 tok/s 全场最高、超 TP2PP4 72%）、边界仅 TP2PP4 可达、与 B300 绝对差 4-5×；报告飞书 wiki `A7V3wZTQeifCB4krdi6cA834nW9`，已整文更新）→ TP2PP4 高并发调参复测（初测 C=64 受 MRR48 封顶 48 活跃+16 排队、decode 图默认覆盖 bs≤256 从未掉图 → MRR64，`deploy_glm53_pp4_mrr64.sh` 逐 token 仅改 MRR；三场景 c32/c64 共 6 点两轮对齐 ≤1.8%（pass-1 + v2 有序版）+ 全新实例 MRR48 对照归因；正文采用 v2 值：4.1/4.2 c64 out +27%/+19%（433/572）、三场景 c64 TTFT P95 全塌缩（20.0/337.9/134.8s → 7.2/7.3/89.2s）、16K c64 到池顶 ~59 活跃 + 3 次回退（确定性行为）、1K 短输入 c32 付 −14% MRR64 配置代价（276→266→228 = −3% 新鲜度 −14% 配置，三实例可复现、机制未定位）、16K c32 无代价；报告已再次整文更新）→ **原容器恢复并三次核验**（rename 回 + start，fired up/health 200/16K 抽测 ok、KV 池 647,040 与启动口径逐字一致、显存水位 77.2/82.3 GiB 与停役前一致） | `profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp4pp2_hicache.env`；实验全量 `experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/`；B300 对标 `experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/`；前史 `experiments/pro6000/glm53_nvfp4_128k_capacity_topology/`（TP2PP4 口径）+ r37 `experiments/pro6000/glm53_ppmtp_r37_verify_graph/` |
-
-## 方案 A-F 一览（GLM-5.3-NVFP4 @ pro6000，2026-09-08 双场景报告口径）
-
-| 方案 | 一句话 | profile / 脚本 |
-|---|---|---|
-| A | TP8 + EAGLE，基准；60.5 生产口径 | `glm53_nvfp4_pro6000_sglang_tp8eagle.env` |
-| B | TP4 PP2 | `glm53_nvfp4_pro6000_sglang_tp4pp2.env` |
-| C | TP4 PP2 + IndexCache(freq=4) | `glm53_nvfp4_pro6000_sglang_tp4pp2_index.env` |
-| D | TP2 PP4；60.1 生产在役 | `glm53_nvfp4_pro6000_sglang_tp2pp4.env` |
-| E | TP8 + DFlash2 投机 | `glm53_nvfp4_pro6000_sglang_tp8dflash2.env` |
-| F | PD 分离双机链（mc-master→prefill→decode→router） | `glm53_nvfp4_pro6000_pd_{master,prefill,decode,router}.env` + `deploy/PD_CHAIN.md` |
-
-压测数据：飞书《GLM-5.3-NVFP4 双场景压测报告》（`SZUSdEqY1oRVxGxgILBcHqPJnEc`）。
-
-## 防漂移
-
-每个在役容器用 `deploy/verify_profile.sh <profile.env>` 定期核验（镜像 digest + 启动参数 +
-端口），发现不一致 = 容器被人手改过，先查清归属再处理。

deploy/PD_CHAIN.md

@@ -1,96 +0,0 @@
-# 方案F：GLM-5.3-NVFP4 PD 分离完整链（双机 6000D）编排手册
-
-> 2026-09-08 实测终态。四角色、两台机、启动顺序强制。吞吐换算口径：**链合计 ÷2 = 单机等效**（与单机方案 A-E 可比）。
-> 完整压测数据见飞书《GLM-5.3-NVFP4 双场景压测报告》方案 F 行（文档 `SZUSdEqY1oRVxGxgILBcHqPJnEc` / wiki `NzbMwzmKviYidRkZYRrc8GYQnrf`）。
-
-## 拓扑
-
-| 角色 | 机器 | 容器 | 端口 | profile |
-|---|---|---|---|---|
-| 1. mc-master（Mooncake 元数据） | 174.1.60.1 | `mc-master` | 50051 | `profiles/pro6000/glm53_nvfp4_pro6000_pd_master.env` |
-| 2. prefill（TP4 PP2 + DFLASH 草稿） | 174.1.60.1 | `glm53-pd-smoke-prefill` | 30000 | `profiles/pro6000/glm53_nvfp4_pro6000_pd_prefill.env` |
-| 3. decode（TP8 + DFLASH v5 配方） | 174.1.60.2 | `glm53-s1-decode` | 30000 | `profiles/pro6000/glm53_nvfp4_pro6000_pd_decode.env` |
-| 4. router（MiniLB） | 174.1.60.2 | `pd-smoke-router` | 31000 | `profiles/pro6000/glm53_nvfp4_pro6000_pd_router.env` |
-
-镜像统一：`lmsysorg/sglang:nightly-dev-20260828-daf63171`
-（digest `sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399`）。
-
-## 启动顺序（强制）
-
-```
-mc-master (60.1)  →  prefill (60.1)  →  decode (60.2)  →  router (60.2)
-```
-
-对应脚本（`experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/`）：
-
-```bash
-# 60.1（注意：60.1 日常跑生产容器 glm53-pp4，先停它，见下节）
-bash /tmp/deploy_pd_smoke_master.sh                                    # 判据 ss :50051
-bash /tmp/deploy_pd_probe.sh p1b_prefill_tp4pp2_cps8k_radix_launch.sh   # 判据 :30000/health 200，~10-15min
-# 60.2
-bash /tmp/deploy_s1_decode.sh                                          # 判据 :30000/health 200，~10min
-bash /tmp/deploy_pd_smoke_router.sh                                    # 判据 ss :31000
-```
-
-场景一（90% 命中）必须用 **radix 变体** launch（`p1b_prefill_tp4pp2_cps8k_radix_launch.sh`，
-与基线唯一差异 = 无 `--disable-radix-cache`）。场景二 0 命中，radix 开销可忽略，两场景共用同一部署。
-
-## 基础设施依赖（缺一不可）
-
-| 依赖 | 位置 | 说明 |
-|---|---|---|
-| sglang 补丁树 | `/data/sglang_patch_glm53` → 容器 `/sgl-workspace/sglang` | 与镜像原版差 **11 个文件**（10 改 + 1 新增 `dflash_pp.py`），清单与 diff 见 `platforms/patches/pro6000/glm53_pd_chain/`。无补丁则 DFlash+PD 冷启动接线缺失，decode 首请求 400 |
-| mooncake wheel | `/data/flashkda_deploy/wheels/mooncake_transfer_engine_cuda13-0.3.12.post1-cp312-cp312-manylinux_2_28_x86_64.whl` | 容器内 launch 脚本 `pip install --no-deps` 自装 |
-| DFlash2 草稿权重 | `/data/hf_models/GLM-5.3-DFlash2` | 两角色都要挂 |
-| IB 设备 | mlx5_0–mlx5_3 + `--device /dev/infiniband` + `--ulimit memlock=-1` | RDMA KV 传输通道 |
-| 环境变量 | `MOONCAKE_MASTER=174.1.60.1:50051` `MOONCAKE_PROTOCOL=rdma` | prefill/decode 容器都要 |
-
-## 质量门（经 router :31000，全链路含 KV transfer + 草稿）
-
-60.2 上生成 router 口径的 gate 副本：
-
-```bash
-sed 's/PORT=30000/PORT=31000/' /root/quality_gate_605.sh > /tmp/quality_gate_31000.sh
-```
-
-判据：GSM8K×5 + 中文推理必须过。tool call 预期不过（链配方无 parser，与方案 B/D 同口径的已知配置缺口，记录放行）。
-2026-09-08 实测 **6/7**（核心项全过，DFlash 草稿无质量损失）。
-
-## 压测
-
-驱动脚本：`run_pd_s1.sh` / `run_pd_s2.sh`（同 scripts 目录），跑在 60.1（语料本地，
-`--url http://174.1.60.2:31000/generate` 走 router，`--container glm53-pd-smoke-prefill` 读 prefill 日志核命中率）。
-
-- **场景一**：128k/64k × cc1-4，nreq8，shared-frac 0.9，canonical 窗口 9301-9308
-- **场景二**：cc8/16/32 canonical 9311-9313（nreq 16/32/32）；cc40/64 全新窗口 pool-override
-  16,384,000 / 17,203,200（nreq 40/64 单轮满波）
-- 指标读取：命中率读 prefill 容器日志（PP2 日志计数 ×2，不影响比值）；accept 从 decode 日志读
-  （router 转发后 usage 字段可能缺失）
-- 每点核验：ok/failed=0、retractions、s1 hit≈0.90、s2 hit=0
-
-### 2026-09-08 实测判决（13/13 干净点）
-
-- **场景一**：128k cc1 TTFT 3.91s = 六方案最低（跨机重叠把 prefill 与 RDMA 传输完全藏住），
-  但 cc≥2 时 decode 侧排队，TTFT 堆到 31.0s；÷2 输出吞吐 21.2-23.5 tok/s，仅为 A 的 30-57%。
-  根因：decode KV 池 214,336 token 按 16k 场景定容 → 128k 仅 1 驻留、64k 仅 3。
-- **场景二**：÷2 单机等效 74-85 tok/s = A 的 75-96%，但 TTFT 较 A 减半。
-  判决：**两机买 TTFT、不买吞吐**。decode MRR12 在 cc16+ 饱和；DFlash accept 在 12-batch
-  verify 下掉到 1.8（单机 EAGLE 2.84）。
-- **PD 机制验证**：跨机 prefill/decode 重叠成立、mooncake RDMA 传输被吸收——机制无罪，
-  容量配置（decode 池按 16k 定容）是判决主因。
-
-## 拆链与生产恢复（顺序固定）
-
-```bash
-# 1. 拆链
-# 60.1: docker rm -f mc-master glm53-pd-smoke-prefill
-# 60.2: docker rm -f glm53-s1-decode pd-smoke-router
-# 2. 等双机显存排空（60.1 <2000MiB；60.2 GPU7 常有外部裸金属任务 main_v2.py，核实归属勿清）
-# 3. 恢复 60.1 生产：bash /tmp/deploy_glm53_pp4.sh → health 200 + docker inspect Args 核对
-```
-
-## 60.1 生产停机窗口提示
-
-60.1 日常跑生产 `glm53-pp4`（方案 D，TP2PP4，8 卡满载）。拉 prefill 角色前必须停生产；
-`deploy_glm53_pp4.sh` 生成的参数与生产容器 `docker inspect` Args 已核对逐字一致，
-恢复即完全复现。全程停机 ~2-2.5h（部署+13 点压测+质量门+恢复）。

deploy/manifests/GLM-5.3-DFlash2.md5

@@ -1,4 +0,0 @@
-302f15789d891aa39620209286252fed  ./README.md
-ce1962554abf73c6ad953aa0ac3f20d7  ./config.json
-5e170425c97cda8f798c74041979a569  ./configuration.json
-32c81842f12e56e6ac2a1feaafd5bfa7  ./model.safetensors

deploy/manifests/GLM-5.3-NVFP4.md5

@@ -1,58 +0,0 @@
-434f8692a4914f39714df2c4493c92d7  ./.gitattributes
-245254422e0ca504a5f09e37719ca718  ./LICENSE
-02e0c207304666abdd93de6a3780c52a  ./README.md
-e4c9e1000a513f680dd796f76d1097e0  ./chat_template.jinja
-d42bba3f99ce9b2705bb19c41930ea0c  ./config.json
-5e170425c97cda8f798c74041979a569  ./configuration.json
-ee2560496cd373851033b3c05ee461fb  ./generation_config.json
-027f9b7a7bb96f30962b5be7e4e4a5f1  ./hf_quant_config.json
-57cdec3085e0805c4ad5875f61c63b68  ./model-00030-of-00047.safetensors
-be93abf8f84e890b8dd1d4187ef540c2  ./model-00001-of-00047.safetensors
-a4a627a33e69e91491c9a3d3a7c9cba9  ./model-00031-of-00047.safetensors
-500a87d1fd1dd6a93400539227361f5d  ./model-00002-of-00047.safetensors
-c732e4ee2f910af666418abd547d34cb  ./model-00032-of-00047.safetensors
-68024a4367c3341402766fcc40d383f6  ./model-00003-of-00047.safetensors
-94188d8acbc7fd47ee99400507793d6a  ./model-00033-of-00047.safetensors
-b5870fe5e8704e879895a7b5a3be5136  ./model-00004-of-00047.safetensors
-87b17b02643b1c14ad2e90900b3dfa89  ./model-00034-of-00047.safetensors
-f2f51d0be2c49ba6569d59d8d1db5236  ./model-00005-of-00047.safetensors
-915184c001e70fea709ffbb5242e4a8c  ./model-00035-of-00047.safetensors
-fe2146b7c71775e51bad2322c52c34d1  ./model-00006-of-00047.safetensors
-56d618d4ba615cc19a0898d02d82bb52  ./model-00036-of-00047.safetensors
-03c0d5ea0471553cce658b70daed7b47  ./model-00007-of-00047.safetensors
-c76ac8dd7dbddbc4a636118ad8a5f713  ./model-00038-of-00047.safetensors
-49e246d6faa068641d2858e36b6bb5cd  ./model-00008-of-00047.safetensors
-8f31262bc0a079c946a9ec1cdeb4051e  ./model-00037-of-00047.safetensors
-b956d449de03a2db14fee14df7aaddac  ./model-00009-of-00047.safetensors
-087456e9659f4d7c8c9a4af093b7c828  ./model-00039-of-00047.safetensors
-f94083fc155df999ab9ff27f6f67b978  ./model-00010-of-00047.safetensors
-bfd9606be27f5efbb34d85dc8fad3cea  ./model-00040-of-00047.safetensors
-775a242af0663e5c65bc1b41a8930314  ./model-00011-of-00047.safetensors
-a4c70502c1470d1cd520f4f2b0f968d8  ./model-00041-of-00047.safetensors
-1d9d4ca43e411b977cded63b3447b8b8  ./model-00012-of-00047.safetensors
-f00a801a8de9a31b2ff686f6f8924d41  ./model-00042-of-00047.safetensors
-5feebc534bb09cee7b7e908aab8e2108  ./model-00013-of-00047.safetensors
-b58238047ce49362d39c36b54761c4cf  ./model-00043-of-00047.safetensors
-c15c8fa6325af6ca302e10425b357199  ./model-00014-of-00047.safetensors
-fb652edaa0a450b7ce333829e363ee7d  ./model.safetensors.index.json
-ce7e000a8d1c076b13346374f378e1f3  ./model-00015-of-00047.safetensors
-752f6cd2e6a4a2ea824d1b513530e0b0  ./tokenizer.json
-96ab98059044ac21eff43da3d9882689  ./model-00016-of-00047.safetensors
-3312b710133454ca7150bccad6381bd8  ./tokenizer_config.json
-82429b288fe2a97cb771e78bea60a0bd  ./model-00017-of-00047.safetensors
-7605255172c381931a496b316faecdb7  ./model-00018-of-00047.safetensors
-e7fdbb1bd3e82f97831f4c1c69e0403a  ./model-00019-of-00047.safetensors
-f5d355f9a737d7aa270d178a1bceeeec  ./model-00020-of-00047.safetensors
-50b23a8dc4932840e5ae2062ca04b60a  ./model-00021-of-00047.safetensors
-285172cbc5e900681efc5783f04a346a  ./model-00022-of-00047.safetensors
-accdb4eb9e0831705bcd2d39e4f3bdcb  ./model-00023-of-00047.safetensors
-9182f13eb2670db2a5312aaca2f9b1da  ./model-00024-of-00047.safetensors
-198809340ec4795bdcfafdf7cd3074c7  ./model-00025-of-00047.safetensors
-da5e3a1afe95fe14c5d2d1f432f4a933  ./model-00026-of-00047.safetensors
-1d15d1696fd6d2cab3ed8726c3b25112  ./model-00027-of-00047.safetensors
-6147f7b5acb9a297eab276b16a58efe7  ./model-00028-of-00047.safetensors
-529b4048cf2f18d9eeaf8d1ce37b60cd  ./model-00029-of-00047.safetensors
-ffe813a8385daf74f69f632d9437085c  ./model-00044-of-00047.safetensors
-f67ddcaf1b68509fd6ab8d0399632a75  ./model-00045-of-00047.safetensors
-489dc1a568e671176a98e7cc23519c17  ./model-00046-of-00047.safetensors
-3a7abe2df1bd98e749ee8733370ea78b  ./model-00047-of-00047.safetensors

deploy/manifests/README.md

@@ -1,15 +0,0 @@
-# 权重完整性清单（md5）
-
-2026-09-08 在 174.1.60.1 `/data/hf_models/` 下对两个模型目录逐文件 `md5sum` 的原样输出。
-用于：新机器部署前核对权重传输完整性、或怀疑权重被改动时做漂移检测。
-
-校验方法（在权重目录下）：
-
-```bash
-md5sum -c /path/to/GLM-5.3-NVFP4.md5     # 清单内路径为 ./ 相对路径
-```
-
-| 清单 | 模型 | 规模 | 用途 |
-|---|---|---|---|
-| `GLM-5.3-NVFP4.md5` | GLM-5.3-NVFP4 主模型 | 47 分片 + 配置共 55 文件 | 方案 A-F 全部部署的目标模型 |
-| `GLM-5.3-DFlash2.md5` | GLM-5.3-DFlash2 草稿模型 | 单分片 model.safetensors + 配置共 4 文件 | 方案 E 与方案 F 的 DFLASH 投机草稿 |

deploy/profiles/pro6000/glm53_nvfp4_pro6000_pd_decode.env

@@ -1,52 +0,0 @@
-# GLM-5.3-NVFP4 PD 分离链 - 角色3: decode 节点（方案F，跑在 6000D-2 = 174.1.60.2，8 卡）。
-# 四角色链之一，启动顺序强制: mc-master -> prefill -> [本角色] -> router。
-# 完整链编排见 deploy/PD_CHAIN.md。
-# 可执行部署脚本: experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_s1_decode.sh
-#   （launch 脚本同目录 s1_decode_launch.sh）
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - 配方 = 方案 E v5（TP8 + DFLASH + MRR12 + fa4 + window2048）+ PD decode flags；
-#     KV 池 214,336 token（比单机 E 的 243,584 少，PD 传输缓冲占显存）
-#   - 容量属性（实测判决的核心）: 池按 16k 场景定容 → 128k 仅容 1 个驻留（cc4 时 TTFT
-#     堆到 31.0s）、64k 容 3、16k 容 12（MRR12 上限）。长上下文负载该池就是瓶颈
-#   - 必须挂补丁树 /data/sglang_patch_glm53:/sgl-workspace/sglang（唯一改动
-#     python/sglang/srt/speculative/spec_info.py 接 build_dflash_family_disagg_draft_input，
-#     DFlash PD 冷启动接线）+ 装 mooncake wheel（--no-deps，launch 脚本自装）
-#   - --network host + --device /dev/infiniband + --ulimit memlock=-1，IB 设备 mlx5_0-3
-#   - DFlash accept 在 12-batch verify 下掉到 1.8（单机 EAGLE 2.84）——decode 侧并发
-#     verify 是 DFlash 的弱势区，场景二吞吐上限由此而来
-#   - 60.2 平时空闲但 GPU7 常有外部裸金属任务（main_v2.py）：动卡前核实归属，勿清
-#
-# 实测成绩: 飞书《GLM-5.3-NVFP4 双场景压测报告》方案 F 行（文档 SZUSdEqY1oRVxGxgILBcHqPJnEc）。
-# 场景二判决: "两机买 TTFT、不买吞吐"（÷2 单机等效 74-85 tok/s = A 的 75-96%，TTFT 减半）。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_pd_decode
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-ROLE=pd-decode
-NODE=174.1.60.2
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=glm53-s1-decode
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-DRAFT_MODEL_PATH=/data/hf_models/GLM-5.3-DFlash2
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=1200
-
-TP=8
-MEM_FRACTION_STATIC=0.85
-MAX_RUNNING_REQUESTS=12
-CHUNKED_PREFILL_SIZE=8192
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="MOONCAKE_MASTER=174.1.60.1:50051 MOONCAKE_PROTOCOL=rdma PYTHONUNBUFFERED=1"
-
-DOCKER_FLAGS="--gpus all --network host --ipc=host --shm-size 64g --ulimit memlock=-1 --device /dev/infiniband --restart unless-stopped"
-VOLUMES="/data/hf_models:/data/hf_models /data/flashkda_deploy/wheels:/mc_wheels:ro /data/sglang_patch_glm53:/sgl-workspace/sglang"
-
-BOOTSTRAP="pip install /mc_wheels/mooncake_transfer_engine_cuda13-0.3.12.post1-cp312-cp312-manylinux_2_28_x86_64.whl --no-deps -q && python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --disable-radix-cache --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --disable-custom-all-reduce --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --speculative-algorithm DFLASH --speculative-draft-model-path ${DRAFT_MODEL_PATH} --speculative-draft-attention-backend fa4 --speculative-draft-window-size 2048 --disaggregation-mode decode --disaggregation-transfer-backend mooncake --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0,mlx5_1,mlx5_2,mlx5_3 --host 0.0.0.0 --port ${PORT} --json-model-override-args {\"index_topk_freq\": 4}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_pd_master.env

@@ -1,29 +0,0 @@
-# GLM-5.3-NVFP4 PD 分离链 - 角色1: Mooncake 元数据服务 mc-master（方案F，双机 6000D-1 + 6000D-2）。
-# 四角色链之一，启动顺序强制: mc-master(60.1:50051) -> prefill(60.1:30000) -> decode(60.2:30000) -> router(60.2:31000)。
-# 完整链编排见 deploy/PD_CHAIN.md；可执行脚本: experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_pd_smoke_master.sh
-#
-# 关键点:
-#   - 镜像自带的 mooncake_master 二进制（/opt/sglang/bin/mooncake_master），无需 GPU
-#   --network host，监听 50051；prefill/decode 容器通过 MOONCAKE_MASTER=174.1.60.1:50051 注册
-#   - 就绪判据: ss -tln | grep :50051（脚本 sleep 3 后检查）
-#   - 拆链时必须先删本容器之外的角色再删它? 否——顺序无依赖，但生产恢复 60.1 时
-#     本容器与 glm53-pd-smoke-prefill 都要删干净、等显存排空再拉生产容器
-#
-# 实测背景（2026-09-08 方案F 双场景压测）: 13/13 干净点，质量门 6/7。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_pd_master
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang-mooncake-master
-RUNTIME=docker
-ROLE=pd-master
-NODE=174.1.60.1
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=mc-master
-PORT=50051
-
-DOCKER_FLAGS="--network host --restart unless-stopped"
-ENTRYPOINT="/opt/sglang/bin/mooncake_master"
-
-READY_CHECK="ss -tln | grep :50051"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_pd_prefill.env

@@ -1,59 +0,0 @@
-# GLM-5.3-NVFP4 PD 分离链 - 角色2: prefill 节点（方案F，跑在 6000D-1 = 174.1.60.1，8 卡）。
-# 四角色链之一，启动顺序强制: mc-master -> [本角色] -> decode -> router。
-# 完整链编排见 deploy/PD_CHAIN.md。
-# 可执行部署脚本: experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_pd_probe.sh
-#   （参数 = launch 脚本名；launch 脚本同目录 p1b_prefill_tp4pp2_cps8k_launch.sh 及其 radix 变体）
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - 拓扑 TP4 PP2（4卡/stage × 2 stage）+ DFLASH 草稿（草稿只跑 prefill 侧草稿 KV，
-#     export SGLANG_DFLASH_PD_DRAFT_KV_TRANSFER=0 = 草稿 KV 不跨机传输）
-#   - mem 0.78 低于单机方案（PD 模式下 mooncake 传输缓冲占显存）；cps 8192（非 D 方案的 16384）
-#   - 基线 launch 脚本带 --disable-radix-cache；场景一（90% 命中）实测用 radix 变体
-#     （p1b_prefill_tp4pp2_cps8k_radix_launch.sh，唯一差异 = 删掉该 flag）
-#   - 必须挂补丁树 /data/sglang_patch_glm53:/sgl-workspace/sglang（含 1 个文件改动:
-#     python/sglang/srt/speculative/spec_info.py 接 build_dflash_family_disagg_draft_input，
-#     DFlash PD 冷启动接线；无此补丁 decode 首请求 400）
-#   - 必须装 mooncake wheel /data/flashkda_deploy/wheels/mooncake_transfer_engine_cuda13-
-#     0.3.12.post1-cp312-cp312-manylinux_2_28_x86_64.whl（--no-deps，容器内 launch 脚本自装）
-#   - --network host + --device /dev/infiniband + --ulimit memlock=-1，IB 设备 mlx5_0-3
-#   - 前置硬检查: mc-master 50051 必须已监听（deploy_pd_probe.sh 自带）
-#   - 60.1 是生产机（日常跑 glm53-pp4）：拉本角色前须停生产容器，测完等显存 <2000MiB
-#     再跑 deploy_glm53_pp4.sh 恢复
-#
-# 实测成绩: 飞书《GLM-5.3-NVFP4 双场景压测报告》方案 F 行（文档 SZUSdEqY1oRVxGxgILBcHqPJnEc）。
-# 128k cc1 TTFT 3.91s = 六方案最低；跨机 prefill/decode 重叠使 RDMA 传输被完全吸收。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_pd_prefill
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-ROLE=pd-prefill
-NODE=174.1.60.1
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=glm53-pd-smoke-prefill
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-DRAFT_MODEL_PATH=/data/hf_models/GLM-5.3-DFlash2
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=1500
-
-TP=4
-PP=2
-MEM_FRACTION_STATIC=0.78
-MAX_RUNNING_REQUESTS=48
-CHUNKED_PREFILL_SIZE=8192
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="MOONCAKE_MASTER=174.1.60.1:50051 MOONCAKE_PROTOCOL=rdma SGLANG_DFLASH_PD_DRAFT_KV_TRANSFER=0 PYTHONUNBUFFERED=1"
-
-DOCKER_FLAGS="--gpus all --network host --ipc=host --shm-size 64g --ulimit memlock=-1 --device /dev/infiniband --restart unless-stopped"
-VOLUMES="/data/hf_models:/data/hf_models /data/flashkda_deploy/wheels:/mc_wheels:ro /data/sglang_patch_glm53:/sgl-workspace/sglang"
-
-BOOTSTRAP="pip install /mc_wheels/mooncake_transfer_engine_cuda13-0.3.12.post1-cp312-cp312-manylinux_2_28_x86_64.whl --no-deps -q && python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --disable-custom-all-reduce --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --speculative-algorithm DFLASH --speculative-draft-model-path ${DRAFT_MODEL_PATH} --speculative-draft-attention-backend fa4 --disaggregation-mode prefill --disaggregation-transfer-backend mooncake --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0,mlx5_1,mlx5_2,mlx5_3 --host 0.0.0.0 --port ${PORT} --json-model-override-args {\"index_topk_freq\": 4}"
-
-# 场景一（90% 命中）变体: LAUNCH_ARGS 追加 --disable-radix-cache 删除（radix 开）。
-# 基线（本 profile 口径）= radix off。

deploy/profiles/pro6000/glm53_nvfp4_pro6000_pd_router.env

@@ -1,35 +0,0 @@
-# GLM-5.3-NVFP4 PD 分离链 - 角色4: MiniLB 路由（方案F，跑在 6000D-2 = 174.1.60.2:31000）。
-# 四角色链之一，启动顺序强制: mc-master -> prefill -> decode -> [本角色]。
-# 完整链编排见 deploy/PD_CHAIN.md。
-# 可执行脚本: experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_pd_smoke_router.sh
-#
-# 关键点:
-#   - 镜像内自带 sglang_router（python3 -m sglang_router.launch_router），无需 GPU
-#   - --prefill 参数形态: URL 后跟 prefill 侧 disaggregation-bootstrap-port（28800），
-#     这是 KV 传输握手口，漏掉则路由建立后首请求挂起；--decode 只有 URL
-#   - bench 与质量门全部经 :31000/generate 打（--url http://174.1.60.2:31000/generate），
-#     即全链路含 KV transfer + DFlash 草稿
-#   - usage/accept 指标 router 转发后可能缺失: 命中率读 prefill 容器日志
-#     （docker logs glm53-pd-smoke-prefill，PP2 下日志计数 ×2 不影响比值），
-#     accept 读 decode 容器日志
-#   - 质量门脚本: 用 quality_gate_605.sh 副本 sed 's/PORT=30000/PORT=31000/' 生成
-#   - 就绪判据: ss -tln | grep :31000
-#
-# 实测背景: 全链 13/13 干净点、质量门 6/7（tool call 为 parser 配置缺口，与 B/D 同口径）。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_pd_router
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang-router
-RUNTIME=docker
-ROLE=pd-router
-NODE=174.1.60.2
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=pd-smoke-router
-PORT=31000
-
-DOCKER_FLAGS="--network host --restart unless-stopped --entrypoint python3"
-LAUNCH_ARGS="-m sglang_router.launch_router --pd-disaggregation --mini-lb --prefill http://174.1.60.1:30000 28800 --decode http://174.1.60.2:30000 --host 0.0.0.0 --port 31000"
-
-READY_CHECK="ss -tln | grep :31000"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4.env

@@ -1,52 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=2 PP=4 deployment profile (single RTX 6000D node, 8 GPUs).
-# 方案 D（2026-09-08 双场景补测；6000D-1 现役生产容器 glm53-pp4 的原样配方）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_glm53_pp4.sh
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - 脚本头注释是早期 TP4PP2 版残留（写"TP4 PP2 + IndexCache"），实际配置 TP=2 PP=4，
-#     以脚本 docker run 段为准；md5 def3c64c5dc19e1d507080e3366c3762
-#   - 串行 prefill 有效速率 6.5-6.7k tok/s 为各方案最高；KV 池 1,040,384 token
-#     （A 的 3.76 倍，≈61.6 请求驻留，超过 MRR 48——池在高并发下不构成约束）
-#   - 场景二（16k 独立输入 cc8-64）成立：cc32 起反超 TP4PP2（方案B），cc40/64 输出
-#     203.8/208.2 tok/s；TTFT p50 五档全档低于 B
-#   - 场景一（90% 命中长上下文）8/8 全败：TP2 长上下文每卡 KV 读量翻倍 + PP4 低并发
-#     流水空泡，单请求 decode 仅 16-19 tok/s。生产态 radix off 前缀命中恒 0，实际表现
-#     比报告 D 行（radix-on 最好情况）更差——长上下文/共享前缀负载勿用
-#   - DSA 实测：上下文长度不影响 TPOT（52.8ms 恒定），并发才是驱动
-#   - 上生产须补 --tool-call-parser glm47 与 --reasoning-parser glm45
-#   - bench 口径：场景二 cc40/64 用 nreq 80/128（与其他方案 40/64 单轮满波不同，
-#     已在报告表注声明）
-#
-# 实测成绩：飞书《GLM-5.3-NVFP4 双场景压测报告》方案 D 行（文档 SZUSdEqY1oRVxGxgILBcHqPJnEc）。
-# 质量：GSM8K×5 + 中文推理通过（6/7，tool call 为 parser 配置缺口）。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_sglang_tp2pp4
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=glm53-pp4
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=600
-CONTAINER_PYTHON=python3
-
-TP=2
-PP=4
-MEM_FRACTION_STATIC=0.85
-MAX_RUNNING_REQUESTS=48
-CHUNKED_PREFILL_SIZE=16384
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1"
-
-DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models"
-
-BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --disable-radix-cache --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --disable-custom-all-reduce --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --host 0.0.0.0 --port ${PORT} --json-model-override-args {\"index_topk_freq\": 4}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4_hicache.env

@@ -1,44 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=2 PP=4 + hicache deployment profile (single RTX 6000D node, 8 GPUs).
-# 2026-09-09 128k 低并发容量扩容实验优胜配置（60.8 现役；60.5 交付 deploy_glm53_605_v2.sh 待执行）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_128k_capacity_topology/scripts/deploy_glm53_605_v2.sh
-#
-# 与方案 D（glm53_nvfp4_pro6000_sglang_tp2pp4.env）的关键差异（勿混淆）：
-#   - radix/hicache 保持开启（60.5 真实流量命中率 90%+，关 radix 不可接受）；D 为 16k 独立输入场景关了 radix
-#   - cu13 镜像 + 9 补丁只读挂载（de-GLOO request_receiver / decode_cuda_graph_runner_fix 等 r37 栈遗产，
-#     nomtp 下 spec 相关补丁为惰性，de-GLOO 为 PP 通用修复）；D 用旧镜像 20260828 无挂载
-#   - --reasoning-parser glm45 --tool-call-parser glm47 齐备（质量门 7/7）；D 当时 6/7
-#   - --context-length 1048576（模型原生 1M；D 未设）；chunk 8192（D 16384）；MRR 16（D 48）
-#   - 实测（60.8，i128k/o512 冷缓存）：KV 池 909,632 token（A 的 3.29×）、并发上限 c6、
-#     单条上限 ~909k（900k 实跑通过）、c4 输入/输出 5,907/23.1 tok/s、TTFT p50 43.4s、
-#     512k 单条 TTFT 88.4s、90% 命中 c4 输入 17,625 tok/s
-#   - memfrac 0.85 为验证档；PP0 stage 空闲 22GB 提示 0.88 有余量（未验证，改动须重跑质量门+容量冒烟）
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_128k_capacity_topology
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729
-CONTAINER_NAME=glm53-nvfp4
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=1800
-CONTAINER_PYTHON=python3
-
-TP=2
-PP=4
-MEM_FRACTION_STATIC=0.85
-MAX_RUNNING_REQUESTS=16
-CHUNKED_PREFILL_SIZE=8192
-CONTEXT_LENGTH=1048576
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 SGLANG_PP_DEGLOO=1 SGLANG_PP_SPEC_FORCE_EAGER_DRAFT=1 SGLANG_PP_FORCE_EAGER_VERIFY=0 SGLANG_PP_SPEC_DEBUG=0"
-
-DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host --cap-add SYS_PTRACE -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models + 9 patch ro-mounts (full list in scripts/deploy_glm53_605_v2.sh; files live in /root + /root/sglang_patch2 on the host)"
-
-BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --reasoning-parser glm45 --tool-call-parser glm47 --enable-hierarchical-cache --hicache-ratio 3 --disable-overlap-schedule --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length ${CONTEXT_LENGTH} --host 0.0.0.0 --port ${PORT}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4_latest_autotune.env

@@ -1,49 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=2 PP=4 + latest镜像 + autotune profile（方案 D 升级版）。
-# 2026-09-09 DSV4 优化点迁移实验优胜配置：五点(16k/512 cc8-64)全胜 +1.2~+3.2%，
-# A/B/A 回切确认因果成立。见 experiments/pro6000/glm53_nvfp4_dsv4_migration_bench/。
-# 已入档未执行；60.1 生产仍跑 nightly 版（glm53-pp4，见同目录 tp2pp4.env）。
-#
-# 关键点（实测，勿随意改）:
-#   - 相对 tp2pp4.env 仅两处变化：镜像 nightly-dev-20260828 → latest(d6e72886)；
-#     删 --disable-flashinfer-autotune。启动参数其余逐字相同
-#   - 镜像中性实测：latest 与 nightly 同配置五点差 ≤0.7%（换镜像无风险无收益）
-#   - autotune 增益钉在 tactic 缓存上：SGLANG_CACHE_DIR 与 flashinfer JIT 缓存
-#     必须挂宿主持久盘，否则重部署重抽签（增益消失/不可复现，DSV4 §12.6 同款教训）
-#   - 代价：available_gpu_mem 15.64→11.79 GB（tactic 缓冲 ≈3.9GB/卡），
-#     KV 池不缩（1,040,384）
-#   - PCIe-IPC AllReduce 包在同一实验中五点全降（-0.4~-3.5%）判负勿叠用：
-#     TP2 单对端 NCCL AR 同 switch P2P 已近最优
-#   - 上生产须补 --tool-call-parser glm47 与 --reasoning-parser glm45（D 系既有缺口）
-#   - 部署器：experiments/pro6000/glm53_nvfp4_dsv4_migration_bench/scripts/deploy_glm53_exp.sh
-#     （EXP_NAME=<名> EXP_IMAGE=lmsysorg/sglang:latest AUTOTUNE=1）
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_sglang_tp2pp4_latest_autotune
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:latest
-DOCKER_IMAGE_DIGEST=sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9
-CONTAINER_NAME=glm53-pp4-autotune
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=600
-CONTAINER_PYTHON=python3
-
-TP=2
-PP=4
-MEM_FRACTION_STATIC=0.85
-MAX_RUNNING_REQUESTS=48
-CHUNKED_PREFILL_SIZE=16384
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 SGLANG_CACHE_DIR=/root/.cache/sglang"
-
-DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models /data/glm53_exp/fi_jit_cache:/root/.cache/flashinfer /data/glm53_exp/sglang_cache:/root/.cache/sglang /data/glm53_exp/triton_cache:/root/.triton"
-
-BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --disable-radix-cache --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-custom-all-reduce --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --host 0.0.0.0 --port ${PORT} --json-model-override-args {\"index_topk_freq\": 4}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp4pp2_hicache.env

@@ -1,47 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=4 PP=2 + hicache deployment profile (single RTX 6000D node, 8 GPUs).
-# 2026-09-09 hit90 场景实验优胜配置（60.8 现役；60.5 交付 deploy_glm53_605_v3.sh 待执行）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/scripts/deploy_glm53_605_v3.sh
-# 60.8 启动口径：bash /root/deploy_ppmtp_r37.sh '--tp 4 --pp-size 2 --disable-overlap-schedule
-#   --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length 1048576' nomtp 8192 0.90 1 1 0 0
-#
-# 与 TP2PP4-hicache（glm53_nvfp4_pro6000_sglang_tp2pp4_hicache.env）的关键差异（勿混淆）：
-#   - tp2/pp4 → tp4/pp2，memfrac 0.85 → 0.90；其余（cu13+9 挂载、chunk 8192、MRR 16、
-#     hicache 3、ctx 1048576、双 parser）逐项同构
-#   - 实测 KV 池 647,040（fp8 KV 18.52GB/rank，PP0 初始化后剩 10.94GB）
-#   - 取胜依据（hit90=90% 命中 i128k/o512 主场景）：out cc1/2/3/4 = 28.8/47.5/63.9/76.2、
-#     cc8/16 = 106.4/125.7（vs TP2PP4：cc4 +14%/cc8 +15%/cc16 +1%）；cap cc2/cc4 = 62.9/98.5
-#     零排队；质量门 7/7
-#   - 让步项（知情选择）：并发独立 128k 文档 4 条（TP2PP4 为 6）；512k 单条 151.9s
-#     （TP2PP4 为 114.4s，PP4 单条巨请求 prefill 流水更优）；无投机解码
-#   - 判决背景：DP attention 对本模型容量负收益、DCP 对 DSA 静默算错、MTP@128k accept 2.07 判负
-#     （见 experiments/pro6000/glm53_nvfp4_hit90_dp_dcp_bench/README.md）
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_hit90_dp_dcp_bench
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729
-CONTAINER_NAME=glm53-nvfp4
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=1800
-CONTAINER_PYTHON=python3
-
-TP=4
-PP=2
-MEM_FRACTION_STATIC=0.90
-MAX_RUNNING_REQUESTS=16
-CHUNKED_PREFILL_SIZE=8192
-CONTEXT_LENGTH=1048576
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 SGLANG_PP_DEGLOO=1 SGLANG_PP_SPEC_FORCE_EAGER_DRAFT=1 SGLANG_PP_FORCE_EAGER_VERIFY=0 SGLANG_PP_SPEC_DEBUG=0"
-
-DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host --cap-add SYS_PTRACE -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models + 9 patch ro-mounts (full list in scripts/deploy_glm53_605_v3.sh; files live in /root + /root/sglang_patch2 on the host, bundle md5 6922e53439991bc13feee72f3760704f)"
-
-BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size ${PP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --reasoning-parser glm45 --tool-call-parser glm47 --enable-hierarchical-cache --hicache-ratio 3 --disable-overlap-schedule --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length ${CONTEXT_LENGTH} --host 0.0.0.0 --port ${PORT}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp4pp2_indexcache.env

@@ -1,43 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=4 PP=2 + IndexCache deployment profile (single RTX 6000D node).
-# 场景二（16k 独立输入、cc8-32、prefill 主导）最优配置：吞吐 +41~79%、cc32 TTFT 减半
-# （对比 TP8+EAGLE 生产配置，2026-09-07 真实语料基线）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_glm53_optimal.sh
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - index_topk_freq=4 为模型原生默认（层轴索引复用省 75% indexer，无质量损失）
-#   - 禁投机解码：PP2 与投机框架不兼容（已实测）
-#   - 本文件为场景二形态（--disable-radix-cache，独立输入无前缀复用）；
-#     场景一 90% 命中对比须启用 radix（唯一差异：去掉 --disable-radix-cache，
-#     见 deploy_glm53_optimal_s1.sh）
-#   - mem 0.85：0.90 下 cuda graph 捕获余量不足会运行时 OOM（KV 池 569,600 = TP8 的 2.06 倍）
-#   - --disable-custom-all-reduce：TP4 over PCIe 用自定义 AR 在本栈无收益
-#   - 已知缺口：未带 --tool-call-parser glm47 --reasoning-parser glm45，质量门 6/7
-#     （tool call 失败纯属参数缺失，非模型问题）；上生产必须补 parser
-#   - 场景一（90% 命中低并发）该配置全面劣于 TP8+EAGLE（输出吞吐 −25~−65%），勿混用选型
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_sglang_tp4pp2_indexcache
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-CONTAINER_NAME=glm53-nvfp4
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=2400
-CONTAINER_PYTHON=python3
-
-TP=4
-DP=1
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1"
-
-DOCKER_FLAGS="--gpus all --ipc=host --shm-size 64g --entrypoint '' -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models:ro"
-
-BOOTSTRAP="exec python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --pp-size 2 --mem-fraction-static 0.85 --max-running-requests 48 --disable-radix-cache --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --disable-custom-all-reduce --chunked-prefill-size 16384 --json-model-override-args '{\"index_topk_freq\": 4}' --host 0.0.0.0 --port ${PORT}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp8dflash2.env

@@ -1,52 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=8 + DFlash2 speculative decoding profile (single RTX 6000D node, 8 GPUs).
-# 方案 E（2026-09-08 场景一补测，6000D-2）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_glm53_tp8_dflash2.sh
-#   （v5 终版底稿，含四轮 OOM 战役完整教训注释；md5 5bf2b47c9349e5855b963e571e35c096）
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - v5 配方核心：MRR 48→12（verify CUDA graph 4.04→0.83GB，真正起作用的杠杆）+
-#     --speculative-draft-window-size 2048 + mem0.85/cps8192。四轮 OOM 根因与推导见脚本头注释
-#   - DFLASH block-diffusion 草稿 7 tokens/步，draft 权重 GLM-5.3-DFlash2，fa4 draft
-#     attention（fa4 会把 draft KV 强制 bf16，fp8 需换 flashinfer/triton 后端，仅省 0.35GB 未用）
-#   - KV 池 243,584 token；kv fp8_e4m3 由模型配置自动带出（无需显式 flag）
-#   - 底稿（本 profile LAUNCH_ARGS）radix/AR 均为禁用；场景一实测变体共四处 delta：
-#     ① 去 --disable-radix-cache（90% 命中前提）② 去 --disable-custom-all-reduce
-#     （v1 CAR 与方案 A 一致开启）③ 加 --context-length 270336 ④ 加 --reasoning-parser
-#     glm45 --tool-call-parser glm47（质量门 7/7 的前提）
-#   - 判决：场景一 8 点全部低于方案 C、7 点低于 A——DFlash accept 低于 EAGLE（同语料
-#     2.53 vs 2.84）而每步墙钟相当，劣势全在接受率。投机栈选型维持 EAGLE3，勿用
-#     DFlash2 替换（性能问题非质量问题）
-#
-# 实测成绩：飞书《GLM-5.3-NVFP4 双场景压测报告》方案 E 行（文档 SZUSdEqY1oRVxGxgILBcHqPJnEc）。
-# 质量：变体配置下质量门 7/7（GSM8K×5、中文推理、tool call 全过），DFlash 草稿无质量损失。
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_sglang_tp8dflash2
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-DOCKER_IMAGE_DIGEST=sha256:28e0d26073161e49ca56eba808d264a4223804a212020f1dfe1b2405b9f8a399
-CONTAINER_NAME=glm53-tp8-dflash2
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-DRAFT_MODEL_PATH=/data/hf_models/GLM-5.3-DFlash2
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=900
-CONTAINER_PYTHON=python3
-
-TP=8
-MEM_FRACTION_STATIC=0.85
-MAX_RUNNING_REQUESTS=12
-CHUNKED_PREFILL_SIZE=8192
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1"
-
-DOCKER_FLAGS="--gpus all --shm-size 64g --ipc=host -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models"
-
-BOOTSTRAP="python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --mem-fraction-static ${MEM_FRACTION_STATIC} --max-running-requests ${MAX_RUNNING_REQUESTS} --disable-radix-cache --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --disable-custom-all-reduce --chunked-prefill-size ${CHUNKED_PREFILL_SIZE} --speculative-algorithm DFLASH --speculative-draft-model-path ${DRAFT_MODEL_PATH} --speculative-draft-attention-backend fa4 --speculative-draft-window-size 2048 --host 0.0.0.0 --port ${PORT} --json-model-override-args {\"index_topk_freq\": 4}"

deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp8eagle.env

@@ -1,44 +0,0 @@
-# GLM-5.3-NVFP4 SGLang TP=8 + EAGLE deployment profile (single RTX 6000D node, 8 GPUs).
-# 生产标准配置（174.1.60.5 团队自用在役，2026-09-07 定稿）。
-# 可执行部署脚本：experiments/pro6000/glm53_nvfp4_pro6000d_sglang_dual_scenario_bench/scripts/deploy_glm53_605.sh
-#
-# 关键点（实测踩坑，勿随意改）:
-#   - SM120 必需三项：--disable-shared-experts-fusion --moe-runner-backend
-#     flashinfer_cutlass --disable-flashinfer-autotune
-#   - EAGLE 4/1/5 为 cc1-2 均衡点（3/1/4、5/1/6 在 ±10% 噪声内；树式 topk>1 在 DSA 上不可用；
-#     EAGLE3 本 nightly 不支持 GLM-5.3）；draft 模型自动从主权重加载
-#   - KV fp8_e4m3 + hicache-ratio 3 是长上下文 90% 前缀命中场景的容量前提（KV 池 276,864 token）
-#   - chunk 8192：16384 已 A/B 证伪（MoE 工作区 OOM 风险，且 AR 带宽受限无次数红利）
-#   - 容器入口必须 python3 -m sglang.launch_server（镜像 entrypoint 无 shebang）
-#   - 重新部署前必须等显存排空（docker rm -f 异步滞留数分钟，不等会把新 KV 池压小）
-#   - 场景二高并发变体（16k 独立输入 cc8-32）：仅改 --max-running-requests 32 +
-#     --cuda-graph-max-bs-decode 16 --cuda-graph-bs-decode 4 8 12 16（KV 池 276,864 ÷
-#     16,896/req = 16.4 驻留上限，decode 批自然 ≤16，图覆盖到 bs16 即可，bs24/32 纯耗显存）
-#   - 上线前后过质量门 quality_gate_605.sh（PASS=7/7）
-
-PLATFORM=pro6000
-EXPERIMENT=glm53_nvfp4_pro6000_sglang_tp8eagle
-MODEL_NAME=GLM-5.3-NVFP4
-ENGINE=sglang
-RUNTIME=docker
-DOCKER_IMAGE=lmsysorg/sglang:nightly-dev-20260828-daf63171
-CONTAINER_NAME=glm53-nvfp4
-MODEL_PATH=/data/hf_models/GLM-5.3-NVFP4
-SERVED_MODEL_NAME=/data/hf_models/GLM-5.3-NVFP4
-PORT=30000
-HEALTH_PATH=/health
-HEALTH_WAIT_S=2400
-CONTAINER_PYTHON=python3
-
-TP=8
-DP=1
-
-DEVICE_VARS="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
-ENGINE_ENV="PYTHONUNBUFFERED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1"
-
-DOCKER_FLAGS="--gpus all --ipc=host --shm-size 64g --entrypoint '' -p ${PORT}:${PORT}"
-VOLUMES="/data/hf_models:/data/hf_models:ro"
-
-BOOTSTRAP="exec python3 -m sglang.launch_server ${LAUNCH_ARGS}"
-
-LAUNCH_ARGS="--model-path ${MODEL_PATH} --tp-size ${TP} --mem-fraction-static 0.90 --max-running-requests 16 --chunked-prefill-size 8192 --max-prefill-tokens 16384 --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune --speculative-algorithm EAGLE --speculative-num-steps 4 --speculative-eagle-topk 1 --speculative-num-draft-tokens 5 --kv-cache-dtype fp8_e4m3 --enable-hierarchical-cache --hicache-ratio 3 --cuda-graph-max-bs-decode 8 --cuda-graph-bs-decode 1 2 3 4 6 8 --cuda-graph-max-bs-prefill 8 --context-length 270336 --reasoning-parser glm45 --tool-call-parser glm47 --host 0.0.0.0 --port ${PORT}"

deploy/verify_profile.sh

@@ -1,91 +0,0 @@
-#!/bin/bash
-# verify_profile.sh —— 防漂移核验：运行中容器 vs 仓库 profile 声明
-# 在目标服务器上运行（需 docker 读权限，无需 GPU）。用法：
-#   bash verify_profile.sh deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4.env
-#   bash verify_profile.sh <profile.env> [container_name]   # container_name 缺省取 profile 的 CONTAINER_NAME
-# 核验项：①容器在跑 ②镜像 registry digest ③launch 参数（token 级比对）④端口监听。
-# PD 链角色的参数在挂载的 launch 脚本内（docker Args 只有 bash /smoke_launch.sh），
-# 第③项自动跳过并提示改用库内脚本 md5 对比（scripts/ 目录各脚本头注释有 md5）。
-set -uo pipefail
-
-PROFILE="$1"
-[ -r "$PROFILE" ] || { echo "FATAL: 无法读取 profile: $PROFILE"; exit 2; }
-# profile 是声明式清单：BOOTSTRAP 行引用 ${LAUNCH_ARGS}（定义在后），set -u 下直接
-# source 会炸；临时关 -u。source 完成后 LAUNCH_ARGS 已是全部变量展开后的实参串。
-set +u
-# shellcheck disable=SC1090
-source "$PROFILE"
-set -u
-CONT="${2:-${CONTAINER_NAME:-}}"
-
-fail=0
-section() { printf '\n== %s ==\n' "$1"; }
-
-section "容器状态"
-if [ -z "$CONT" ]; then echo "FATAL: profile 未定义 CONTAINER_NAME 且未显式传入"; exit 2; fi
-if docker ps --format '{{.Names}}' | grep -qx "$CONT"; then
-  echo "OK  $CONT 在跑"
-else
-  echo "FAIL  $CONT 未在运行"; docker ps -a --filter "name=$CONT" --format '{{.Names}} {{.Status}}' | head -3
-  exit 1
-fi
-
-section "镜像 digest"
-if [ -n "${DOCKER_IMAGE_DIGEST:-}" ]; then
-  # RepoDigests 在 image 对象上：container 先取 image ID，再 image inspect
-  img_id=$(docker inspect --format '{{.Image}}' "$CONT" 2>/dev/null)
-  repo_digests=$(docker image inspect --format '{{join .RepoDigests "\n"}}' "$img_id" 2>/dev/null)
-  digest_hex="${DOCKER_IMAGE_DIGEST#sha256:}"
-  if echo "$repo_digests" | grep -q "$digest_hex"; then
-    echo "OK  registry digest 一致: ${DOCKER_IMAGE_DIGEST}"
-  elif [ "$img_id" = "sha256:${digest_hex}" ]; then
-    echo "OK  image ID 一致: ${DOCKER_IMAGE_DIGEST}"
-  else
-    echo "FAIL  digest 不一致"
-    echo "  profile: ${DOCKER_IMAGE_DIGEST}"
-    echo "  实际 image ID: ${img_id}"
-    echo "  实际 RepoDigests: $(echo "$repo_digests" | head -2 | tr '\n' ' ')"
-    fail=1
-  fi
-else
-  echo "SKIP  profile 未定义 DOCKER_IMAGE_DIGEST"
-fi
-
-section "启动参数"
-args=$(docker inspect --format '{{join .Args " "}}' "$CONT" 2>/dev/null)
-if echo "$args" | grep -q launch_server; then
-  actual=$(echo "$args" | sed 's/.*launch_server //')
-  expected="${LAUNCH_ARGS:-}"
-  tr ' ' '\n' <<<"$expected" | sed '/^$/d' | sort > /tmp/vp_expected.$$
-  tr ' ' '\n' <<<"$actual"   | sed '/^$/d' | sort > /tmp/vp_actual.$$
-  if diff -q /tmp/vp_expected.$$ /tmp/vp_actual.$$ >/dev/null; then
-    echo "OK  参数一致（token 比对，共 $(wc -l < /tmp/vp_expected.$$) 项）"
-  else
-    echo "FAIL  参数有漂移（< profile 声明 / > 容器实际）:"
-    diff /tmp/vp_expected.$$ /tmp/vp_actual.$$ | sed 's/^/    /'
-    fail=1
-  fi
-  rm -f /tmp/vp_expected.$$ /tmp/vp_actual.$$
-else
-  echo "SKIP  docker Args 为 '$args' —— 参数在挂载的 launch 脚本内"
-  echo "      改用库内脚本 md5 对比：experiments/.../scripts/ 各脚本头注释"
-fi
-
-section "端口 ${PORT:-?}"
-if [ -n "${PORT:-}" ]; then
-  if ss -tln | grep -q ":${PORT} "; then
-    echo "OK  :${PORT} 在监听"
-  else
-    echo "FAIL  :${PORT} 未监听"; fail=1
-  fi
-  if [ -n "${HEALTH_PATH:-}" ]; then
-    code=$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:${PORT}${HEALTH_PATH}" 2>/dev/null)
-    [ "$code" = "200" ] && echo "OK  health ${HEALTH_PATH} -> 200" || { echo "FAIL  health ${HEALTH_PATH} -> ${code}"; fail=1; }
-  fi
-else
-  echo "SKIP  profile 未定义 PORT"
-fi
-
-echo
-[ $fail -eq 0 ] && echo "VERDICT: PASS" || echo "VERDICT: DRIFT DETECTED"
-exit $fail

experiments/p800/glm52_w4a8_sglang_tp8_single_node/README.md

@@ -1,11 +0,0 @@
-# GLM-5.2 W4A8 P800 single-node smoke benchmark
-
-- Host: `sskj-P800-4` (`gpu051`)
-- Image: `iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746`
-- Model: `/data2/GLM-5.2-w4a8`
-- Parallelism: TP=8, EP=8, DP=1
-- Quantization: `w4a8_int4`; KV cache: `int8`
-- Dataset: `/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json`
-- Benchmark: ShareGPT, 64 prompts, concurrency 8, output length 128, request rate unlimited
-
-Run `bash start_glm52_w4a8.sh`, wait for `http://127.0.0.1:30000/health`, then run `bash run_bench.sh`.

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-1555-sharegpt-c8-o128/container_inspect.json

@@ -1,231 +0,0 @@
-[
-    {
-        "Id": "5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180",
-        "Created": "2026-08-17T15:52:21.087868771Z",
-        "Path": "bash",
-        "Args": [
-            "-lc",
-            "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-        ],
-        "State": {
-            "Status": "running",
-            "Running": true,
-            "Paused": false,
-            "Restarting": false,
-            "OOMKilled": false,
-            "Dead": false,
-            "Pid": 137722,
-            "ExitCode": 0,
-            "Error": "",
-            "StartedAt": "2026-08-17T15:52:21.126324465Z",
-            "FinishedAt": "0001-01-01T00:00:00Z"
-        },
-        "Image": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-        "ResolvConfPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/resolv.conf",
-        "HostnamePath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hostname",
-        "HostsPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hosts",
-        "LogPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180-json.log",
-        "Name": "/glm52-w4a8-tp8",
-        "RestartCount": 0,
-        "Driver": "overlayfs",
-        "Platform": "linux",
-        "MountLabel": "",
-        "ProcessLabel": "",
-        "AppArmorProfile": "unconfined",
-        "ExecIDs": null,
-        "HostConfig": {
-            "Binds": [
-                "/data2/GLM-5.2-w4a8:/models:ro",
-                "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro",
-                "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke:/workspace/run"
-            ],
-            "ContainerIDFile": "",
-            "LogConfig": {
-                "Type": "json-file",
-                "Config": {}
-            },
-            "NetworkMode": "host",
-            "PortBindings": {},
-            "RestartPolicy": {
-                "Name": "no",
-                "MaximumRetryCount": 0
-            },
-            "AutoRemove": false,
-            "VolumeDriver": "",
-            "VolumesFrom": null,
-            "ConsoleSize": [
-                0,
-                0
-            ],
-            "CapAdd": null,
-            "CapDrop": null,
-            "CgroupnsMode": "private",
-            "Dns": null,
-            "DnsOptions": [],
-            "DnsSearch": [],
-            "ExtraHosts": null,
-            "GroupAdd": null,
-            "IpcMode": "host",
-            "Cgroup": "",
-            "Links": null,
-            "OomScoreAdj": 0,
-            "PidMode": "",
-            "Privileged": true,
-            "PublishAllPorts": false,
-            "ReadonlyRootfs": false,
-            "SecurityOpt": [
-                "seccomp=unconfined",
-                "label=disable"
-            ],
-            "UTSMode": "host",
-            "UsernsMode": "",
-            "ShmSize": 274877906944,
-            "Runtime": "runc",
-            "Isolation": "",
-            "CpuShares": 0,
-            "Memory": 0,
-            "NanoCpus": 0,
-            "CgroupParent": "",
-            "BlkioWeight": 0,
-            "BlkioWeightDevice": [],
-            "BlkioDeviceReadBps": [],
-            "BlkioDeviceWriteBps": [],
-            "BlkioDeviceReadIOps": [],
-            "BlkioDeviceWriteIOps": [],
-            "CpuPeriod": 0,
-            "CpuQuota": 0,
-            "CpuRealtimePeriod": 0,
-            "CpuRealtimeRuntime": 0,
-            "CpusetCpus": "",
-            "CpusetMems": "",
-            "Devices": [],
-            "DeviceCgroupRules": null,
-            "DeviceRequests": null,
-            "MemoryReservation": 0,
-            "MemorySwap": 0,
-            "MemorySwappiness": null,
-            "OomKillDisable": null,
-            "PidsLimit": null,
-            "Ulimits": [
-                {
-                    "Name": "core",
-                    "Hard": -1,
-                    "Soft": -1
-                }
-            ],
-            "CpuCount": 0,
-            "CpuPercent": 0,
-            "IOMaximumIOps": 0,
-            "IOMaximumBandwidth": 0,
-            "MaskedPaths": null,
-            "ReadonlyPaths": null
-        },
-        "Storage": {
-            "RootFS": {
-                "Snapshot": {
-                    "Name": "overlayfs"
-                }
-            }
-        },
-        "Mounts": [
-            {
-                "Type": "bind",
-                "Source": "/data2/GLM-5.2-w4a8",
-                "Destination": "/models",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Destination": "/workspace/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke",
-                "Destination": "/workspace/run",
-                "Mode": "",
-                "RW": true,
-                "Propagation": "rprivate"
-            }
-        ],
-        "Config": {
-            "Hostname": "gpu051",
-            "Domainname": "",
-            "User": "",
-            "AttachStdin": false,
-            "AttachStdout": false,
-            "AttachStderr": false,
-            "Tty": false,
-            "OpenStdin": false,
-            "StdinOnce": false,
-            "Env": [
-                "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "CUDA_DISABLE_PRINTF=1",
-                "GLOO_SOCKET_TIMEOUT=60000",
-                "PATH=/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/torch_xmlir/xre/bin/:/root/miniconda/bin:/root/.cargo/bin:/usr/local/go/bin:/usr/local/cuda-11.7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "TZ=Asia/Shanghai",
-                "LANG=zh_CN.UTF-8"
-            ],
-            "Cmd": [
-                "bash",
-                "-lc",
-                "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-            ],
-            "Image": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-            "Volumes": null,
-            "WorkingDir": "/workspace/run",
-            "Entrypoint": null,
-            "Labels": {
-                "autofix_time": "2025-12-03T10:19:26",
-                "description": "infer_ubuntu2204_dev runtime + miniconda",
-                "maintainer": "wenli <wenli01@baidu.com>",
-                "org.opencontainers.image.ref.name": "ubuntu",
-                "org.opencontainers.image.version": "22.04",
-                "version": "infer_ubuntu2204_dev:miniconda_0.1"
-            }
-        },
-        "NetworkSettings": {
-            "SandboxID": "81c79e65bbac2dcf756cbff44c4a0ba01bd8be20cc29b1b7907701dfb0a30619",
-            "SandboxKey": "/var/run/docker/netns/default",
-            "Ports": {},
-            "Networks": {
-                "host": {
-                    "IPAMConfig": null,
-                    "Links": null,
-                    "Aliases": null,
-                    "DriverOpts": null,
-                    "GwPriority": 0,
-                    "NetworkID": "0e3fdd7bc7639e6dc911f6f09344de480a35fa47ce2e83ff21a8160e319f48ac",
-                    "EndpointID": "35fe2742654dbbedd58999151fd8a77234c95ea9204ebd61074cd27d27d85871",
-                    "Gateway": "",
-                    "IPAddress": "",
-                    "MacAddress": "",
-                    "IPPrefixLen": 0,
-                    "IPv6Gateway": "",
-                    "GlobalIPv6Address": "",
-                    "GlobalIPv6PrefixLen": 0,
-                    "DNSNames": null
-                }
-            }
-        },
-        "ImageManifestDescriptor": {
-            "mediaType": "application/vnd.oci.image.manifest.v1+json",
-            "digest": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-            "size": 3632,
-            "annotations": {
-                "io.containerd.image.name": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-                "org.opencontainers.image.ref.name": "20260715_5746"
-            },
-            "platform": {
-                "architecture": "amd64",
-                "os": "linux"
-            }
-        }
-    }
-]

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-1555-sharegpt-c8-o128/xpu_after_bench.txt

@@ -1,111 +0,0 @@
-Mon Aug 17 15:56:53 2026       
-+-----------------------------------------------------------------------------+
-| XPU-SMI               Driver Version: 5.19.0.0     XPU-RT Version: 5.19.0   |
-|-------------------------------+----------------------+----------------------+
-| XPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
-| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | XPU-Util  Compute M. |
-|                               |             L3-Usage |            SR-IOV M. |
-|===============================+======================+======================|
-|   0  P800 OAM           N/A   | 00000000:03:00.0 N/A |                    0 |
-| N/A   42C  N/A     89W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   1  P800 OAM           N/A   | 00000000:05:00.0 N/A |                    0 |
-| N/A   46C  N/A     90W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   2  P800 OAM           N/A   | 00000000:63:00.0 N/A |                    0 |
-| N/A   41C  N/A     90W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   3  P800 OAM           N/A   | 00000000:65:00.0 N/A |                    0 |
-| N/A   47C  N/A     90W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   4  P800 OAM           N/A   | 00000000:83:00.0 N/A |                    0 |
-| N/A   41C  N/A     86W / 400W |  88310MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   5  P800 OAM           N/A   | 00000000:85:00.0 N/A |                    0 |
-| N/A   47C  N/A     88W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   6  P800 OAM           N/A   | 00000000:A3:00.0 N/A |                    0 |
-| N/A   43C  N/A     89W / 400W |  87914MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   7  P800 OAM           N/A   | 00000000:A5:00.0 N/A |                    0 |
-| N/A   50C  N/A     90W / 400W |  88292MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-                                                                               
-+-----------------------------------------------------------------------------+
-| Processes:                                                                  |
-|  XPU   XI   CI        PID   Type   Process name                  XPU Memory |
-|        ID   ID                                                   Usage      |
-|=============================================================================|
-|    0   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    0   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    0   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    0   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3       87826MiB |
-|    0   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    0   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    0   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    0   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    1   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    1   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    1   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2       87826MiB |
-|    1   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    1   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    1   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    1   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    1   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    2   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0       87826MiB |
-|    2   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    2   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    2   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    2   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    2   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    2   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    2   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    3   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    3   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1       87826MiB |
-|    3   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    3   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    3   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    3   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    3   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    3   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    4   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    4   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    4   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    4   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    4   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    4   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    4   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    4   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7       88178MiB |
-|    5   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    5   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    5   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    5   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    5   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    5   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    5   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6       87826MiB |
-|    5   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    6   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    6   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    6   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    6   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    6   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4       87782MiB |
-|    6   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    6   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    6   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    7   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    7   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    7   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    7   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    7   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    7   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5       88160MiB |
-|    7   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    7   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-+-----------------------------------------------------------------------------+

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-1555-sharegpt-c8-o128/xpu_before_bench.txt

@@ -1,111 +0,0 @@
-Mon Aug 17 15:55:17 2026       
-+-----------------------------------------------------------------------------+
-| XPU-SMI               Driver Version: 5.19.0.0     XPU-RT Version: 5.19.0   |
-|-------------------------------+----------------------+----------------------+
-| XPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
-| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | XPU-Util  Compute M. |
-|                               |             L3-Usage |            SR-IOV M. |
-|===============================+======================+======================|
-|   0  P800 OAM           N/A   | 00000000:03:00.0 N/A |                    0 |
-| N/A   39C  N/A     88W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   1  P800 OAM           N/A   | 00000000:05:00.0 N/A |                    0 |
-| N/A   43C  N/A     89W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   2  P800 OAM           N/A   | 00000000:63:00.0 N/A |                    0 |
-| N/A   38C  N/A     90W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   3  P800 OAM           N/A   | 00000000:65:00.0 N/A |                    0 |
-| N/A   43C  N/A     93W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   4  P800 OAM           N/A   | 00000000:83:00.0 N/A |                    0 |
-| N/A   39C  N/A     86W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   5  P800 OAM           N/A   | 00000000:85:00.0 N/A |                    0 |
-| N/A   43C  N/A     91W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   6  P800 OAM           N/A   | 00000000:A3:00.0 N/A |                    0 |
-| N/A   41C  N/A     89W / 400W |  85774MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   7  P800 OAM           N/A   | 00000000:A5:00.0 N/A |                    0 |
-| N/A   47C  N/A     89W / 400W |  85772MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-                                                                               
-+-----------------------------------------------------------------------------+
-| Processes:                                                                  |
-|  XPU   XI   CI        PID   Type   Process name                  XPU Memory |
-|        ID   ID                                                   Usage      |
-|=============================================================================|
-|    0   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    0   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    0   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    0   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3       85642MiB |
-|    0   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    0   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    0   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    0   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    1   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    1   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    1   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2       85642MiB |
-|    1   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    1   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    1   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    1   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    1   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    2   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0       85642MiB |
-|    2   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    2   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    2   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    2   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    2   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    2   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    2   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    3   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    3   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1       85642MiB |
-|    3   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    3   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    3   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    3   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    3   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    3   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    4   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    4   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    4   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    4   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    4   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    4   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    4   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    4   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7       85642MiB |
-|    5   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    5   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    5   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    5   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    5   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    5   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    5   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6       85642MiB |
-|    5   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    6   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    6   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    6   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    6   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    6   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4       85642MiB |
-|    6   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    6   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    6   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    7   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    7   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    7   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    7   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    7   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    7   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5       85640MiB |
-|    7   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    7   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-+-----------------------------------------------------------------------------+

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-sharegpt-c1-c32-sweep/container_inspect.json

@@ -1,231 +0,0 @@
-[
-    {
-        "Id": "5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180",
-        "Created": "2026-08-17T15:52:21.087868771Z",
-        "Path": "bash",
-        "Args": [
-            "-lc",
-            "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-        ],
-        "State": {
-            "Status": "running",
-            "Running": true,
-            "Paused": false,
-            "Restarting": false,
-            "OOMKilled": false,
-            "Dead": false,
-            "Pid": 137722,
-            "ExitCode": 0,
-            "Error": "",
-            "StartedAt": "2026-08-17T15:52:21.126324465Z",
-            "FinishedAt": "0001-01-01T00:00:00Z"
-        },
-        "Image": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-        "ResolvConfPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/resolv.conf",
-        "HostnamePath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hostname",
-        "HostsPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hosts",
-        "LogPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180-json.log",
-        "Name": "/glm52-w4a8-tp8",
-        "RestartCount": 0,
-        "Driver": "overlayfs",
-        "Platform": "linux",
-        "MountLabel": "",
-        "ProcessLabel": "",
-        "AppArmorProfile": "unconfined",
-        "ExecIDs": null,
-        "HostConfig": {
-            "Binds": [
-                "/data2/GLM-5.2-w4a8:/models:ro",
-                "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro",
-                "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke:/workspace/run"
-            ],
-            "ContainerIDFile": "",
-            "LogConfig": {
-                "Type": "json-file",
-                "Config": {}
-            },
-            "NetworkMode": "host",
-            "PortBindings": {},
-            "RestartPolicy": {
-                "Name": "no",
-                "MaximumRetryCount": 0
-            },
-            "AutoRemove": false,
-            "VolumeDriver": "",
-            "VolumesFrom": null,
-            "ConsoleSize": [
-                0,
-                0
-            ],
-            "CapAdd": null,
-            "CapDrop": null,
-            "CgroupnsMode": "private",
-            "Dns": null,
-            "DnsOptions": [],
-            "DnsSearch": [],
-            "ExtraHosts": null,
-            "GroupAdd": null,
-            "IpcMode": "host",
-            "Cgroup": "",
-            "Links": null,
-            "OomScoreAdj": 0,
-            "PidMode": "",
-            "Privileged": true,
-            "PublishAllPorts": false,
-            "ReadonlyRootfs": false,
-            "SecurityOpt": [
-                "seccomp=unconfined",
-                "label=disable"
-            ],
-            "UTSMode": "host",
-            "UsernsMode": "",
-            "ShmSize": 274877906944,
-            "Runtime": "runc",
-            "Isolation": "",
-            "CpuShares": 0,
-            "Memory": 0,
-            "NanoCpus": 0,
-            "CgroupParent": "",
-            "BlkioWeight": 0,
-            "BlkioWeightDevice": [],
-            "BlkioDeviceReadBps": [],
-            "BlkioDeviceWriteBps": [],
-            "BlkioDeviceReadIOps": [],
-            "BlkioDeviceWriteIOps": [],
-            "CpuPeriod": 0,
-            "CpuQuota": 0,
-            "CpuRealtimePeriod": 0,
-            "CpuRealtimeRuntime": 0,
-            "CpusetCpus": "",
-            "CpusetMems": "",
-            "Devices": [],
-            "DeviceCgroupRules": null,
-            "DeviceRequests": null,
-            "MemoryReservation": 0,
-            "MemorySwap": 0,
-            "MemorySwappiness": null,
-            "OomKillDisable": null,
-            "PidsLimit": null,
-            "Ulimits": [
-                {
-                    "Name": "core",
-                    "Hard": -1,
-                    "Soft": -1
-                }
-            ],
-            "CpuCount": 0,
-            "CpuPercent": 0,
-            "IOMaximumIOps": 0,
-            "IOMaximumBandwidth": 0,
-            "MaskedPaths": null,
-            "ReadonlyPaths": null
-        },
-        "Storage": {
-            "RootFS": {
-                "Snapshot": {
-                    "Name": "overlayfs"
-                }
-            }
-        },
-        "Mounts": [
-            {
-                "Type": "bind",
-                "Source": "/data2/GLM-5.2-w4a8",
-                "Destination": "/models",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Destination": "/workspace/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke",
-                "Destination": "/workspace/run",
-                "Mode": "",
-                "RW": true,
-                "Propagation": "rprivate"
-            }
-        ],
-        "Config": {
-            "Hostname": "gpu051",
-            "Domainname": "",
-            "User": "",
-            "AttachStdin": false,
-            "AttachStdout": false,
-            "AttachStderr": false,
-            "Tty": false,
-            "OpenStdin": false,
-            "StdinOnce": false,
-            "Env": [
-                "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "CUDA_DISABLE_PRINTF=1",
-                "GLOO_SOCKET_TIMEOUT=60000",
-                "PATH=/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/torch_xmlir/xre/bin/:/root/miniconda/bin:/root/.cargo/bin:/usr/local/go/bin:/usr/local/cuda-11.7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "TZ=Asia/Shanghai",
-                "LANG=zh_CN.UTF-8"
-            ],
-            "Cmd": [
-                "bash",
-                "-lc",
-                "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-            ],
-            "Image": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-            "Volumes": null,
-            "WorkingDir": "/workspace/run",
-            "Entrypoint": null,
-            "Labels": {
-                "autofix_time": "2025-12-03T10:19:26",
-                "description": "infer_ubuntu2204_dev runtime + miniconda",
-                "maintainer": "wenli <wenli01@baidu.com>",
-                "org.opencontainers.image.ref.name": "ubuntu",
-                "org.opencontainers.image.version": "22.04",
-                "version": "infer_ubuntu2204_dev:miniconda_0.1"
-            }
-        },
-        "NetworkSettings": {
-            "SandboxID": "81c79e65bbac2dcf756cbff44c4a0ba01bd8be20cc29b1b7907701dfb0a30619",
-            "SandboxKey": "/var/run/docker/netns/default",
-            "Ports": {},
-            "Networks": {
-                "host": {
-                    "IPAMConfig": null,
-                    "Links": null,
-                    "Aliases": null,
-                    "DriverOpts": null,
-                    "GwPriority": 0,
-                    "NetworkID": "0e3fdd7bc7639e6dc911f6f09344de480a35fa47ce2e83ff21a8160e319f48ac",
-                    "EndpointID": "35fe2742654dbbedd58999151fd8a77234c95ea9204ebd61074cd27d27d85871",
-                    "Gateway": "",
-                    "IPAddress": "",
-                    "MacAddress": "",
-                    "IPPrefixLen": 0,
-                    "IPv6Gateway": "",
-                    "GlobalIPv6Address": "",
-                    "GlobalIPv6PrefixLen": 0,
-                    "DNSNames": null
-                }
-            }
-        },
-        "ImageManifestDescriptor": {
-            "mediaType": "application/vnd.oci.image.manifest.v1+json",
-            "digest": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-            "size": 3632,
-            "annotations": {
-                "io.containerd.image.name": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-                "org.opencontainers.image.ref.name": "20260715_5746"
-            },
-            "platform": {
-                "architecture": "amd64",
-                "os": "linux"
-            }
-        }
-    }
-]

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-sharegpt-c1-c32-sweep/xpu_after_sweep.txt

@@ -1,111 +0,0 @@
-Mon Aug 17 16:11:51 2026       
-+-----------------------------------------------------------------------------+
-| XPU-SMI               Driver Version: 5.19.0.0     XPU-RT Version: 5.19.0   |
-|-------------------------------+----------------------+----------------------+
-| XPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
-| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | XPU-Util  Compute M. |
-|                               |             L3-Usage |            SR-IOV M. |
-|===============================+======================+======================|
-|   0  P800 OAM           N/A   | 00000000:03:00.0 N/A |                    0 |
-| N/A   41C  N/A     89W / 400W |  87970MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   1  P800 OAM           N/A   | 00000000:05:00.0 N/A |                    0 |
-| N/A   47C  N/A     90W / 400W |  87970MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   2  P800 OAM           N/A   | 00000000:63:00.0 N/A |                    0 |
-| N/A   40C  N/A     90W / 400W |  87970MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   3  P800 OAM           N/A   | 00000000:65:00.0 N/A |                    0 |
-| N/A   48C  N/A     91W / 400W |  87968MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   4  P800 OAM           N/A   | 00000000:83:00.0 N/A |                    0 |
-| N/A   41C  N/A     86W / 400W |  88318MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   5  P800 OAM           N/A   | 00000000:85:00.0 N/A |                    0 |
-| N/A   48C  N/A     88W / 400W |  87970MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   6  P800 OAM           N/A   | 00000000:A3:00.0 N/A |                    0 |
-| N/A   43C  N/A     88W / 400W |  87922MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   7  P800 OAM           N/A   | 00000000:A5:00.0 N/A |                    0 |
-| N/A   51C  N/A     90W / 400W |  88306MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-                                                                               
-+-----------------------------------------------------------------------------+
-| Processes:                                                                  |
-|  XPU   XI   CI        PID   Type   Process name                  XPU Memory |
-|        ID   ID                                                   Usage      |
-|=============================================================================|
-|    0   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    0   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    0   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    0   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3       87838MiB |
-|    0   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    0   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    0   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    0   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    1   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    1   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    1   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2       87838MiB |
-|    1   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    1   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    1   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    1   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    1   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    2   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0       87838MiB |
-|    2   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    2   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    2   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    2   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    2   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    2   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    2   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    3   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    3   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1       87836MiB |
-|    3   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    3   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    3   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    3   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    3   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    3   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    4   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    4   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    4   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    4   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    4   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    4   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    4   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    4   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7       88186MiB |
-|    5   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    5   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    5   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    5   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    5   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    5   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    5   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6       87838MiB |
-|    5   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    6   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    6   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    6   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    6   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    6   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4       87790MiB |
-|    6   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    6   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    6   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    7   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    7   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    7   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    7   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    7   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    7   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5       88174MiB |
-|    7   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    7   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-+-----------------------------------------------------------------------------+

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260817-sharegpt-c1-c32-sweep/xpu_before_sweep.txt

@@ -1,111 +0,0 @@
-Mon Aug 17 16:00:14 2026       
-+-----------------------------------------------------------------------------+
-| XPU-SMI               Driver Version: 5.19.0.0     XPU-RT Version: 5.19.0   |
-|-------------------------------+----------------------+----------------------+
-| XPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
-| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | XPU-Util  Compute M. |
-|                               |             L3-Usage |            SR-IOV M. |
-|===============================+======================+======================|
-|   0  P800 OAM           N/A   | 00000000:03:00.0 N/A |                    0 |
-| N/A   39C  N/A     89W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   1  P800 OAM           N/A   | 00000000:05:00.0 N/A |                    0 |
-| N/A   43C  N/A     94W / 400W |  87958MiB / 98304MiB |      0%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   2  P800 OAM           N/A   | 00000000:63:00.0 N/A |                    0 |
-| N/A   38C  N/A     95W / 400W |  87958MiB / 98304MiB |     33%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   3  P800 OAM           N/A   | 00000000:65:00.0 N/A |                    0 |
-| N/A   44C  N/A     92W / 400W |  87958MiB / 98304MiB |     32%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   4  P800 OAM           N/A   | 00000000:83:00.0 N/A |                    0 |
-| N/A   39C  N/A     90W / 400W |  88310MiB / 98304MiB |     29%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   5  P800 OAM           N/A   | 00000000:85:00.0 N/A |                    0 |
-| N/A   44C  N/A     95W / 400W |  87958MiB / 98304MiB |     29%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   6  P800 OAM           N/A   | 00000000:A3:00.0 N/A |                    0 |
-| N/A   41C  N/A     92W / 400W |  87914MiB / 98304MiB |     30%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-|   7  P800 OAM           N/A   | 00000000:A5:00.0 N/A |                    0 |
-| N/A   48C  N/A     89W / 400W |  88292MiB / 98304MiB |      3%      Default |
-|                               |     92MiB /    96MiB |             Disabled |
-+-------------------------------+----------------------+----------------------+
-                                                                               
-+-----------------------------------------------------------------------------+
-| Processes:                                                                  |
-|  XPU   XI   CI        PID   Type   Process name                  XPU Memory |
-|        ID   ID                                                   Usage      |
-|=============================================================================|
-|    0   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    0   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    0   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    0   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3       87826MiB |
-|    0   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    0   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    0   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    0   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    1   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    1   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    1   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2       87826MiB |
-|    1   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    1   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    1   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    1   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    1   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    2   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0       87826MiB |
-|    2   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    2   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    2   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    2   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    2   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    2   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    2   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    3   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    3   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1       87826MiB |
-|    3   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    3   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    3   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    3   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    3   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    3   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    4   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    4   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    4   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    4   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    4   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    4   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    4   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    4   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7       88178MiB |
-|    5   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    5   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    5   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    5   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    5   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    5   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    5   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6       87826MiB |
-|    5   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    6   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    6   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    6   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    6   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    6   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4       87782MiB |
-|    6   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5          32MiB |
-|    6   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    6   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-|    7   N/A  N/A    137940      C   sglang::scheduler_TP0_EP0          32MiB |
-|    7   N/A  N/A    137941      C   sglang::scheduler_TP1_EP1          32MiB |
-|    7   N/A  N/A    137942      C   sglang::scheduler_TP2_EP2          32MiB |
-|    7   N/A  N/A    137943      C   sglang::scheduler_TP3_EP3          32MiB |
-|    7   N/A  N/A    137944      C   sglang::scheduler_TP4_EP4          32MiB |
-|    7   N/A  N/A    137945      C   sglang::scheduler_TP5_EP5       88160MiB |
-|    7   N/A  N/A    137946      C   sglang::scheduler_TP6_EP6          32MiB |
-|    7   N/A  N/A    137947      C   sglang::scheduler_TP7_EP7          32MiB |
-+-----------------------------------------------------------------------------+

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/container_inspect.json

@@ -1,231 +0,0 @@
-[
-    {
-        "Id": "5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180",
-        "Created": "2026-08-17T15:52:21.087868771Z",
-        "Path": "bash",
-        "Args": [
-            "-lc",
-            "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-        ],
-        "State": {
-            "Status": "running",
-            "Running": true,
-            "Paused": false,
-            "Restarting": false,
-            "OOMKilled": false,
-            "Dead": false,
-            "Pid": 137722,
-            "ExitCode": 0,
-            "Error": "",
-            "StartedAt": "2026-08-17T15:52:21.126324465Z",
-            "FinishedAt": "0001-01-01T00:00:00Z"
-        },
-        "Image": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-        "ResolvConfPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/resolv.conf",
-        "HostnamePath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hostname",
-        "HostsPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hosts",
-        "LogPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180-json.log",
-        "Name": "/glm52-w4a8-tp8",
-        "RestartCount": 0,
-        "Driver": "overlayfs",
-        "Platform": "linux",
-        "MountLabel": "",
-        "ProcessLabel": "",
-        "AppArmorProfile": "unconfined",
-        "ExecIDs": null,
-        "HostConfig": {
-            "Binds": [
-                "/data2/GLM-5.2-w4a8:/models:ro",
-                "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro",
-                "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke:/workspace/run"
-            ],
-            "ContainerIDFile": "",
-            "LogConfig": {
-                "Type": "json-file",
-                "Config": {}
-            },
-            "NetworkMode": "host",
-            "PortBindings": {},
-            "RestartPolicy": {
-                "Name": "no",
-                "MaximumRetryCount": 0
-            },
-            "AutoRemove": false,
-            "VolumeDriver": "",
-            "VolumesFrom": null,
-            "ConsoleSize": [
-                0,
-                0
-            ],
-            "CapAdd": null,
-            "CapDrop": null,
-            "CgroupnsMode": "private",
-            "Dns": null,
-            "DnsOptions": [],
-            "DnsSearch": [],
-            "ExtraHosts": null,
-            "GroupAdd": null,
-            "IpcMode": "host",
-            "Cgroup": "",
-            "Links": null,
-            "OomScoreAdj": 0,
-            "PidMode": "",
-            "Privileged": true,
-            "PublishAllPorts": false,
-            "ReadonlyRootfs": false,
-            "SecurityOpt": [
-                "seccomp=unconfined",
-                "label=disable"
-            ],
-            "UTSMode": "host",
-            "UsernsMode": "",
-            "ShmSize": 274877906944,
-            "Runtime": "runc",
-            "Isolation": "",
-            "CpuShares": 0,
-            "Memory": 0,
-            "NanoCpus": 0,
-            "CgroupParent": "",
-            "BlkioWeight": 0,
-            "BlkioWeightDevice": [],
-            "BlkioDeviceReadBps": [],
-            "BlkioDeviceWriteBps": [],
-            "BlkioDeviceReadIOps": [],
-            "BlkioDeviceWriteIOps": [],
-            "CpuPeriod": 0,
-            "CpuQuota": 0,
-            "CpuRealtimePeriod": 0,
-            "CpuRealtimeRuntime": 0,
-            "CpusetCpus": "",
-            "CpusetMems": "",
-            "Devices": [],
-            "DeviceCgroupRules": null,
-            "DeviceRequests": null,
-            "MemoryReservation": 0,
-            "MemorySwap": 0,
-            "MemorySwappiness": null,
-            "OomKillDisable": null,
-            "PidsLimit": null,
-            "Ulimits": [
-                {
-                    "Name": "core",
-                    "Hard": -1,
-                    "Soft": -1
-                }
-            ],
-            "CpuCount": 0,
-            "CpuPercent": 0,
-            "IOMaximumIOps": 0,
-            "IOMaximumBandwidth": 0,
-            "MaskedPaths": null,
-            "ReadonlyPaths": null
-        },
-        "Storage": {
-            "RootFS": {
-                "Snapshot": {
-                    "Name": "overlayfs"
-                }
-            }
-        },
-        "Mounts": [
-            {
-                "Type": "bind",
-                "Source": "/data2/GLM-5.2-w4a8",
-                "Destination": "/models",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Destination": "/workspace/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke",
-                "Destination": "/workspace/run",
-                "Mode": "",
-                "RW": true,
-                "Propagation": "rprivate"
-            }
-        ],
-        "Config": {
-            "Hostname": "gpu051",
-            "Domainname": "",
-            "User": "",
-            "AttachStdin": false,
-            "AttachStdout": false,
-            "AttachStderr": false,
-            "Tty": false,
-            "OpenStdin": false,
-            "StdinOnce": false,
-            "Env": [
-                "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "CUDA_DISABLE_PRINTF=1",
-                "GLOO_SOCKET_TIMEOUT=60000",
-                "PATH=/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/torch_xmlir/xre/bin/:/root/miniconda/bin:/root/.cargo/bin:/usr/local/go/bin:/usr/local/cuda-11.7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "TZ=Asia/Shanghai",
-                "LANG=zh_CN.UTF-8"
-            ],
-            "Cmd": [
-                "bash",
-                "-lc",
-                "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-            ],
-            "Image": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-            "Volumes": null,
-            "WorkingDir": "/workspace/run",
-            "Entrypoint": null,
-            "Labels": {
-                "autofix_time": "2025-12-03T10:19:26",
-                "description": "infer_ubuntu2204_dev runtime + miniconda",
-                "maintainer": "wenli <wenli01@baidu.com>",
-                "org.opencontainers.image.ref.name": "ubuntu",
-                "org.opencontainers.image.version": "22.04",
-                "version": "infer_ubuntu2204_dev:miniconda_0.1"
-            }
-        },
-        "NetworkSettings": {
-            "SandboxID": "81c79e65bbac2dcf756cbff44c4a0ba01bd8be20cc29b1b7907701dfb0a30619",
-            "SandboxKey": "/var/run/docker/netns/default",
-            "Ports": {},
-            "Networks": {
-                "host": {
-                    "IPAMConfig": null,
-                    "Links": null,
-                    "Aliases": null,
-                    "DriverOpts": null,
-                    "GwPriority": 0,
-                    "NetworkID": "0e3fdd7bc7639e6dc911f6f09344de480a35fa47ce2e83ff21a8160e319f48ac",
-                    "EndpointID": "35fe2742654dbbedd58999151fd8a77234c95ea9204ebd61074cd27d27d85871",
-                    "Gateway": "",
-                    "IPAddress": "",
-                    "MacAddress": "",
-                    "IPPrefixLen": 0,
-                    "IPv6Gateway": "",
-                    "GlobalIPv6Address": "",
-                    "GlobalIPv6PrefixLen": 0,
-                    "DNSNames": null
-                }
-            }
-        },
-        "ImageManifestDescriptor": {
-            "mediaType": "application/vnd.oci.image.manifest.v1+json",
-            "digest": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-            "size": 3632,
-            "annotations": {
-                "io.containerd.image.name": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-                "org.opencontainers.image.ref.name": "20260715_5746"
-            },
-            "platform": {
-                "architecture": "amd64",
-                "os": "linux"
-            }
-        }
-    }
-]

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/container_inspect.resume.20260818-020923.json

@@ -1,231 +0,0 @@
-[
-    {
-        "Id": "5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180",
-        "Created": "2026-08-17T15:52:21.087868771Z",
-        "Path": "bash",
-        "Args": [
-            "-lc",
-            "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-        ],
-        "State": {
-            "Status": "running",
-            "Running": true,
-            "Paused": false,
-            "Restarting": false,
-            "OOMKilled": false,
-            "Dead": false,
-            "Pid": 194559,
-            "ExitCode": 0,
-            "Error": "",
-            "StartedAt": "2026-08-18T02:07:51.684912653Z",
-            "FinishedAt": "2026-08-18T01:49:40.51417112Z"
-        },
-        "Image": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-        "ResolvConfPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/resolv.conf",
-        "HostnamePath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hostname",
-        "HostsPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/hosts",
-        "LogPath": "/data1/sora/evalscope/docker/images/containers/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180/5526e1da90d6b5fbd4304d4e407b198cf1a6b2ad76af31ee4b76077d35ece180-json.log",
-        "Name": "/glm52-w4a8-tp8",
-        "RestartCount": 0,
-        "Driver": "overlayfs",
-        "Platform": "linux",
-        "MountLabel": "",
-        "ProcessLabel": "",
-        "AppArmorProfile": "unconfined",
-        "ExecIDs": null,
-        "HostConfig": {
-            "Binds": [
-                "/data2/GLM-5.2-w4a8:/models:ro",
-                "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro",
-                "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke:/workspace/run"
-            ],
-            "ContainerIDFile": "",
-            "LogConfig": {
-                "Type": "json-file",
-                "Config": {}
-            },
-            "NetworkMode": "host",
-            "PortBindings": {},
-            "RestartPolicy": {
-                "Name": "no",
-                "MaximumRetryCount": 0
-            },
-            "AutoRemove": false,
-            "VolumeDriver": "",
-            "VolumesFrom": null,
-            "ConsoleSize": [
-                0,
-                0
-            ],
-            "CapAdd": null,
-            "CapDrop": null,
-            "CgroupnsMode": "private",
-            "Dns": null,
-            "DnsOptions": [],
-            "DnsSearch": [],
-            "ExtraHosts": null,
-            "GroupAdd": null,
-            "IpcMode": "host",
-            "Cgroup": "",
-            "Links": null,
-            "OomScoreAdj": 0,
-            "PidMode": "",
-            "Privileged": true,
-            "PublishAllPorts": false,
-            "ReadonlyRootfs": false,
-            "SecurityOpt": [
-                "seccomp=unconfined",
-                "label=disable"
-            ],
-            "UTSMode": "host",
-            "UsernsMode": "",
-            "ShmSize": 274877906944,
-            "Runtime": "runc",
-            "Isolation": "",
-            "CpuShares": 0,
-            "Memory": 0,
-            "NanoCpus": 0,
-            "CgroupParent": "",
-            "BlkioWeight": 0,
-            "BlkioWeightDevice": [],
-            "BlkioDeviceReadBps": [],
-            "BlkioDeviceWriteBps": [],
-            "BlkioDeviceReadIOps": [],
-            "BlkioDeviceWriteIOps": [],
-            "CpuPeriod": 0,
-            "CpuQuota": 0,
-            "CpuRealtimePeriod": 0,
-            "CpuRealtimeRuntime": 0,
-            "CpusetCpus": "",
-            "CpusetMems": "",
-            "Devices": [],
-            "DeviceCgroupRules": null,
-            "DeviceRequests": null,
-            "MemoryReservation": 0,
-            "MemorySwap": 0,
-            "MemorySwappiness": null,
-            "OomKillDisable": null,
-            "PidsLimit": null,
-            "Ulimits": [
-                {
-                    "Name": "core",
-                    "Hard": -1,
-                    "Soft": -1
-                }
-            ],
-            "CpuCount": 0,
-            "CpuPercent": 0,
-            "IOMaximumIOps": 0,
-            "IOMaximumBandwidth": 0,
-            "MaskedPaths": null,
-            "ReadonlyPaths": null
-        },
-        "Storage": {
-            "RootFS": {
-                "Snapshot": {
-                    "Name": "overlayfs"
-                }
-            }
-        },
-        "Mounts": [
-            {
-                "Type": "bind",
-                "Source": "/data2/GLM-5.2-w4a8",
-                "Destination": "/models",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Destination": "/workspace/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke",
-                "Destination": "/workspace/run",
-                "Mode": "",
-                "RW": true,
-                "Propagation": "rprivate"
-            }
-        ],
-        "Config": {
-            "Hostname": "gpu051",
-            "Domainname": "",
-            "User": "",
-            "AttachStdin": false,
-            "AttachStdout": false,
-            "AttachStderr": false,
-            "Tty": false,
-            "OpenStdin": false,
-            "StdinOnce": false,
-            "Env": [
-                "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "CUDA_DISABLE_PRINTF=1",
-                "GLOO_SOCKET_TIMEOUT=60000",
-                "PATH=/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/torch_xmlir/xre/bin/:/root/miniconda/bin:/root/.cargo/bin:/usr/local/go/bin:/usr/local/cuda-11.7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "TZ=Asia/Shanghai",
-                "LANG=zh_CN.UTF-8"
-            ],
-            "Cmd": [
-                "bash",
-                "-lc",
-                "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-            ],
-            "Image": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-            "Volumes": null,
-            "WorkingDir": "/workspace/run",
-            "Entrypoint": null,
-            "Labels": {
-                "autofix_time": "2025-12-03T10:19:26",
-                "description": "infer_ubuntu2204_dev runtime + miniconda",
-                "maintainer": "wenli <wenli01@baidu.com>",
-                "org.opencontainers.image.ref.name": "ubuntu",
-                "org.opencontainers.image.version": "22.04",
-                "version": "infer_ubuntu2204_dev:miniconda_0.1"
-            }
-        },
-        "NetworkSettings": {
-            "SandboxID": "02aa573ca306f134cdbd46e5e4f7df8d14de30e27b9ef68bc83e41ef6db028c3",
-            "SandboxKey": "/var/run/docker/netns/default",
-            "Ports": {},
-            "Networks": {
-                "host": {
-                    "IPAMConfig": null,
-                    "Links": null,
-                    "Aliases": null,
-                    "DriverOpts": null,
-                    "GwPriority": 0,
-                    "NetworkID": "0e3fdd7bc7639e6dc911f6f09344de480a35fa47ce2e83ff21a8160e319f48ac",
-                    "EndpointID": "9d8f2b98c63551f3a9576d7381f86190bf36916fceaee65c116c1cc1bd83053a",
-                    "Gateway": "",
-                    "IPAddress": "",
-                    "MacAddress": "",
-                    "IPPrefixLen": 0,
-                    "IPv6Gateway": "",
-                    "GlobalIPv6Address": "",
-                    "GlobalIPv6PrefixLen": 0,
-                    "DNSNames": null
-                }
-            }
-        },
-        "ImageManifestDescriptor": {
-            "mediaType": "application/vnd.oci.image.manifest.v1+json",
-            "digest": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-            "size": 3632,
-            "annotations": {
-                "io.containerd.image.name": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-                "org.opencontainers.image.ref.name": "20260715_5746"
-            },
-            "platform": {
-                "architecture": "amd64",
-                "os": "linux"
-            }
-        }
-    }
-]

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/container_inspect.resume.20260818-035808.json

@@ -1,232 +0,0 @@
-[
-    {
-        "Id": "ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87",
-        "Created": "2026-08-18T03:55:32.13494019Z",
-        "Path": "bash",
-        "Args": [
-            "-lc",
-            "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-        ],
-        "State": {
-            "Status": "running",
-            "Running": true,
-            "Paused": false,
-            "Restarting": false,
-            "OOMKilled": false,
-            "Dead": false,
-            "Pid": 256057,
-            "ExitCode": 0,
-            "Error": "",
-            "StartedAt": "2026-08-18T03:55:32.172163502Z",
-            "FinishedAt": "0001-01-01T00:00:00Z"
-        },
-        "Image": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-        "ResolvConfPath": "/data1/sora/evalscope/docker/images/containers/ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87/resolv.conf",
-        "HostnamePath": "/data1/sora/evalscope/docker/images/containers/ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87/hostname",
-        "HostsPath": "/data1/sora/evalscope/docker/images/containers/ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87/hosts",
-        "LogPath": "/data1/sora/evalscope/docker/images/containers/ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87/ec1f7a67284e4e9815888959d86a56b7749357017d8a4b0f04b409d0a1120f87-json.log",
-        "Name": "/glm52-w4a8-tp8",
-        "RestartCount": 0,
-        "Driver": "overlayfs",
-        "Platform": "linux",
-        "MountLabel": "",
-        "ProcessLabel": "",
-        "AppArmorProfile": "unconfined",
-        "ExecIDs": null,
-        "HostConfig": {
-            "Binds": [
-                "/data2/GLM-5.2-w4a8:/models:ro",
-                "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro",
-                "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke:/workspace/run"
-            ],
-            "ContainerIDFile": "",
-            "LogConfig": {
-                "Type": "json-file",
-                "Config": {}
-            },
-            "NetworkMode": "host",
-            "PortBindings": {},
-            "RestartPolicy": {
-                "Name": "no",
-                "MaximumRetryCount": 0
-            },
-            "AutoRemove": false,
-            "VolumeDriver": "",
-            "VolumesFrom": null,
-            "ConsoleSize": [
-                0,
-                0
-            ],
-            "CapAdd": null,
-            "CapDrop": null,
-            "CgroupnsMode": "private",
-            "Dns": null,
-            "DnsOptions": [],
-            "DnsSearch": [],
-            "ExtraHosts": null,
-            "GroupAdd": null,
-            "IpcMode": "host",
-            "Cgroup": "",
-            "Links": null,
-            "OomScoreAdj": 0,
-            "PidMode": "",
-            "Privileged": true,
-            "PublishAllPorts": false,
-            "ReadonlyRootfs": false,
-            "SecurityOpt": [
-                "seccomp=unconfined",
-                "label=disable"
-            ],
-            "UTSMode": "host",
-            "UsernsMode": "",
-            "ShmSize": 274877906944,
-            "Runtime": "runc",
-            "Isolation": "",
-            "CpuShares": 0,
-            "Memory": 0,
-            "NanoCpus": 0,
-            "CgroupParent": "",
-            "BlkioWeight": 0,
-            "BlkioWeightDevice": [],
-            "BlkioDeviceReadBps": [],
-            "BlkioDeviceWriteBps": [],
-            "BlkioDeviceReadIOps": [],
-            "BlkioDeviceWriteIOps": [],
-            "CpuPeriod": 0,
-            "CpuQuota": 0,
-            "CpuRealtimePeriod": 0,
-            "CpuRealtimeRuntime": 0,
-            "CpusetCpus": "",
-            "CpusetMems": "",
-            "Devices": [],
-            "DeviceCgroupRules": null,
-            "DeviceRequests": null,
-            "MemoryReservation": 0,
-            "MemorySwap": 0,
-            "MemorySwappiness": null,
-            "OomKillDisable": null,
-            "PidsLimit": null,
-            "Ulimits": [
-                {
-                    "Name": "core",
-                    "Hard": -1,
-                    "Soft": -1
-                }
-            ],
-            "CpuCount": 0,
-            "CpuPercent": 0,
-            "IOMaximumIOps": 0,
-            "IOMaximumBandwidth": 0,
-            "MaskedPaths": null,
-            "ReadonlyPaths": null,
-            "Init": true
-        },
-        "Storage": {
-            "RootFS": {
-                "Snapshot": {
-                    "Name": "overlayfs"
-                }
-            }
-        },
-        "Mounts": [
-            {
-                "Type": "bind",
-                "Source": "/data2/GLM-5.2-w4a8",
-                "Destination": "/models",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Destination": "/workspace/ShareGPT_V3_unfiltered_cleaned_split.json",
-                "Mode": "ro",
-                "RW": false,
-                "Propagation": "rprivate"
-            },
-            {
-                "Type": "bind",
-                "Source": "/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke",
-                "Destination": "/workspace/run",
-                "Mode": "",
-                "RW": true,
-                "Propagation": "rprivate"
-            }
-        ],
-        "Config": {
-            "Hostname": "gpu051",
-            "Domainname": "",
-            "User": "",
-            "AttachStdin": false,
-            "AttachStdout": false,
-            "AttachStderr": false,
-            "Tty": false,
-            "OpenStdin": false,
-            "StdinOnce": false,
-            "Env": [
-                "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7",
-                "CUDA_DISABLE_PRINTF=1",
-                "GLOO_SOCKET_TIMEOUT=60000",
-                "PATH=/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/torch_xmlir/xre/bin/:/root/miniconda/bin:/root/.cargo/bin:/usr/local/go/bin:/usr/local/cuda-11.7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
-                "TZ=Asia/Shanghai",
-                "LANG=zh_CN.UTF-8"
-            ],
-            "Cmd": [
-                "bash",
-                "-lc",
-                "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server     --host 0.0.0.0     --port 30000     --model-path /models     --served-model-name GLM-5.2-w4a8     --attention-backend nsa     --nsa-prefill klxdsa     --nsa-decode klxdsa     --prefill-attention-backend klxdsa     --decode-attention-backend klxdsa     --trust-remote-code     --disable-custom-all-reduce     --disable-shared-experts-fusion     --chunked-prefill-size 32768     --page-size 64     --mem-fraction-static 0.82     --max-prefill-tokens 32768     --max-running-requests 64     --tp-size 8     --ep-size 8     --quantization w4a8_int4     --kv-cache-dtype int8     --cuda-graph-max-bs 8     --disable-cuda-graph-padding     --watchdog-timeout 3000000     --tool-call-parser glm47     --reasoning-parser glm45     --json-model-override-args '{\"index_topk_freq\": 4}'     --allow-auto-truncate     --enable-metrics"
-            ],
-            "Image": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-            "Volumes": null,
-            "WorkingDir": "/workspace/run",
-            "Entrypoint": null,
-            "Labels": {
-                "autofix_time": "2025-12-03T10:19:26",
-                "description": "infer_ubuntu2204_dev runtime + miniconda",
-                "maintainer": "wenli <wenli01@baidu.com>",
-                "org.opencontainers.image.ref.name": "ubuntu",
-                "org.opencontainers.image.version": "22.04",
-                "version": "infer_ubuntu2204_dev:miniconda_0.1"
-            }
-        },
-        "NetworkSettings": {
-            "SandboxID": "40813e57fb14f319ce4dc5e1f06a130d63d9b1cd36916bc1ebc7c33eafe04cc2",
-            "SandboxKey": "/var/run/docker/netns/default",
-            "Ports": {},
-            "Networks": {
-                "host": {
-                    "IPAMConfig": null,
-                    "Links": null,
-                    "Aliases": null,
-                    "DriverOpts": null,
-                    "GwPriority": 0,
-                    "NetworkID": "0e3fdd7bc7639e6dc911f6f09344de480a35fa47ce2e83ff21a8160e319f48ac",
-                    "EndpointID": "5eb7dfb7d5c85d6d8c88dc3595bfa83deff49cff1c3bcc1ab946a9b88b4bc3d4",
-                    "Gateway": "",
-                    "IPAddress": "",
-                    "MacAddress": "",
-                    "IPPrefixLen": 0,
-                    "IPv6Gateway": "",
-                    "GlobalIPv6Address": "",
-                    "GlobalIPv6PrefixLen": 0,
-                    "DNSNames": null
-                }
-            }
-        },
-        "ImageManifestDescriptor": {
-            "mediaType": "application/vnd.oci.image.manifest.v1+json",
-            "digest": "sha256:5079ea4ecbeff690457c801261cc4ef4f80cd29012b8bda5fd6742138b82069d",
-            "size": 3632,
-            "annotations": {
-                "io.containerd.image.name": "iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746",
-                "org.opencontainers.image.ref.name": "20260715_5746"
-            },
-            "platform": {
-                "architecture": "amd64",
-                "os": "linux"
-            }
-        }
-    }
-]

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failure_boundaries.tsv

@@ -1,17 +0,0 @@
-8192	2048	64	c64_i8192_o2048	kv_cache_decode_oom
-8192	4096	64	c64_i8192_o4096	kv_cache_decode_oom
-16384	512	32	c32_i16384_o512	xpu_kernel_launch_timeout
-16384	2048	32	c32_i16384_o2048	out_of_memory
-16384	4096	32	c32_i16384_o4096	kv_cache_decode_oom
-32768	128	1	c1_i32768_o128	out_of_memory
-65536	128	1	c1_i65536_o128	out_of_memory
-65536	128	8	c8_i65536_o128	out_of_memory
-65536	256	1	c1_i65536_o256	out_of_memory
-65536	512	1	c1_i65536_o512	out_of_memory
-65536	1024	1	c1_i65536_o1024	out_of_memory
-65536	2048	1	c1_i65536_o2048	out_of_memory
-131072	128	1	c1_i131072_o128	out_of_memory
-131072	128	4	c4_i131072_o128	out_of_memory
-131072	256	1	c1_i131072_o256	out_of_memory
-131072	512	1	c1_i131072_o512	out_of_memory
-131072	1024	1	c1_i131072_o1024	out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i131072_o1024.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i131072_o128.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i131072_o256.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i131072_o512.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i32768_o128.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i65536_o1024.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i65536_o128.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i65536_o2048.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i65536_o256.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c1_i65536_o512.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c32_i16384_o2048.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c32_i16384_o4096.reason

@@ -1 +0,0 @@
-kv_cache_decode_oom

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c32_i16384_o512.reason

@@ -1 +0,0 @@
-previous_failure

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c4_i131072_o128.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c64_i8192_o2048.reason

@@ -1 +0,0 @@
-kv_cache_decode_oom

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c64_i8192_o4096.reason

@@ -1 +0,0 @@
-previous_failure

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/failures/c8_i65536_o128.reason

@@ -1 +0,0 @@
-out_of_memory

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/matrix.json

@@ -1,93 +0,0 @@
-{
-  "comment": "ISL/OSL matrix for dsv4_p800_sglang_tp_dp_matrix. Y=must test, P=optional (record skipped on failure), N=skip.",
-  "mode": "Y",
-  "notes": "P800 INT8 sustains at most ~131k input tokens, so ISL stops at 131072. Concurrency ranges match the H20 matrix; scenarios that exceed server capacity are recorded as failed/skipped and the run moves on.",
-  "matrix": {
-    "1024": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "Y",
-      "4096": "Y"
-    },
-    "4096": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "Y",
-      "4096": "Y"
-    },
-    "8192": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "Y",
-      "4096": "Y"
-    },
-    "16384": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "Y",
-      "4096": "P"
-    },
-    "32768": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "Y",
-      "4096": "P"
-    },
-    "65536": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "Y",
-      "2048": "P",
-      "4096": "N"
-    },
-    "131072": {
-      "128": "Y",
-      "256": "Y",
-      "512": "Y",
-      "1024": "P",
-      "2048": "N",
-      "4096": "N"
-    }
-  },
-  "concurrency": {
-    "1024": {
-      "low": 1,
-      "high": 128
-    },
-    "4096": {
-      "low": 1,
-      "high": 64
-    },
-    "8192": {
-      "low": 1,
-      "high": 64
-    },
-    "16384": {
-      "low": 1,
-      "high": 32
-    },
-    "32768": {
-      "low": 1,
-      "high": 16
-    },
-    "65536": {
-      "low": 1,
-      "high": 8
-    },
-    "131072": {
-      "low": 1,
-      "high": 4
-    }
-  }
-}

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/scenarios.tsv

@@ -1,79 +0,0 @@
-mark	input_len	output_len	concurrency	num_prompts
-Y	1024	128	1	5
-Y	1024	128	128	640
-Y	1024	256	1	5
-Y	1024	256	128	640
-Y	1024	512	1	5
-Y	1024	512	128	640
-Y	1024	1024	1	5
-Y	1024	1024	128	640
-Y	1024	2048	1	5
-Y	1024	2048	128	640
-Y	1024	4096	1	5
-Y	1024	4096	128	640
-Y	4096	128	1	5
-Y	4096	128	64	320
-Y	4096	256	1	5
-Y	4096	256	64	320
-Y	4096	512	1	5
-Y	4096	512	64	320
-Y	4096	1024	1	5
-Y	4096	1024	64	320
-Y	4096	2048	1	5
-Y	4096	2048	64	320
-Y	4096	4096	1	5
-Y	4096	4096	64	320
-Y	8192	128	1	5
-Y	8192	128	64	320
-Y	8192	256	1	5
-Y	8192	256	64	320
-Y	8192	512	1	5
-Y	8192	512	64	320
-Y	8192	1024	1	5
-Y	8192	1024	64	320
-Y	8192	2048	1	5
-Y	8192	2048	64	320
-Y	8192	4096	1	5
-Y	8192	4096	64	320
-Y	16384	128	1	5
-Y	16384	128	32	160
-Y	16384	256	1	5
-Y	16384	256	32	160
-Y	16384	512	1	5
-Y	16384	512	32	160
-Y	16384	1024	1	5
-Y	16384	1024	32	160
-Y	16384	2048	1	5
-Y	16384	2048	32	160
-P	16384	4096	1	5
-P	16384	4096	32	160
-Y	32768	128	1	5
-Y	32768	128	16	80
-Y	32768	256	1	5
-Y	32768	256	16	80
-Y	32768	512	1	5
-Y	32768	512	16	80
-Y	32768	1024	1	5
-Y	32768	1024	16	80
-Y	32768	2048	1	5
-Y	32768	2048	16	80
-P	32768	4096	1	5
-P	32768	4096	16	80
-Y	65536	128	1	5
-Y	65536	128	8	40
-Y	65536	256	1	5
-Y	65536	256	8	40
-Y	65536	512	1	5
-Y	65536	512	8	40
-Y	65536	1024	1	5
-Y	65536	1024	8	40
-P	65536	2048	1	5
-P	65536	2048	8	40
-Y	131072	128	1	5
-Y	131072	128	4	20
-Y	131072	256	1	5
-Y	131072	256	4	20
-Y	131072	512	1	5
-Y	131072	512	4	20
-P	131072	1024	1	5
-P	131072	1024	4	20

experiments/p800/glm52_w4a8_sglang_tp8_single_node/results/20260818-glm52-w4a8-full-matrix/summary.tsv

@@ -1,79 +0,0 @@
-mark	input_len	output_len	concurrency	num_prompts	scenario	status	completed	duration_s	request_throughput	input_throughput	output_throughput	total_token_throughput	mean_ttft_ms	p99_ttft_ms	mean_tpot_ms	p99_tpot_ms
-Y	1024	128	1	5	c1_i1024_o128	"completed	rc=0	duration_s=39"	5	17.02591361099985	0.2936699970549406	300.7180769842592	37.5897596230324		281.2652568012709	297.18383764105965	24.568037981093877	24.614217375115466
-Y	1024	128	128	640	c128_i1024_o128	"completed	rc=0	duration_s=365"	640	342.486345420999	1.8686876383736828	1913.5361416946512	239.1920177118314		41669.024385146906	45741.00833293076	183.7266033815086	187.1878648276444
-Y	1024	256	1	5	c1_i1024_o256	"completed	rc=0	duration_s=55"	5	32.81497869300074	0.1523694422226297	156.0263088359728	39.0065772089932		278.39411059903796	294.2181943576725	24.633620097259097	24.658055767057437
-Y	1024	256	128	640	c128_i1024_o256	"completed	rc=0	duration_s=597"	640	574.0754678120029	1.1148360030768392	1141.5920671506833	285.3980167876708		62527.62441754065	68956.42354271989	182.28569678729144	184.03258206838555
-Y	1024	512	1	5	c1_i1024_o512	"completed	rc=0	duration_s=87"	5	65.10708004299886	0.07679656339522269	78.63968091670803	39.319840458354015		279.30358499943395	294.0611118795641	24.93063542035209	24.953084254637705
-Y	1024	512	128	640	c128_i1024_o512	"completed	rc=0	duration_s=1065"	640	1042.2514380899993	0.6140552812983843	628.7926080495455	314.39630402477275		104649.0250889984	115665.911804529	182.58666003007582	184.05825803236687
-Y	1024	1024	1	5	c1_i1024_o1024	"completed	rc=0	duration_s=153"	5	131.8887679899999	0.03791073399350513	38.82059160934925	38.82059160934925		281.0054517998651	295.132770438795	25.5066858453576	25.511918811260603
-Y	1024	1024	128	640	c128_i1024_o1024	"completed	rc=0	duration_s=1983"	640	1956.396799541999	0.32713200111032015	334.98316913696783	334.98316913696783		186932.53216034532	207552.52155184935	180.55811861379217	181.34573211966844
-Y	1024	2048	1	5	c1_i1024_o2048	"completed	rc=0	duration_s=290"	5	268.2550993270015	0.01863897466457861	19.086310056528497	38.17262011305699		284.6344647994556	294.75402423980995	26.06863725754749	26.112281927875404
-Y	1024	2048	128	640	c128_i1024_o2048	"completed	rc=0	duration_s=3843"	640	3815.7466828769975	0.16772601883452407	171.75144328655264	343.5028865731053		354123.60505508434	393690.0640932885	181.0707906767069	181.93800417857398
-Y	1024	4096	1	5	c1_i1024_o4096	"completed	rc=0	duration_s=560"	5	537.174628175002	0.009307960089230216	9.531351131371741	38.125404525486964		287.3897543991916	295.1975400408264	26.164411530549657	26.182873973002035
-Y	1024	4096	128	640	c128_i1024_o4096	"completed	rc=0	duration_s=7630"	640	7595.250456256996	0.0842631857482416	86.2855022061994	345.1420088247976		694281.792547897	772382.6850643696	182.80351444481107	183.29835544574874
-Y	4096	128	1	5	c1_i4096_o128	"completed	rc=0	duration_s=49"	5	23.562466644005326	0.21220189191321703	869.178949276537	27.16184216489178		1348.1614620017353	1376.2398038062383	26.4617152897642	26.536813717162214
-Y	4096	128	64	320	c64_i4096_o128	"completed	rc=0	duration_s=539"	320	513.9221596100033	0.6226623896561228	2550.425148031479	79.70078587598371		52487.32021765952	81989.41991875166	395.8419637399347	719.1297852202871
-Y	4096	256	1	5	c1_i4096_o256	"completed	rc=0	duration_s=65"	5	40.363449060001585	0.12387444870153036	507.38974188146835	31.711858867591772		1348.8969976024237	1376.5715911236475	26.354579003910104	26.36532255057268
-Y	4096	256	64	320	c64_i4096_o256	"completed	rc=0	duration_s=650"	320	625.7480431649965	0.5113879356001801	2094.644984218338	130.91531151364612		51983.60868599389	80408.54887854264	286.82753786230273	448.62066245765556
-Y	4096	512	1	5	c1_i4096_o512	"completed	rc=0	duration_s=99"	5	73.99429440300446	0.0675727776086068	276.77809708485347	34.597262135606684		1350.9769304000656	1376.4231815637322	26.310007427002912	26.334555245552153
-Y	4096	512	64	320	c64_i4096_o512	"completed	rc=0	duration_s=880"	320	855.845617013998	0.37389920990243986	1531.4911637603936	191.4363954700492		51983.15175365303	80405.63672760378	233.1860918822221	314.11634894583517
-Y	4096	1024	1	5	c1_i4096_o1024	"completed	rc=0	duration_s=165"	5	141.22379078199447	0.03540479951935607	145.01805883128247	36.25451470782062		1351.8061094000586	1379.9816434006789	26.28578575601136	26.32495529572055
-Y	4096	1024	64	320	c64_i4096_o1024	"completed	rc=0	duration_s=1345"	320	1320.7957280260016	0.24227819125237235	992.3714713697171	248.09286784242929		51987.80489426904	80406.50598565124	207.37264829075608	247.74751702926565
-Y	4096	2048	1	5	c1_i4096_o2048	"completed	rc=0	duration_s=299"	5	276.0557316109989	0.01811228468549133	74.18791807177249	37.09395903588624		1359.5692171991686	1377.9620103552588	26.305917187592286	26.323675634277176
-Y	4096	2048	64	320	c64_i4096_o2048	"completed	rc=0	duration_s=2280"	320	2253.1854364559986	0.14202115583674427	581.7186543073045	290.85932715365226		51999.37922932156	80419.80255557828	194.7249239024334	215.13688206616618
-Y	4096	4096	1	5	c1_i4096_o4096	"completed	rc=0	duration_s=570"	5	546.1440214710019	0.009155094267136421	37.49926611819078	37.49926611819078		1356.901797800674	1381.4528299999074	26.341520792234874	26.363355306765136
-Y	4096	4096	64	320	c64_i4096_o4096	"completed	rc=0	duration_s=4137"	320	4106.892298439998	0.07791779689999466	319.1512961023781	319.1512961023781		52028.30068695314	80428.11315494073	187.86320509006177	197.8918995976436
-Y	8192	128	1	5	c1_i8192_o128	"completed	rc=0	duration_s=58"	5	31.345879535998392	0.1595105983310462	1306.7108215279304	20.417356586373913		2901.3523344008718	2943.8824014391867	26.498316146440686	26.53415173224786
-Y	8192	128	64	320	c64_i8192_o128	"completed	rc=0	duration_s=1027"	320	1001.4245687750008	0.3195447864749735	2617.7108908029827	40.901732668796605		102210.96021643761	179702.23191498037	772.015502173622	1472.9441806528812
-Y	8192	256	1	5	c1_i8192_o256	"completed	rc=0	duration_s=75"	5	48.29129336799815	0.10353833271554946	848.1860216057812	26.505813175180663		2907.958858400525	2941.109013320529	26.463175448623264	26.479798227135436
-Y	8192	256	64	320	c64_i8192_o256	"completed	rc=0	duration_s=1144"	320	1118.6142376200005	0.28606823446199137	2343.4709767126333	73.23346802226979		102227.62581890015	179676.9426185613	476.33626962411694	825.4012724687083
-Y	8192	512	1	5	c1_i8192_o512	"completed	rc=0	duration_s=110"	5	82.16094908700325	0.06085616166270568	498.53367634088494	31.15835477130531		2904.7368975981954	2941.0467594809597	26.467229087280366	26.505994761013312
-Y	8192	512	64	320	c64_i8192_o512	"completed	rc=0	duration_s=1380"	320	1352.1895057329966	0.23665321956964455	1938.6631747145282	121.16644841965801		102229.57628892828	179680.07008601096	329.1102014473765	503.40242396277046
-Y	8192	1024	1	5	c1_i8192_o1024	"completed	rc=0	duration_s=174"	5	150.05084697799612	0.03332203783383651	272.9741339347887	34.12176674184859		2906.111534401134	2942.81814436079	26.49209734721495	26.533382691103906
-Y	8192	1024	64	320	c64_i8192_o1024	"completed	rc=0	duration_s=1845"	320	1819.0112392769952	0.17591974864717758	1441.1345809176787	180.14182261470984		102239.29165678738	179664.01260017834	255.6465966483472	343.00634268297966
-Y	8192	2048	1	5	c1_i8192_o2048	"completed	rc=0	duration_s=310"	5	285.48406155599514	0.017514112601411534	143.47561043076328	35.86890260769082		2907.4056262004888	2944.1087911635987	26.47123923947269	26.497441076600033
-Y	8192	2048	64	320	c64_i8192_o2048	"failed	rc=1	duration_s=553"	0									
-Y	8192	4096	1	5	c1_i8192_o4096	"completed	rc=0	duration_s=584"	5	558.9306328860039	0.008945653907324436	73.28279680880178	36.64139840440089		2908.0996894001146	2962.2204470040742	26.58702289313796	26.61703713931074
-Y	8192	4096	64	320	c64_i8192_o4096	"failed	rc=1	duration_s=719	reason=kv_cache_decode_oom"	0									
-Y	16384	128	1	5	c1_i16384_o128	"completed	rc=0	duration_s=76"	5	47.921742649996304	0.10433677332058344	1709.4536940844391	13.35510698503468		6168.107185598637	6217.074746359431	26.87305271969036	26.958400565712882
-Y	16384	128	32	160	c32_i16384_o128	"completed	rc=0	duration_s=1104"	160	1080.5961769310015	0.1480664131668646	2425.92011332591	18.95250088535867		107607.05564736869	194812.25933963628	854.2913843793324	1609.9457858553585
-Y	16384	256	1	5	c1_i16384_o256	"completed	rc=0	duration_s=93"	5	65.06508007799857	0.07684613611488852	1259.0470941063336	19.672610845411462		6165.073253607261	6211.129082086845	26.841394899609302	26.881746652878494
-Y	16384	256	32	160	c32_i16384_o256	"completed	rc=0	duration_s=1214"	160	1191.412069234997	0.1342944260273741	2200.279876032497	34.37937306300777		107601.49405923122	194837.96724378655	512.4159163919377	888.8134474421043
-Y	16384	512	1	5	c1_i16384_o512	"completed	rc=0	duration_s=127"	5	99.36944190099894	0.05031727968223334	824.398310313711	25.76244719730347		6175.6319466017885	6251.701926806127	26.801992351080585	26.82276618959694
-Y	16384	512	32	160	c32_i16384_o512	"failed	rc=1	duration_s=560	reason=xpu_kernel_launch_timeout"	0									
-Y	16384	1024	1	5	c1_i16384_o1024	"completed	rc=0	duration_s=224"	5	171.96120902399707	0.029076324994332027	476.38650870713593	29.774156794195996		6928.77221259987	9792.07814124471	26.842475365785894	26.88422662580788
-Y	16384	1024	32	160	c32_i16384_o1024	"completed	rc=0	duration_s=1882"	160	1853.834564181001	0.08630759350993428	1414.0636120667632	88.3789757541727		107601.72120737507	194817.03514240915	257.23059973136543	351.1609817709484
-Y	16384	2048	1	5	c1_i16384_o2048	"completed	rc=0	duration_s=334"	5	305.36463945500145	0.016373867023122695	268.26943730684224	33.53367966335528		6173.947739601135	6222.383297723136	26.817756513432563	26.837409417542258
-Y	16384	2048	32	160	c32_i16384_o2048	"failed	rc=137	duration_s=167	reason=out_of_memory"	0									
-P	16384	4096	1	5	c1_i16384_o4096	"completed	rc=0	duration_s=608"	5	580.1729553189944	0.008618119741984299	141.19927385267076	35.29981846316769		6178.636959602591	6234.941079312703	26.82611517767935	26.86785587242856
-P	16384	4096	32	160	c32_i16384_o4096	"failed	rc=137	duration_s=903	reason=kv_cache_decode_oom"	0									
-Y	32768	128	1	5	c1_i32768_o128	"failed	rc=137	duration_s=29	reason=out_of_memory"	0									
-Y	32768	128	16	80	c16_i32768_o128	"completed	rc=0	duration_s=1225"	80	1168.7748753930064	0.06844774103575729	2242.895578259695	8.761310852576933		124317.03163680068	214679.87077928378	861.6436795331623	1645.4117601102328
-Y	32768	256	1	5	c1_i32768_o256	"completed	rc=0	duration_s=134"	5	100.38509806500224	0.0498081896255394	1632.114757649675	12.750896544138087		13168.107665798743	13302.10535181046	27.085759039219543	27.132425062000003
-Y	32768	256	16	80	c16_i32768_o256	"completed	rc=0	duration_s=1286"	80	1263.4621468810074	0.06331808214238047	2074.806915641523	16.2094290284494		122892.92924403744	210721.14786341554	508.9795435045581	891.4333398982063
-Y	32768	512	1	5	c1_i32768_o512	"completed	rc=0	duration_s=170"	5	135.05323958600638	0.03702243659853738	1213.1512024608728	18.955487538451138		13167.90345320187	13303.137884192984	27.085512908023635	27.110341254001163
-Y	32768	512	16	80	c16_i32768_o512	"completed	rc=0	duration_s=1505"	80	1482.1555456469941	0.053975441535104335	1768.6672682222988	27.63542606597342		122898.48796358638	210686.75379800465	339.5751358392154	530.3086735966258
-Y	32768	1024	1	5	c1_i32768_o1024	"completed	rc=0	duration_s=238"	5	204.34472282300703	0.024468456688899888	801.7823887818715	25.055699649433485		13170.243986597052	13308.293756752973	27.073815555036006	27.08208811331908
-Y	32768	1024	16	80	c16_i32768_o1024	"completed	rc=0	duration_s=1944"	80	1921.5543885630032	0.04163296156286601	1364.2288844919933	42.63215264037479		122883.37932371232	210621.58506435342	255.5393236051565	350.83823580940947
-Y	32768	2048	1	5	c1_i32768_o2048	"completed	rc=0	duration_s=377"	5	342.98018497200974	0.014578101648665345	477.695234823466	29.855952176466626		13179.81927599758	13319.727632589638	27.07057905745223	27.088217141381012
-Y	32768	2048	16	80	c16_i32768_o2048	"completed	rc=0	duration_s=2827"	80	2803.750278225998	0.02853321161348862	934.9762781507951	58.436017384424694		122899.37849756316	210673.82780143176	213.8925572432094	261.5382891697284
-P	32768	4096	1	5	c1_i32768_o4096	"completed	rc=0	duration_s=655"	5	620.6336467729998	0.008056282520288137	263.98826562480167	32.99853320310021		13185.259522197885	13320.523270551348	27.091213466128522	27.119225907115602
-P	32768	4096	16	80	c16_i32768_o4096	"completed	rc=0	duration_s=4598"	80	4574.679185062007	0.017487565086799776	573.032532764255	71.62906659553188		122919.97263983685	210664.48323895675	193.40684656185272	217.6030520283007
-Y	65536	128	1	5	c1_i65536_o128	"failed	rc=137	duration_s=33	reason=out_of_memory"	0									
-Y	65536	128	8	40	c8_i65536_o128	"failed	rc=137	duration_s=29	reason=out_of_memory"	0									
-Y	65536	256	1	5	c1_i65536_o256	"failed	rc=137	duration_s=54	reason=out_of_memory"	0									
-Y	65536	256	8	40	c8_i65536_o256	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-Y	65536	512	1	5	c1_i65536_o512	"failed	rc=137	duration_s=33	reason=out_of_memory"	0									
-Y	65536	512	8	40	c8_i65536_o512	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-Y	65536	1024	1	5	c1_i65536_o1024	"failed	rc=137	duration_s=34	reason=out_of_memory"	0									
-Y	65536	1024	8	40	c8_i65536_o1024	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-P	65536	2048	1	5	c1_i65536_o2048	"failed	rc=137	duration_s=33	reason=out_of_memory"	0									
-P	65536	2048	8	40	c8_i65536_o2048	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-Y	131072	128	1	5	c1_i131072_o128	"failed	rc=137	duration_s=34	reason=out_of_memory"	0									
-Y	131072	128	4	20	c4_i131072_o128	"failed	rc=137	duration_s=34	reason=out_of_memory"	0									
-Y	131072	256	1	5	c1_i131072_o256	"failed	rc=137	duration_s=33	reason=out_of_memory"	0									
-Y	131072	256	4	20	c4_i131072_o256	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-Y	131072	512	1	5	c1_i131072_o512	"failed	rc=137	duration_s=33	reason=out_of_memory"	0									
-Y	131072	512	4	20	c4_i131072_o512	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									
-P	131072	1024	1	5	c1_i131072_o1024	"failed	rc=1	duration_s=28	reason=out_of_memory"	0									
-P	131072	1024	4	20	c4_i131072_o1024	"skipped	reason=strictly_above_failed_boundary	boundary=c1_i32768_o128"	0									

experiments/p800/glm52_w4a8_sglang_tp8_single_node/run_bench.sh

@@ -1,43 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')}"
-RESULT_DIR="${RUN_DIR}/results/${RUN_ID}"
-PYTHON="/root/miniconda/envs/python310_torch25_cuda/bin/python"
-
-mkdir -p "${RESULT_DIR}"
-curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null
-
-docker inspect "${CONTAINER}" >"${RESULT_DIR}/container_inspect.json"
-docker logs "${CONTAINER}" >"${RESULT_DIR}/server_before_bench.log" 2>&1
-xpu-smi >"${RESULT_DIR}/xpu_before_bench.txt"
-
-docker exec "${CONTAINER}" env \
-  HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_DATASETS_OFFLINE=1 \
-  "${PYTHON}" -m sglang.bench_serving \
-  --backend sglang \
-  --host 127.0.0.1 \
-  --port "${PORT}" \
-  --dataset-name sharegpt \
-  --dataset-path /workspace/ShareGPT_V3_unfiltered_cleaned_split.json \
-  --model /models \
-  --served-model-name GLM-5.2-w4a8 \
-  --tokenizer /models \
-  --num-prompts 64 \
-  --sharegpt-output-len 128 \
-  --sharegpt-context-len 32768 \
-  --request-rate inf \
-  --max-concurrency 8 \
-  --warmup-requests 2 \
-  --seed 1 \
-  --output-file "/workspace/run/results/${RUN_ID}/bench_sharegpt_c8_o128.jsonl" \
-  --output-details \
-  2>&1 | tee "${RESULT_DIR}/bench_console.log"
-
-docker logs "${CONTAINER}" >"${RESULT_DIR}/server_after_bench.log" 2>&1
-xpu-smi >"${RESULT_DIR}/xpu_after_bench.txt"
-printf '%s\n' "${RESULT_DIR}"
-

experiments/p800/glm52_w4a8_sglang_tp8_single_node/run_concurrency_sweep.sh

@@ -1,46 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')-sharegpt-sweep}"
-RESULT_DIR="${RUN_DIR}/results/${RUN_ID}"
-PYTHON="/root/miniconda/envs/python310_torch25_cuda/bin/python"
-CONCURRENCIES="${CONCURRENCIES:-1 2 4 8 16 32}"
-NUM_PROMPTS="${NUM_PROMPTS:-64}"
-OUTPUT_LEN="${OUTPUT_LEN:-128}"
-
-mkdir -p "${RESULT_DIR}"
-curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null
-docker inspect "${CONTAINER}" >"${RESULT_DIR}/container_inspect.json"
-xpu-smi >"${RESULT_DIR}/xpu_before_sweep.txt"
-
-for concurrency in ${CONCURRENCIES}; do
-  echo "===== concurrency=${concurrency} =====" | tee -a "${RESULT_DIR}/sweep.log"
-  docker exec "${CONTAINER}" env \
-    HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_DATASETS_OFFLINE=1 \
-    "${PYTHON}" -m sglang.bench_serving \
-    --backend sglang \
-    --host 127.0.0.1 \
-    --port "${PORT}" \
-    --dataset-name sharegpt \
-    --dataset-path /workspace/ShareGPT_V3_unfiltered_cleaned_split.json \
-    --model /models \
-    --served-model-name GLM-5.2-w4a8 \
-    --tokenizer /models \
-    --num-prompts "${NUM_PROMPTS}" \
-    --sharegpt-output-len "${OUTPUT_LEN}" \
-    --sharegpt-context-len 32768 \
-    --request-rate inf \
-    --max-concurrency "${concurrency}" \
-    --warmup-requests 2 \
-    --seed 1 \
-    --output-file "/workspace/run/results/${RUN_ID}/bench_sharegpt_c${concurrency}_o${OUTPUT_LEN}.jsonl" \
-    --output-details \
-    2>&1 | tee "${RESULT_DIR}/bench_c${concurrency}.log" | tee -a "${RESULT_DIR}/sweep.log"
-done
-
-docker logs "${CONTAINER}" >"${RESULT_DIR}/server_after_sweep.log" 2>&1
-xpu-smi >"${RESULT_DIR}/xpu_after_sweep.txt"
-printf '%s\n' "${RESULT_DIR}"

experiments/p800/glm52_w4a8_sglang_tp8_single_node/run_full_matrix.sh

@@ -1,234 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-REFERENCE_DIR="/data1/wxy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix"
-RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')-full-matrix}"
-RESULT_DIR="${RUN_DIR}/results/${RUN_ID}"
-RAW_DIR="${RESULT_DIR}/raw_outputs"
-LOG_DIR="${RESULT_DIR}/logs"
-PYTHON="/root/miniconda/envs/python310_torch25_cuda/bin/python"
-SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-28800}"
-MATRIX_MODE="${MATRIX_MODE:-Y+P}"
-SERVICE_RECOVERY_TIMEOUT_S="${SERVICE_RECOVERY_TIMEOUT_S:-900}"
-FAILURE_DIR="${RESULT_DIR}/failures"
-FAILURE_BOUNDARIES_FILE="${RESULT_DIR}/failure_boundaries.tsv"
-
-mkdir -p "${RAW_DIR}" "${LOG_DIR}" "${FAILURE_DIR}"
-exec > >(tee -a "${RESULT_DIR}/orchestrator.log") 2>&1
-
-ensure_service() {
-  if curl --fail --silent --show-error --max-time 10 \
-      "http://127.0.0.1:${PORT}/health" >/dev/null 2>&1; then
-    return 0
-  fi
-
-  if [[ "$(docker inspect -f '{{.State.Running}}' "${CONTAINER}" 2>/dev/null || true)" == "true" ]]; then
-    echo "service_recovery=restart container=${CONTAINER}"
-    if ! docker restart "${CONTAINER}" >/dev/null; then
-      echo "service_recovery=recreate reason=restart_failed"
-      bash "${RUN_DIR}/start_glm52_w4a8.sh" >/dev/null
-    fi
-  else
-    echo "service_recovery=start container=${CONTAINER}"
-    if ! docker start "${CONTAINER}" >/dev/null; then
-      echo "service_recovery=recreate reason=start_failed"
-      bash "${RUN_DIR}/start_glm52_w4a8.sh" >/dev/null
-    fi
-  fi
-
-  local deadline=$(( $(date +%s) + SERVICE_RECOVERY_TIMEOUT_S ))
-  until curl --fail --silent --show-error --max-time 10 \
-      "http://127.0.0.1:${PORT}/health" >/dev/null 2>&1; do
-    if (( $(date +%s) >= deadline )); then
-      echo "service_recovery_failed timeout_s=${SERVICE_RECOVERY_TIMEOUT_S}"
-      return 1
-    fi
-    if [[ "$(docker inspect -f '{{.State.Running}}' "${CONTAINER}" 2>/dev/null || true)" != "true" ]]; then
-      echo "service_recovery_failed container_exited"
-      return 1
-    fi
-    sleep 10
-  done
-  echo "service_recovery=healthy container=${CONTAINER}"
-}
-
-add_failure_boundary() {
-  local failed_isl="$1" failed_osl="$2" failed_concurrency="$3" failed_scenario="$4" reason="$5"
-  touch "${FAILURE_BOUNDARIES_FILE}"
-  if ! awk -F '\t' -v i="${failed_isl}" -v o="${failed_osl}" -v c="${failed_concurrency}" \
-      '$1 == i && $2 == o && $3 == c { found=1 } END { exit !found }' \
-      "${FAILURE_BOUNDARIES_FILE}"; then
-    printf '%s\t%s\t%s\t%s\t%s\n' \
-      "${failed_isl}" "${failed_osl}" "${failed_concurrency}" "${failed_scenario}" "${reason}" \
-      >>"${FAILURE_BOUNDARIES_FILE}"
-  fi
-}
-
-strictly_above_failed_boundary() {
-  local candidate_isl="$1" candidate_osl="$2" candidate_concurrency="$3"
-  local failed_isl failed_osl failed_concurrency failed_scenario reason
-  [[ -s "${FAILURE_BOUNDARIES_FILE}" ]] || return 1
-  while IFS=$'\t' read -r failed_isl failed_osl failed_concurrency failed_scenario reason; do
-    if (( candidate_isl > failed_isl && candidate_osl > failed_osl && candidate_concurrency > failed_concurrency )); then
-      printf '%s' "${failed_scenario}"
-      return 0
-    fi
-  done <"${FAILURE_BOUNDARIES_FILE}"
-  return 1
-}
-
-failure_reason_from_log() {
-  local failure_log="$1"
-  if grep -q "Decode out of memory" "${failure_log}" 2>/dev/null; then
-    printf '%s' "kv_cache_decode_oom"
-  elif grep -qi "launch timed out and was terminated" "${failure_log}" 2>/dev/null; then
-    printf '%s' "xpu_kernel_launch_timeout"
-  elif grep -qiE "out of memory|OOM" "${failure_log}" 2>/dev/null; then
-    printf '%s' "out_of_memory"
-  elif grep -qiE "Xid|KL_XID|device.*error" "${failure_log}" 2>/dev/null; then
-    printf '%s' "xpu_device_error"
-  elif grep -qiE "watchdog|timeout" "${failure_log}" 2>/dev/null; then
-    printf '%s' "timeout_or_watchdog"
-  else
-    printf '%s' "bench_or_service_failure"
-  fi
-}
-
-echo "run_id=${RUN_ID} matrix_mode=${MATRIX_MODE} timeout=${SCENARIO_TIMEOUT_S}"
-ensure_service
-docker inspect "${CONTAINER}" >"${RESULT_DIR}/container_inspect.resume.$(date +%Y%m%d-%H%M%S).json"
-if [[ ! -s "${RESULT_DIR}/scenarios.tsv" ]]; then
-  cp "${REFERENCE_DIR}/matrix.json" "${RESULT_DIR}/matrix.json"
-  python3 "${REFERENCE_DIR}/generate_scenarios.py" \
-    --matrix "${RESULT_DIR}/matrix.json" \
-    --mode "${MATRIX_MODE}" \
-    >"${RESULT_DIR}/scenarios.tsv"
-fi
-
-scenario_count="$(tail -n +2 "${RESULT_DIR}/scenarios.tsv" | wc -l)"
-echo "scenario_count=${scenario_count}"
-
-(
-  while true; do
-    date -u '+timestamp=%Y-%m-%dT%H:%M:%SZ'
-    xpu-smi
-    sleep 60
-  done
-) >>"${RESULT_DIR}/xpu_monitor.log" 2>&1 &
-monitor_pid=$!
-trap 'kill "${monitor_pid}" >/dev/null 2>&1 || true' EXIT
-
-index=0
-tail -n +2 "${RESULT_DIR}/scenarios.tsv" | while IFS=$'\t' read -r mark isl osl concurrency num_prompts; do
-  index=$((index + 1))
-  scenario="c${concurrency}_i${isl}_o${osl}"
-  output_file="${RAW_DIR}/${scenario}.jsonl"
-  status_file="${LOG_DIR}/${scenario}.status"
-  detail_log="${LOG_DIR}/${scenario}.log"
-
-  if [[ -s "${output_file}" ]] && python3 "${RUN_DIR}/summarize_matrix.py" \
-      --check-completed "${output_file}" "${num_prompts}"; then
-    echo "[$index/$scenario_count] skip completed ${scenario}"
-    continue
-  fi
-
-  if [[ -s "${status_file}" ]] && grep -qE '^(failed|timeout)' "${status_file}"; then
-    existing_reason="previous_failure"
-    if [[ "${scenario}" == "c64_i8192_o2048" ]]; then
-      existing_reason="kv_cache_decode_oom"
-    fi
-    printf '%s\n' "${existing_reason}" >"${FAILURE_DIR}/${scenario}.reason"
-    add_failure_boundary "${isl}" "${osl}" "${concurrency}" "${scenario}" "${existing_reason}"
-    echo "[$index/$scenario_count] skip recorded failure ${scenario} reason=${existing_reason}"
-    continue
-  fi
-
-  failed_boundary=""
-  if failed_boundary="$(strictly_above_failed_boundary "${isl}" "${osl}" "${concurrency}")"; then
-    printf 'skipped\treason=strictly_above_failed_boundary\tboundary=%s\n' \
-      "${failed_boundary}" >"${status_file}"
-    echo "[$index/$scenario_count] skip strictly-above ${scenario} boundary=${failed_boundary}"
-    continue
-  fi
-
-  if ! ensure_service; then
-    echo "service_unrecoverable_before=${scenario}" | tee "${status_file}"
-    exit 1
-  fi
-
-  echo "[$index/$scenario_count] start mark=${mark} isl=${isl} osl=${osl} concurrency=${concurrency} prompts=${num_prompts}"
-  started="$(date +%s)"
-  started_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
-  bench_rc=0
-  timeout "${SCENARIO_TIMEOUT_S}" docker exec "${CONTAINER}" env \
-    HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_DATASETS_OFFLINE=1 \
-    "${PYTHON}" -m sglang.bench_serving \
-    --backend sglang \
-    --host 127.0.0.1 \
-    --port "${PORT}" \
-    --dataset-name random \
-    --dataset-path /workspace/ShareGPT_V3_unfiltered_cleaned_split.json \
-    --model /models \
-    --served-model-name GLM-5.2-w4a8 \
-    --tokenizer /models \
-    --random-input-len "${isl}" \
-    --random-output-len "${osl}" \
-    --random-range-ratio 1.0 \
-    --num-prompts "${num_prompts}" \
-    --max-concurrency "${concurrency}" \
-    --request-rate 10000 \
-    --seed 1 \
-    --flush-cache \
-    --output-file "/workspace/run/results/${RUN_ID}/raw_outputs/${scenario}.jsonl" \
-    --output-details \
-    >"${detail_log}" 2>&1 || bench_rc=$?
-  duration=$(( $(date +%s) - started ))
-
-  if [[ "${bench_rc}" -eq 0 ]]; then
-    printf 'completed\trc=0\tduration_s=%s\n' "${duration}" >"${status_file}"
-    echo "[$index/$scenario_count] completed ${scenario} duration_s=${duration}"
-  elif [[ "${bench_rc}" -eq 124 ]]; then
-    failure_log="${FAILURE_DIR}/${scenario}.server.log"
-    docker logs --since "${started_iso}" "${CONTAINER}" >"${failure_log}" 2>&1 || true
-    reason="$(failure_reason_from_log "${failure_log}")"
-    printf 'timeout\trc=124\tduration_s=%s\treason=%s\n' "${duration}" "${reason}" >"${status_file}"
-    printf '%s\n' "${reason}" >"${FAILURE_DIR}/${scenario}.reason"
-    add_failure_boundary "${isl}" "${osl}" "${concurrency}" "${scenario}" "${reason}"
-    echo "[$index/$scenario_count] timeout ${scenario} duration_s=${duration} reason=${reason}"
-  else
-    failure_log="${FAILURE_DIR}/${scenario}.server.log"
-    docker logs --since "${started_iso}" "${CONTAINER}" >"${failure_log}" 2>&1 || true
-    reason="$(failure_reason_from_log "${failure_log}")"
-    printf 'failed\trc=%s\tduration_s=%s\treason=%s\n' \
-      "${bench_rc}" "${duration}" "${reason}" >"${status_file}"
-    printf '%s\n' "${reason}" >"${FAILURE_DIR}/${scenario}.reason"
-    add_failure_boundary "${isl}" "${osl}" "${concurrency}" "${scenario}" "${reason}"
-    echo "[$index/$scenario_count] failed ${scenario} rc=${bench_rc} duration_s=${duration} reason=${reason}"
-  fi
-
-  python3 "${RUN_DIR}/summarize_matrix.py" \
-    --scenarios "${RESULT_DIR}/scenarios.tsv" \
-    --raw-dir "${RAW_DIR}" \
-    --log-dir "${LOG_DIR}" \
-    --output "${RESULT_DIR}/summary.tsv"
-
-  if ! curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null; then
-    echo "service_unhealthy_after=${scenario}"
-    docker logs --tail 500 "${CONTAINER}" >"${FAILURE_DIR}/${scenario}.server_tail.log" 2>&1 || true
-    if ! ensure_service; then
-      echo "service_unrecoverable_after=${scenario}"
-      exit 1
-    fi
-  fi
-done
-
-python3 "${RUN_DIR}/summarize_matrix.py" \
-  --scenarios "${RESULT_DIR}/scenarios.tsv" \
-  --raw-dir "${RAW_DIR}" \
-  --log-dir "${LOG_DIR}" \
-  --output "${RESULT_DIR}/summary.tsv"
-docker logs "${CONTAINER}" >"${RESULT_DIR}/server_after_matrix.log" 2>&1
-echo "matrix_complete result_dir=${RESULT_DIR}"

experiments/p800/glm52_w4a8_sglang_tp8_single_node/run_full_matrix.sh.before-resume-20260818

@@ -1,118 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-REFERENCE_DIR="/data1/wxy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix"
-RUN_ID="${RUN_ID:-$(date '+%Y%m%d-%H%M%S')-full-matrix}"
-RESULT_DIR="${RUN_DIR}/results/${RUN_ID}"
-RAW_DIR="${RESULT_DIR}/raw_outputs"
-LOG_DIR="${RESULT_DIR}/logs"
-PYTHON="/root/miniconda/envs/python310_torch25_cuda/bin/python"
-SCENARIO_TIMEOUT_S="${SCENARIO_TIMEOUT_S:-28800}"
-MATRIX_MODE="${MATRIX_MODE:-Y+P}"
-
-mkdir -p "${RAW_DIR}" "${LOG_DIR}"
-exec > >(tee -a "${RESULT_DIR}/orchestrator.log") 2>&1
-
-echo "run_id=${RUN_ID} matrix_mode=${MATRIX_MODE} timeout=${SCENARIO_TIMEOUT_S}"
-curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null
-docker inspect "${CONTAINER}" >"${RESULT_DIR}/container_inspect.json"
-cp "${REFERENCE_DIR}/matrix.json" "${RESULT_DIR}/matrix.json"
-python3 "${REFERENCE_DIR}/generate_scenarios.py" \
-  --matrix "${RESULT_DIR}/matrix.json" \
-  --mode "${MATRIX_MODE}" \
-  >"${RESULT_DIR}/scenarios.tsv"
-
-scenario_count="$(tail -n +2 "${RESULT_DIR}/scenarios.tsv" | wc -l)"
-echo "scenario_count=${scenario_count}"
-
-(
-  while true; do
-    date -u '+timestamp=%Y-%m-%dT%H:%M:%SZ'
-    xpu-smi
-    sleep 60
-  done
-) >"${RESULT_DIR}/xpu_monitor.log" 2>&1 &
-monitor_pid=$!
-trap 'kill "${monitor_pid}" >/dev/null 2>&1 || true' EXIT
-
-index=0
-tail -n +2 "${RESULT_DIR}/scenarios.tsv" | while IFS=$'\t' read -r mark isl osl concurrency num_prompts; do
-  index=$((index + 1))
-  scenario="c${concurrency}_i${isl}_o${osl}"
-  output_file="${RAW_DIR}/${scenario}.jsonl"
-  status_file="${LOG_DIR}/${scenario}.status"
-  detail_log="${LOG_DIR}/${scenario}.log"
-
-  if [[ -s "${output_file}" ]] && python3 "${RUN_DIR}/summarize_matrix.py" \
-      --check-completed "${output_file}" "${num_prompts}"; then
-    echo "[$index/$scenario_count] skip completed ${scenario}"
-    continue
-  fi
-
-  if ! curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null; then
-    echo "service_unhealthy_before=${scenario}" | tee "${status_file}"
-    exit 1
-  fi
-
-  echo "[$index/$scenario_count] start mark=${mark} isl=${isl} osl=${osl} concurrency=${concurrency} prompts=${num_prompts}"
-  started="$(date +%s)"
-  bench_rc=0
-  timeout "${SCENARIO_TIMEOUT_S}" docker exec "${CONTAINER}" env \
-    HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_DATASETS_OFFLINE=1 \
-    "${PYTHON}" -m sglang.bench_serving \
-    --backend sglang \
-    --host 127.0.0.1 \
-    --port "${PORT}" \
-    --dataset-name random \
-    --dataset-path /workspace/ShareGPT_V3_unfiltered_cleaned_split.json \
-    --model /models \
-    --served-model-name GLM-5.2-w4a8 \
-    --tokenizer /models \
-    --random-input-len "${isl}" \
-    --random-output-len "${osl}" \
-    --random-range-ratio 1.0 \
-    --num-prompts "${num_prompts}" \
-    --max-concurrency "${concurrency}" \
-    --request-rate 10000 \
-    --seed 1 \
-    --flush-cache \
-    --output-file "/workspace/run/results/${RUN_ID}/raw_outputs/${scenario}.jsonl" \
-    --output-details \
-    >"${detail_log}" 2>&1 || bench_rc=$?
-  duration=$(( $(date +%s) - started ))
-
-  if [[ "${bench_rc}" -eq 0 ]]; then
-    printf 'completed\trc=0\tduration_s=%s\n' "${duration}" >"${status_file}"
-    echo "[$index/$scenario_count] completed ${scenario} duration_s=${duration}"
-  elif [[ "${bench_rc}" -eq 124 ]]; then
-    printf 'timeout\trc=124\tduration_s=%s\n' "${duration}" >"${status_file}"
-    echo "[$index/$scenario_count] timeout ${scenario} duration_s=${duration}"
-  else
-    printf 'failed\trc=%s\tduration_s=%s\n' "${bench_rc}" "${duration}" >"${status_file}"
-    echo "[$index/$scenario_count] failed ${scenario} rc=${bench_rc} duration_s=${duration}"
-  fi
-
-  python3 "${RUN_DIR}/summarize_matrix.py" \
-    --scenarios "${RESULT_DIR}/scenarios.tsv" \
-    --raw-dir "${RAW_DIR}" \
-    --log-dir "${LOG_DIR}" \
-    --output "${RESULT_DIR}/summary.tsv"
-
-  if ! curl --fail --silent --show-error --max-time 10 "http://127.0.0.1:${PORT}/health" >/dev/null; then
-    echo "service_unhealthy_after=${scenario}"
-    docker logs --tail 500 "${CONTAINER}" >"${RESULT_DIR}/server_failure.log" 2>&1 || true
-    exit 1
-  fi
-done
-
-python3 "${RUN_DIR}/summarize_matrix.py" \
-  --scenarios "${RESULT_DIR}/scenarios.tsv" \
-  --raw-dir "${RAW_DIR}" \
-  --log-dir "${LOG_DIR}" \
-  --output "${RESULT_DIR}/summary.tsv"
-docker logs "${CONTAINER}" >"${RESULT_DIR}/server_after_matrix.log" 2>&1
-echo "matrix_complete result_dir=${RESULT_DIR}"
-

experiments/p800/glm52_w4a8_sglang_tp8_single_node/start_glm52_w4a8.sh

@@ -1,65 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-IMAGE="iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746"
-MODEL_DIR="/data2/GLM-5.2-w4a8"
-DATASET="/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-
-test -f "${MODEL_DIR}/config.json"
-test -f "${DATASET}"
-mkdir -p "${RUN_DIR}/results"
-
-docker rm -f "${CONTAINER}" >/dev/null 2>&1 || true
-
-docker run -d \
-  --init \
-  --name "${CONTAINER}" \
-  --privileged \
-  --ulimit core=-1 \
-  --security-opt seccomp=unconfined \
-  --net=host --uts=host --ipc=host \
-  --shm-size=256g \
-  -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e CUDA_DISABLE_PRINTF=1 \
-  -e GLOO_SOCKET_TIMEOUT=60000 \
-  -v "${MODEL_DIR}:/models:ro" \
-  -v "${DATASET}:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro" \
-  -v "${RUN_DIR}:/workspace/run" \
-  -w /workspace/run \
-  "${IMAGE}" \
-  bash -lc "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server \
-    --host 0.0.0.0 \
-    --port ${PORT} \
-    --model-path /models \
-    --served-model-name GLM-5.2-w4a8 \
-    --attention-backend nsa \
-    --nsa-prefill klxdsa \
-    --nsa-decode klxdsa \
-    --prefill-attention-backend klxdsa \
-    --decode-attention-backend klxdsa \
-    --trust-remote-code \
-    --disable-custom-all-reduce \
-    --disable-shared-experts-fusion \
-    --chunked-prefill-size 32768 \
-    --page-size 64 \
-    --mem-fraction-static 0.82 \
-    --max-prefill-tokens 32768 \
-    --max-running-requests 64 \
-    --tp-size 8 \
-    --ep-size 8 \
-    --quantization w4a8_int4 \
-    --kv-cache-dtype int8 \
-    --cuda-graph-max-bs 8 \
-    --disable-cuda-graph-padding \
-    --watchdog-timeout 3000000 \
-    --tool-call-parser glm47 \
-    --reasoning-parser glm45 \
-    --json-model-override-args '{\"index_topk_freq\": 4}' \
-    --allow-auto-truncate \
-    --enable-metrics"
-
-echo "container=${CONTAINER} port=${PORT}"

experiments/p800/glm52_w4a8_sglang_tp8_single_node/start_glm52_w4a8.sh.before-init-20260818

@@ -1,64 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-IMAGE="iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260715_5746"
-MODEL_DIR="/data2/GLM-5.2-w4a8"
-DATASET="/data1/wxy/sskj/datasets/ShareGPT_V3_unfiltered_cleaned_split.json"
-RUN_DIR="/data1/wxy/sskj/experiments/p800/glm52_w4a8_sglang_tp8_smoke"
-CONTAINER="glm52-w4a8-tp8"
-PORT="${PORT:-30000}"
-
-test -f "${MODEL_DIR}/config.json"
-test -f "${DATASET}"
-mkdir -p "${RUN_DIR}/results"
-
-docker rm -f "${CONTAINER}" >/dev/null 2>&1 || true
-
-docker run -d \
-  --name "${CONTAINER}" \
-  --privileged \
-  --ulimit core=-1 \
-  --security-opt seccomp=unconfined \
-  --net=host --uts=host --ipc=host \
-  --shm-size=256g \
-  -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e XPU_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e CUDA_DISABLE_PRINTF=1 \
-  -e GLOO_SOCKET_TIMEOUT=60000 \
-  -v "${MODEL_DIR}:/models:ro" \
-  -v "${DATASET}:/workspace/ShareGPT_V3_unfiltered_cleaned_split.json:ro" \
-  -v "${RUN_DIR}:/workspace/run" \
-  -w /workspace/run \
-  "${IMAGE}" \
-  bash -lc "exec /root/miniconda/envs/python310_torch25_cuda/bin/python -m sglang.launch_server \
-    --host 0.0.0.0 \
-    --port ${PORT} \
-    --model-path /models \
-    --served-model-name GLM-5.2-w4a8 \
-    --attention-backend nsa \
-    --nsa-prefill klxdsa \
-    --nsa-decode klxdsa \
-    --prefill-attention-backend klxdsa \
-    --decode-attention-backend klxdsa \
-    --trust-remote-code \
-    --disable-custom-all-reduce \
-    --disable-shared-experts-fusion \
-    --chunked-prefill-size 32768 \
-    --page-size 64 \
-    --mem-fraction-static 0.82 \
-    --max-prefill-tokens 32768 \
-    --max-running-requests 64 \
-    --tp-size 8 \
-    --ep-size 8 \
-    --quantization w4a8_int4 \
-    --kv-cache-dtype int8 \
-    --cuda-graph-max-bs 8 \
-    --disable-cuda-graph-padding \
-    --watchdog-timeout 3000000 \
-    --tool-call-parser glm47 \
-    --reasoning-parser glm45 \
-    --json-model-override-args '{\"index_topk_freq\": 4}' \
-    --allow-auto-truncate \
-    --enable-metrics"
-
-echo "container=${CONTAINER} port=${PORT}"

experiments/p800/glm52_w4a8_sglang_tp8_single_node/summarize_matrix.py

@@ -1,71 +0,0 @@
-#!/usr/bin/env python3
-import argparse
-import csv
-import json
-from pathlib import Path
-
-
-def read_first_json(path: Path):
-    try:
-        with path.open("r", encoding="utf-8") as stream:
-            for line in stream:
-                line = line.strip()
-                if line:
-                    return json.loads(line)
-    except (OSError, json.JSONDecodeError):
-        return None
-    return None
-
-
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--check-completed", nargs=2, metavar=("FILE", "EXPECTED"))
-    parser.add_argument("--scenarios")
-    parser.add_argument("--raw-dir")
-    parser.add_argument("--log-dir")
-    parser.add_argument("--output")
-    args = parser.parse_args()
-
-    if args.check_completed:
-        path, expected = args.check_completed
-        data = read_first_json(Path(path)) or {}
-        raise SystemExit(0 if int(data.get("completed", 0)) >= int(expected) else 1)
-
-    required = (args.scenarios, args.raw_dir, args.log_dir, args.output)
-    if not all(required):
-        parser.error("summary mode requires --scenarios, --raw-dir, --log-dir, and --output")
-
-    raw_dir = Path(args.raw_dir)
-    log_dir = Path(args.log_dir)
-    rows = []
-    with Path(args.scenarios).open("r", encoding="utf-8", newline="") as stream:
-        for scenario in csv.DictReader(stream, delimiter="\t"):
-            name = f"c{scenario['concurrency']}_i{scenario['input_len']}_o{scenario['output_len']}"
-            data = read_first_json(raw_dir / f"{name}.jsonl") or {}
-            status_path = log_dir / f"{name}.status"
-            status = status_path.read_text(encoding="utf-8").strip() if status_path.exists() else "pending"
-            rows.append({
-                **scenario,
-                "scenario": name,
-                "status": status,
-                "completed": data.get("completed", 0),
-                "duration_s": data.get("duration", ""),
-                "request_throughput": data.get("request_throughput", ""),
-                "input_throughput": data.get("input_throughput", ""),
-                "output_throughput": data.get("output_throughput", ""),
-                "total_token_throughput": data.get("total_token_throughput", ""),
-                "mean_ttft_ms": data.get("mean_ttft_ms", ""),
-                "p99_ttft_ms": data.get("p99_ttft_ms", ""),
-                "mean_tpot_ms": data.get("mean_tpot_ms", ""),
-                "p99_tpot_ms": data.get("p99_tpot_ms", ""),
-            })
-
-    fields = list(rows[0]) if rows else []
-    with Path(args.output).open("w", encoding="utf-8", newline="") as stream:
-        writer = csv.DictWriter(stream, fieldnames=fields, delimiter="\t")
-        writer.writeheader()
-        writer.writerows(rows)
-
-
-if __name__ == "__main__":
-    main()

experiments/pro6000/glm53_nvfp4_128k_capacity_topology/README.md

@@ -1,135 +0,0 @@
-# GLM-5.3-NVFP4 128k 低并发容量扩容与拓扑选型实验报告 — 2026-09-09 @ 174.1.60.8
-
-## 一、结论速览
-
-**优胜配置 = TP2PP4 nomtp**（radix/hicache 保持开启、`--context-length 1048576`、cu13 栈 9 挂载、memfrac 0.85、chunk 8192）。
-
-| 维度 | 60.5 现役（TP8+EAGLE） | 优胜配置（TP2PP4 nomtp） | 提升 |
-|---|---|---|---|
-| KV 池（token） | 276,864 | **909,632** | 3.29× |
-| i128k 并发上限 | c2（c3 起排队） | **c6**（c7 起排队，实测） | 3× |
-| 单条请求上限 | 264k（ctx 270,336 参数所限） | **~909k**（900k 实跑通过） | 3.4× |
-| c4 输入吞吐 | 3,228 tok/s | **5,907 tok/s** | 1.83× |
-| c4 输出吞吐 | 12.6 tok/s | **23.1 tok/s** | 1.83× |
-| c4 TTFT p50 | 119.6s | **43.4s** | −64% |
-| c4 e2e p50 | 161.3s | **88.8s** | −45% |
-| 90% 命中真实流量形态（c4） | — | 输入 **17,625 tok/s**、TTFT 7.3s | — |
-
-- 质量门 7/7 通过（GSM8K×5 + 中文推理 + tool call，PP4+radix+hicache+parser 新组合无 correctness 问题）。
-- **60.8 已按用户决定部署优胜配置留役**（restart=unless-stopped，替代 r37）。
-- **60.5 交付 `deploy_glm53_605_v2.sh`（只交未执行）**，原 `deploy_glm53_605.sh` 原样留盘即回滚路径。
-
-## 二、问题诊断（60.5 生产日志实证）
-
-现役容器 `glm53-nvfp4`（TP8 + EAGLE 4/1/5，memfrac 0.90，fp8 KV，hicache-ratio 3，`--context-length 270336`）：
-
-1. **"单条最高 256k" 不是模型限制**：模型原生 `max_position_embeddings = 1,048,576`（1M）。上限来自部署参数 `--context-length 270336`，且 KV 池 276,864 也刚好只装得下一条 26.4 万请求。
-2. **排队是纯容量算术**：单条 16~22 万 token 请求占池 0.58~0.90；近期一万行日志 ~13% 的 decode 步有 `#queue-req>0`，09-08 有一次池满 retract。日志实例：一条 173k（95% 命中）请求在 160k 请求 decode 期间排队无法准入。
-3. **命中不省并发容量**：radix/hicache 只加速重复前缀的 prefill；不同文档的并发请求各自全额驻留 KV，照样排队。60.5 真实流量 12万~22万 token、命中率 90%+，仍然撞容量墙。
-4. **TP8 调参无解（结构性）**：每 token KV 57.3KB（MLA 44.9 + DSA indexer ~10），fp8_e4m3 已是断言封死的硬顶（FP4 启动即 AssertionError）；TP8 全系调参上限 314,944（60.6 实测 0.89+hicache4）< c3×128k 需求 394,752。
-
-## 三、机理：为什么只有 PP 拓扑能解
-
-GLM-5.3 是 MLA/DSA 家族，**KV 在 TP 组内每卡全量复制、只按 PP 分层切**：TP8 每卡存全部 78 层 KV（15.85GB→276,864 token）；PP2 每卡存半数层（池×2）；PP4 每卡存 1/4 层（池×4，实际因 hicache/图开销打折）。**KV 池容量 ≈ PP 度数倍增，与 TP 度数无关**。这就是 TP4PP2≈2.37×、TP2PP4≈3.29×（实测）的来源。另：MTP/EAGLE 的 draft KV + verify 图额外吃池（r37 同 memfrac 下 589,696→384,960，−35%）。
-
-## 四、实验方法
-
-- **平台**：60.8（8×RTX 6000D 85.6GB，与 60.5 同型）；四配置臂各部署后跑同窗口。
-- **场景**：i131072 / o512 / cc∈{1,2,3,4}，nreq=8/点，PG19 真实语料（bench_corpus.py，input_ids 直发 /generate，temp 0，ignore_eos，stream）。
-- **公平性协议**：每点独立语料窗口（cc1→base 10M，cc2→11,048,576，cc3→12,097,152，cc4→13,145,728），**所有臂用同一映射=同文本**；每点前 `POST /flush_cache`（实测连 host 层一起清，复跑命中率 0.0）；每臂先做一次不计分的 131k 形状预热（吸收内核首触成本，实测不做会污染首点：16k 首跑 1653 vs 复跑 4526 tok/s）。
-- **指标口径**（沿用 rev20 双场景协议）：输入吞吐 = 总输入 token/墙钟；输出吞吐 = 服务端 completion_tokens 总和/墙钟；TTFT = 发出→首个流式响应（含排队）；命中率从 TP0 Prefill 日志核算。queue-req 日志用于排队取证。
-- **配置臂**：
-  - **arm0 r37**：60.8 在役原样（TP4PP2+EAGLE 3/1/4+verify 图，memfrac 0.88，池 384,960）
-  - **arm1 A-mirror**：60.5 现役逐参数复刻（旧镜像 20260828，池 276,864，ctx 270,336）
-  - **arm2 B'**：TP4PP2 nomtp，memfrac 0.88，ctx 524,288（池 589,696）
-  - **arm3 TP2PP4**：TP2PP4 nomtp，memfrac 0.85，ctx 1048576，radix/hicache 开，含 glm45/glm47 parser（池 909,632）
-
-## 五、主扫数据（i128k/o512，冷缓存）
-
-**输入吞吐（tok/s）**
-
-| cc | A-mirror（池276,864） | r37（池384,960） | B'（池589,696） | TP2PP4（池909,632） |
-|---|---|---|---|---|
-| 1 | 3,026 | **4,110** | 3,737 | 3,262 |
-| 2 | 3,142 | **4,754** | 4,619 | 4,635 |
-| 3 | 3,214 | 5,016 | 4,843 | **5,017** |
-| 4 | 3,228 | 5,061 | 5,256 | **5,907** |
-
-**输出吞吐（tok/s）**
-
-| cc | A-mirror | r37 | B' | TP2PP4 |
-|---|---|---|---|---|
-| 1 | 11.8 | **16.1** | 14.6 | 12.7 |
-| 2 | 12.3 | **18.6** | 18.0 | 18.1 |
-| 3 | 12.6 | 19.6 | 18.9 | **19.6** |
-| 4 | 12.6 | 19.8 | 20.5 | **23.1** |
-
-**TTFT p50 / max（秒）**（排队签名 = p50 跳升整请求时长倍数）
-
-| cc | A-mirror | r37 | B' | TP2PP4 |
-|---|---|---|---|---|
-| 1 | 37.8 | 21.1 | 21.0 | **15.5** |
-| 2 | 37.8 / 76.9 | 21.1 / 41.7 | 41.3 | 29.0 |
-| 3 | **77.2** / 120.3 | 43.6 / 73.8 | 41.3 / 61.6 | **29.6** / 43.4 |
-| 4 | **119.6** / 156.4 | 72.2 / 93.9 | 61.6 / 81.9 | **43.4** / 57.3 |
-
-**e2e p50 / max（秒）**
-
-| cc | A-mirror | r37 | B' | TP2PP4 |
-|---|---|---|---|---|
-| 1 | 43.8 | **31.7** | 35.0 | 40.2 |
-| 2 | 83.7 / 121.6 | **55.3** / 76.6 | 56.8 | 56.5 |
-| 3 | 121.7 / 163.3 | **75.2** / 105.1 | 80.0 | 75.8 |
-| 4 | 161.3 / 201.8 | 103.3 / 125.2 | 99.7 | **88.8** |
-
-**排队证据与机理注解**：
-- A-mirror：池只容 2 条并发，c3/c4 排队（TTFT p50 77→120s）；c4 输入吞吐被排队锁死在 3,228（prefill 带宽根本没用满）。
-- r37：池 384,960 同样只容 2 条 + 1 条错峰补位，c3/c4 排队（TTFT p50 43.6→72.2s；日志 518 行 queue-req>0）。**60.8 原在役配置在本场景也不合格。**
-- B'：c3/c4 全并发准入、零容量排队（稳态 usage 精确停在 0.22/0.45/0.67/0.89；e2e p50≈max 波内同步）。日志中的 queue-req 行是 chunk 调度/波间 radix 逐出瞬态（秒级），非容量排队——判据：TTFT 缩放=纯 prefill 带宽分摊（cc2 p50=max=41.3≈2×21.0）。
-- TP2PP4：同上零容量排队，且 TTFT 全场最优（4 级流水 prefill 重叠最深：cc1 TTFT 15.5s、串行 prefill 等效 ~8.5k tok/s）；代价是 TPOT 最慢（cc1 48.4ms vs B' 27.3 vs r37 20.7 vs A 12.0），c1 短输出场景 e2e 吃亏（40.2s），c2 起被并发摊平、c3/c4 反超。
-- MTP/EAGLE 观察：accept 随并发上升（r37 2.91→3.64，A-mirror 3.31→4.26，verify batch 越大接受越高）；但 MTP 的 35% 池代价在本容量场景不划算——r37 全程吞吐被 B'/TP2PP4 压制或打平。
-
-## 六、优胜者（TP2PP4）附加验证
-
-| 验证项 | 结果 |
-|---|---|
-| 质量门 | **7/7**（GSM8K 72/3/60/63/10 + 鸡兔同笼 23 + tool call get_weather北京） |
-| 512k 单条（523,776+512 真实语料） | ✅ TTFT 88.4s，e2e 114.4s，TPOT 50.9ms（旧配置直接拒绝） |
-| 900k 单条（900,000+512） | ✅ TTFT 210.6s，e2e 237.8s —— **单条上限 ~909k 实证**（池减 512 后的余量） |
-| cc5 | ✅ in 5,747 / out 22.5，e2e p50≈max 107s，零容量排队 |
-| cc6 | ✅ in 5,849 / out 22.9，e2e p50≈max 122.7s，零容量排队 —— **并发上限 = c6** |
-| cc7 | ⚠️ 第 7 条排队（TTFT max 138.7s、e2e max 179.7s）—— 边界与池算术吻合（7×131,584=921,088 > 909,632） |
-| 90% 命中 c4（60.5 真实流量形态，实测命中 0.8999） | in **17,625** / out 68.9 tok/s，TTFT p50 **7.3s**，e2e 29.9s —— radix 去重+hicache 对重复查询流量再放大 ~3× |
-
-## 七、方案对比与推荐
-
-| 方案 | 池 | c4 in/out | 容量定位 | 判定 |
-|---|---|---|---|---|
-| A（TP8+EAGLE，60.5 现役） | 276,864 | 3,228/12.6 | c2、单条 264k | 本场景被全面支配，淘汰 |
-| r37（TP4PP2+MTP，60.8 昨日在役） | 384,960 | 5,061/19.8 | c2、单条 ~380k | c3 起排队，容量不合格；仅 c1 最优 |
-| B'（TP4PP2 nomtp） | 589,696 | 5,256/20.5 | c4、单条 ~588k（512k 单条会占 89% 池、堵死并发） | 强力备选：16k 短文本场景历史成绩好 |
-| **TP2PP4 nomtp（优胜）** | **909,632** | **5,907/23.1** | **c6、单条 ~909k、512k 单条+2 并发共存** | 用户需求（512k 单条+长上下文为主）下的正解 |
-
-**推荐**：60.5 采用 TP2PP4 nomtp（`deploy_glm53_605_v2.sh`，参数与 60.8 现役完全一致）。决策依据：c3/c4/c5/c6 吞吐全场第一 + 唯一满足"512k 单条与并发共存" + 质量门 7/7 + TTFT 全档最优。若 60.5 未来 16k 短文本流量占比显著上升，再评估切 B'（其 s1 短文本历史成绩比 TP2PP4 好 27~48%）。
-
-**留观旋钮**（未验证，报告只记录不推荐）：memfrac 0.85→0.88 或可再抬池（PP0 空闲 22GB 最大的 stage 不均匀提示有空间）；TP2PP4+MTP 可修 c1 短输出短板（池约降至 ~59 万，恰为 B' 水平）；hicache-ratio 3→4 扩 host 前缀池（60.6 在 TP8 验证过）。
-
-**r37 角色变化说明**：r37 的 MTP 优势区间是 i16k/cc≤16（09-09 判决 +6%@cc16），本次让位给容量优先的 TP2PP4 是按用户明确选择执行；若 60.8 未来主要服务短文本低并发，可用 deploy_ppmtp_r37.sh mtp 模式一键切回。
-
-## 八、已证伪 / 排除项（本轮+引用前判）
-
-- TP8 任何调参：fp8 KV 断言封顶 + 上限 314,944 < c3 需求 394,752（60.6 数据）。
-- r37 现役直接顶上：池 384,960，c3 差 1 万 token 仍排队（本轮实测）。
-- MTP/EAGLE 换 KV 池：draft+verify 图 −35% 池，本场景不划算。
-- HiSparse 超池驻留：本 nightly 自旋不可用（60.6 前判）。
-- TP1PP8：池 2M 但吞吐 −12~33%（60.2 前判），仅当需要 >c6 且能接受慢时再议。
-- PD 分离/DP/EP：前判劣化或不可用，与本问题正交。
-
-## 九、资产与复现
-
-- **60.8**：容器 glm53-nvfp4:30000 = 优胜配置，restart=unless-stopped。启动命令 = `bash /root/deploy_ppmtp_r37.sh '--tp 2 --pp-size 4 --disable-overlap-schedule --max-prefill-tokens 16384 --disable-custom-all-reduce --context-length 1048576' nomtp 8192 0.85`。实验日志全量：`/root/bench_logs/128kexp/`（26 点 SUMMARY 已提取为 all_summaries.json）。
-- **60.5**：交付 `deploy_glm53_605_v2.sh`（内嵌完整 docker run+前置检查+显存归零等待+健康门+回滚提示）；回滚 = `bash /root/deploy_glm53_605.sh`。执行前置检查会列出缺失的补丁/镜像清单（60.8 /root 均有）。
-- **压测复现**：`bash /root/arm_runner.sh <arm>`（c1-4 四点+形状预热）、`bash /root/val_runner.sh <arm>`（512k/900k/cc5-7/hit90）；语料窗口映射与 flush 协议见第四节；质量门 `bash /root/quality_gate_605.sh`。
-- **仓库归档**：`experiments/pro6000/glm53_nvfp4_128k_capacity_topology/`（README=本报告、scripts、results/20260909）；profile `deploy/profiles/pro6000/glm53_nvfp4_pro6000_sglang_tp2pp4_hicache.env`；CURRENT.md 已更新 60.8 行。
-
-*报告与数据：ZCode 实验 2026-09-09；压测窗口 run-id 9501-9507/9601-9604/9990-9991；全部冷缓存口径。*

experiments/pro6000/glm53_nvfp4_128k_capacity_topology/results/20260909/md5s.txt

@@ -1,3 +0,0 @@
-e782a8767d4478711757669f83adb64b *scripts/arm_runner.sh
-f6f5a49937ac6e8a0157ba1eb11b8ee7 *scripts/deploy_glm53_605_v2.sh
-8eb75693f01f6055b15ecc3d62feff08 *scripts/val_runner.sh

experiments/pro6000/glm53_nvfp4_128k_capacity_topology/scripts/arm_runner.sh

@@ -1,36 +0,0 @@
-#!/bin/bash
-# 128k capacity experiment - per-arm bench runner (i131072/o512, cc 1-4, nreq 8)
-# usage: nohup bash arm_runner.sh <arm-name> >/dev/null 2>&1 &
-# per-cc-point corpus windows are distinct but the mapping is identical for every arm:
-#   cc1 base 10000000, cc2 11048576, cc3 12097152, cc4 13145728 (all in consumed space, flushed)
-ARM=${1:?arm name required}
-URL=http://127.0.0.1:30000
-LOGD=/root/bench_logs/128kexp
-mkdir -p $LOGD
-echo "$(date +%T) [$ARM] start" >> $LOGD/${ARM}_progress.log
-
-# wait until server idle (no running/queued requests in recent logs)
-for i in $(seq 1 60); do
-  R=$(docker logs --since 20s glm53-nvfp4 2>&1 | grep -cE 'running-req: [1-9]|queue-req: [1-9]')
-  if [ "$R" -eq 0 ]; then break; fi
-  sleep 10
-done
-
-flush() { curl -s -X POST $URL/flush_cache >/dev/null; sleep 2; }
-
-# shape warmup (not scored): one 128k prefill to absorb kernel first-touch cost
-flush
-python3 /root/bench_corpus.py --input-len 131072 --output-len 64 --shared-frac 0 \
-  --concurrency 1 --num-requests 1 --run-id 9991 --pool-override 17000000 \
-  > $LOGD/${ARM}_warmup.log 2>&1
-echo "$(date +%T) [$ARM] warmup done" >> $LOGD/${ARM}_progress.log
-
-declare -A BASE=( [1]=10000000 [2]=11048576 [3]=12097152 [4]=13145728 )
-for CC in 1 2 3 4; do
-  flush
-  python3 /root/bench_corpus.py --input-len 131072 --output-len 512 --shared-frac 0 \
-    --concurrency $CC --num-requests 8 --run-id 950${CC} --pool-override ${BASE[$CC]} \
-    > $LOGD/${ARM}_cc${CC}.log 2>&1
-  echo "$(date +%T) [$ARM] cc$CC done rc=$?" >> $LOGD/${ARM}_progress.log
-done
-echo "ALL-DONE" >> $LOGD/${ARM}_progress.log

experiments/pro6000/glm53_nvfp4_128k_capacity_topology/scripts/deploy_glm53_605_v2.sh

@@ -1,129 +0,0 @@
-#!/bin/bash
-# deploy_glm53_605_v2.sh — GLM-5.3-NVFP4 60.5 生产容量扩容配置（2026-09-09 交付）
-#
-# 目的：解决 60.5 现役 TP8+EAGLE 配置的 KV 池容量问题（池 276,864 token）：
-#   - 单条请求上限 264k（--context-length 270336 所致，模型原生支持 1M）
-#   - 并发>1 且总 token>~26 万时排队（i128k 下 c3 起必然排队）
-# 本配置 = 2026-09-09 在 60.8 同型机（8×RTX 6000D）四臂对拍优胜者：
-#   TP2 PP4 nomtp + radix/hicache 开 + ctx 1M + cu13 栈 9 挂载
-#   实测池 909,632 token（3.29×）：i128k 并发上限 c2→c6，单条上限 264k→909k，
-#   c4 输入/输出吞吐 5907/23.1 tok/s（现役口径 3228/12.6 的 1.83×/1.83×），
-#   质量门 7/7。完整数据：experiments/pro6000/glm53_nvfp4_128k_capacity_topology/
-#
-# ⚠️ 运维提示：
-#   1) 在维护窗口执行：会移除现役 glm53-nvfp4 容器，停机约 10-15 分钟（权重加载+健康）。
-#   2) 回滚 = 原脚本原样留盘：bash /root/deploy_glm53_605.sh（TP8+EAGLE 原配置）。
-#   3) 与 60.8 现役（本配置已留役）一致，脚本可互相对拍 md5。
-#
-# 用法: bash deploy_glm53_605_v2.sh          # 部署到 30000 端口
-#       ROLLBACK_ONLY=1 bash deploy_glm53_605_v2.sh   # 仅回滚到原配置
-
-set -u
-IMAGE=lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729
-PORT=30000
-
-# ---- 前置检查（缺什么给什么清单，不猜） ----
-MISSING=0
-if [ -z "$(docker images -q $IMAGE 2>/dev/null)" ]; then
-  echo "[MISS] 镜像不存在: $IMAGE"
-  echo "       拉取: docker pull $IMAGE"
-  echo "       或从 60.8 导: ssh 60.8 'docker save $IMAGE | gzip' | gunzip | docker load"
-  MISSING=1
-fi
-MOUNTS=(
-  "/root/sglang_patch2/layer_setup.py"
-  "/root/sglang_patch2/validation_hook.py"
-  "/root/eagle_worker_v2_mask.py"
-  "/root/sglang_patch2/eagle_worker_common.py"
-  "/root/sglang_patch2/deepseek_nextn.py"
-  "/root/scheduler_pp_mixin_r35.py"
-  "/root/request_receiver_degloo.py"
-  "/root/sglang_patch2/deepseek_v2.py"
-  "/root/decode_cuda_graph_runner_fix.py"
-)
-for f in "${MOUNTS[@]}"; do
-  if [ ! -f "$f" ]; then
-    echo "[MISS] 补丁文件缺失: $f"
-    MISSING=1
-  fi
-done
-if [ "$MISSING" = "1" ]; then
-  echo "[STOP] 上述文件在 60.8 /root 均有（含 sglang_patch2/ 子目录），scp 全部补齐后重跑本脚本。"
-  exit 1
-fi
-
-if [ "${ROLLBACK_ONLY:-0}" = "1" ]; then
-  echo "[rollback] 恢复 60.5 原生产配置（TP8+EAGLE）..."
-  exec bash /root/deploy_glm53_605.sh
-fi
-
-# ---- 移除旧容器（等待显存归零，纪律：重部署前必等显存<1500MiB） ----
-docker update --restart=no glm53-nvfp4 >/dev/null 2>&1
-for i in 1 2 3 4 5 6 7 8; do
-  docker rm -f glm53-nvfp4 >/dev/null 2>&1
-  docker ps -a --format '{{.Names}}' | grep -q '^glm53-nvfp4$' || break
-  sleep 5
-done
-if docker ps -a --format '{{.Names}}' | grep -q '^glm53-nvfp4$'; then
-  echo "[ERROR] 旧容器删不掉（zombie 时重试即可）"; exit 1
-fi
-echo "[wait] 等待显存释放 (<1500MiB/卡)..."
-for i in $(seq 1 60); do
-  MAXMI=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | sort -rn | head -1)
-  [ "$MAXMI" -lt 1500 ] && break
-  sleep 5
-done
-echo "[wait] 显存就绪 (${MAXMI}MiB max)"
-
-# ---- 部署优胜配置（与 60.8 实验臂 arm3_TP2PP4 完全一致） ----
-docker run -d --name glm53-nvfp4 --gpus all --shm-size 64g --ipc=host --cap-add SYS_PTRACE \
-  -v /root/sglang_patch2/layer_setup.py:/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner_components/layer_setup.py:ro \
-  -v /root/sglang_patch2/validation_hook.py:/sgl-workspace/sglang/python/sglang/srt/arg_groups/validation_hook.py:ro \
-  -v /root/eagle_worker_v2_mask.py:/sgl-workspace/sglang/python/sglang/srt/speculative/eagle_worker_v2.py:ro \
-  -v /root/sglang_patch2/eagle_worker_common.py:/sgl-workspace/sglang/python/sglang/srt/speculative/eagle_worker_common.py:ro \
-  -v /root/sglang_patch2/deepseek_nextn.py:/sgl-workspace/sglang/python/sglang/srt/models/deepseek_nextn.py:ro \
-  -v /root/scheduler_pp_mixin_r35.py:/sgl-workspace/sglang/python/sglang/srt/managers/scheduler_pp_mixin.py:ro \
-  -v /root/request_receiver_degloo.py:/sgl-workspace/sglang/python/sglang/srt/managers/scheduler_components/request_receiver.py:ro \
-  -v /root/sglang_patch2/deepseek_v2.py:/sgl-workspace/sglang/python/sglang/srt/models/deepseek_v2.py:ro \
-  -v /root/decode_cuda_graph_runner_fix.py:/sgl-workspace/sglang/python/sglang/srt/model_executor/runner/decode_cuda_graph_runner.py:ro \
-  -e SGLANG_PP_DEGLOO=1 -e SGLANG_PP_SPEC_FORCE_EAGER_DRAFT=1 -e SGLANG_PP_FORCE_EAGER_VERIFY=0 -e SGLANG_PP_SPEC_DEBUG=0 \
-  --restart no -p ${PORT}:${PORT} \
-  -v /data/hf_models:/data/hf_models \
-  $IMAGE \
-  python3 -m sglang.launch_server \
-    --model-path /data/hf_models/GLM-5.3-NVFP4 \
-    --tp 8 \
-    --mem-fraction-static 0.85 \
-    --max-running-requests 16 \
-    --chunked-prefill-size 8192 \
-    --disable-shared-experts-fusion \
-    --moe-runner-backend flashinfer_cutlass \
-    --disable-flashinfer-autotune \
-    --reasoning-parser glm45 --tool-call-parser glm47 \
-    --enable-hierarchical-cache --hicache-ratio 3 \
-    --tp 2 --pp-size 4 --disable-overlap-schedule --max-prefill-tokens 16384 \
-    --disable-custom-all-reduce --context-length 1048576 \
-    --host 0.0.0.0 --port ${PORT}
-
-echo "[deploy] TP2PP4-nomtp 容量扩容配置启动，等待健康（最长 1800s）..."
-for i in $(seq 10 10 1800); do
-  code=$(curl -s -o /dev/null -m3 -w '%{http_code}' http://127.0.0.1:${PORT}/health 2>/dev/null)
-  if [ "$code" = "200" ]; then
-    echo "[OK] healthy after ${i}s"
-    docker update --restart=unless-stopped glm53-nvfp4 >/dev/null && echo "[OK] restart=unless-stopped 已设置"
-    POOL=$(docker logs glm53-nvfp4 2>&1 | grep -m1 -oE 'max_total_num_tokens=[0-9]+')
-    echo "[INFO] ${POOL}  (60.8 同型机实测 909632；若显著低于此值请停下核查)"
-    echo "[NEXT] 质量门: bash /root/quality_gate_605.sh   期望 PASS=7 FAIL=0"
-    echo "[NEXT] 容量冒烟: 300k 单条请求应能正常完成（旧配置会直接拒绝）"
-    exit 0
-  fi
-  if ! docker ps --format '{{.Names}}' | grep -q '^glm53-nvfp4$'; then
-    echo "[DIED] 容器启动失败，最近错误："
-    docker logs --tail 40 glm53-nvfp4 2>&1 | grep -iE 'error|assert|not support' | tail -8
-    echo "[回滚] bash /root/deploy_glm53_605.sh"
-    exit 1
-  fi
-  sleep 10
-done
-echo "[TIMEOUT] 健康等待超时；回滚: bash /root/deploy_glm53_605.sh"
-exit 1

experiments/pro6000/glm53_nvfp4_128k_capacity_topology/scripts/val_runner.sh

@@ -1,48 +0,0 @@
-#!/bin/bash
-# winner validation runner (TP2PP4): big singles, cc boundary, 90%-hit point
-# usage: nohup bash val_runner.sh <arm-name> >/dev/null 2>&1 &
-ARM=${1:?arm name required}
-URL=http://127.0.0.1:30000
-LOGD=/root/bench_logs/128kexp
-echo "$(date +%T) [$ARM] val start" >> $LOGD/${ARM}_progress.log
-
-for i in $(seq 1 60); do
-  R=$(docker logs --since 20s glm53-nvfp4 2>&1 | grep -cE 'running-req: [1-9]|queue-req: [1-9]')
-  if [ "$R" -eq 0 ]; then break; fi
-  sleep 10
-done
-
-flush() { curl -s -X POST $URL/flush_cache >/dev/null; sleep 2; }
-
-# 1) 512k single (real corpus): 523776+512 = 524288
-flush
-python3 /root/bench_corpus.py --input-len 523776 --output-len 512 --shared-frac 0 \
-  --concurrency 1 --num-requests 1 --run-id 9601 --pool-override 10000000 \
-  > $LOGD/${ARM}_single512k.log 2>&1
-echo "$(date +%T) [$ARM] single512k rc=$?" >> $LOGD/${ARM}_progress.log
-
-# 2) 900k single (pool ceiling probe): 900000+512 = 900512 vs pool 909632
-flush
-python3 /root/bench_corpus.py --input-len 900000 --output-len 512 --shared-frac 0 \
-  --concurrency 1 --num-requests 1 --run-id 9602 --pool-override 10000000 \
-  > $LOGD/${ARM}_single900k.log 2>&1
-echo "$(date +%T) [$ARM] single900k rc=$?" >> $LOGD/${ARM}_progress.log
-
-# 3-5) cc boundary: 5 / 6 fit, 7 exceeds pool (921088 > 909632) -> expect queuing
-declare -A BASEB=( [5]=10000000 [6]=11048576 [7]=12097152 )
-for CC in 5 6 7; do
-  flush
-  python3 /root/bench_corpus.py --input-len 131072 --output-len 512 --shared-frac 0 \
-    --concurrency $CC --num-requests 8 --run-id 950${CC} --pool-override ${BASEB[$CC]} \
-    > $LOGD/${ARM}_cc${CC}.log 2>&1
-  echo "$(date +%T) [$ARM] cc$CC rc=$?" >> $LOGD/${ARM}_progress.log
-done
-
-# 6) 90%-hit cc4 point (mirrors 60.5 real traffic: shared 117968 + unique 13104x8)
-flush
-python3 /root/bench_corpus.py --input-len 131072 --output-len 512 --shared-frac 0.9 \
-  --concurrency 4 --num-requests 8 --run-id 9604 --pool-override 15000000 \
-  > $LOGD/${ARM}_hit90_cc4.log 2>&1
-echo "$(date +%T) [$ARM] hit90_cc4 rc=$?" >> $LOGD/${ARM}_progress.log
-
-echo "VAL-ALL-DONE" >> $LOGD/${ARM}_progress.log

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/README.md

@@ -1,82 +0,0 @@
-# GLM-5.3-NVFP4 双方案 B300 对标场景矩阵压测 — 60.8（6000D）
-
-日期：2026-09-10 ｜ 机器：174.1.60.8（8×RTX 6000D，96GB GDDR7，无 NVLink）
-模型：GLM-5.3-NVFP4（modelopt）｜ 镜像：`nightly-dev-20260828-daf63171`（两臂同）
-对标基线：飞书《GLM 5.3 ｜ SGLang ｜ Low-latency & High-Throughput 测试结果》（B300 报告，wiki UPB2w4Y5yi65qwkMxJJcZko5nUc）
-完整报告：本目录 `REPORT.md`（= 飞书发布版 A7V3wZTQeifCB4krdi6cA834nW9；两臂高并发复测后整文更新，正文一律采用复测值）
-
-## 目标
-
-在 6000D 上复刻 B300 报告的全部场景（主场景 16K→512、4.1 短输入、4.2 长输出、5.1 长上下文、5.2 边界），对两套在役部署方案各跑一遍完整矩阵，产出对齐 B300 8 章结构的对标报告。两臂各完成一轮当日调参复测：E7b 初测暴露 MRR16 + decode 图 bs1-8 的高并发掉图断崖，按用户决策调参为 MRR64 + 图桶 1-64（`deploy_glm53_e7b_hicc.sh`，其余配方逐字不变）复测三场景 C=8/16/32/64 共 10 点；TP2PP4 初测的 C=64 点受 MRR48 封顶（48 活跃+16 排队；decode 图栈默认覆盖 bs≤256、本臂从未掉图），按用户决策调参为 MRR64（`deploy_glm53_pp4_mrr64.sh`，逐 token diff 仅 MRR 一处）复测三场景 C=32/C=64 共 6 点两轮（pass-1 + v2 有序版，全部点位对齐 ≤1.8%），另以全新实例跑原版 MRR48 的 4.1 C=32 单点做归因对照。**报告正文一律采用复测值**（C=1 与 5.1/5.2 点受池上限约束、与调参无关，沿用初测值）。测后 60.8 在役服务（TP4PP2@0.90）原容器恢复，当日三轮拆台均按 rename→start 流程保全恢复并逐轮验证。
-
-## 实验臂
-
-| 臂 | 方案 | 关键配置 | 质量门 |
-|---|---|---|---|
-| tp2pp4 | **D 生产口径**（deploy_glm53_pp4.sh） | TP2PP4、mem0.85、MRR48、cps16384、radix 关、KV fp8_e4m3 池 1,040,384、无投机、index_topk_freq=4（=原生默认，恒等）、ctx 1,048,576 | 6/7（仅 tool-call：无 parser，历史已知） |
-| e7b | **TP8+EAGLE3+AR 初测**（deploy_glm53_607_exp.sh + CAR 补丁注入） | TP8、EAGLE 4/1/5、mem0.90、MRR16、cps8192、radix 开+hicache×3、KV fp8_e4m3 GPU 池 276,480、decode 图 bs1-8、ctx 270,336、custom-AR 1stage（`SGLANG_CUSTOM_ALLREDUCE_ALGO=1stage`，8 rank `SSKJ_CAR_PATCH_ACTIVE` 验证） | 7/7 |
-| e7b64 | **TP8+EAGLE3+AR 高并发复测**（deploy_glm53_e7b_hicc.sh，= e7b 仅改 MRR 16→64 + decode 图桶 1-64） | 其余配方与 e7b 逐字一致；图捕获后 avail 6.11 GB/卡，KV 池 276,480 不变 | 7/7 |
-| pp4mrr64 | **D 生产口径高并发复测**（deploy_glm53_pp4_mrr64.sh，= tp2pp4 仅改 MRR 48→64） | 其余配方与 tp2pp4 逐字一致；decode 图栈默认覆盖 bs≤256（无需扩图）；KV 池 1,040,384 不变；16K 场景活跃上限转为池 ~59 条 | 6/7（v2 实例复验，仅 tool-call 已知项） |
-| mrr48ctl | **归因对照**（原版 deploy_glm53_pp4.sh 全新实例，4.1 C=32 单点） | = tp2pp4 逐字一致（MRR48）；用于分解 4.1 C=32 锚点 −17%（−3% 实例新鲜度 + −14% MRR64 配置代价） | 未跑（配置同 tp2pp4 已验） |
-
-## 场景矩阵与并发档位（用户裁决收敛：16K 封 64、64K/128K 封 8/4）
-
-| B300 章节 | 场景 | 并发档位 | TP2PP4 活跃上限 | E7b 活跃上限 |
-|---|---|---|---|---|
-| §3 主场景 | 16K→512 | 1/8/16/32/64 | ~59（池 1,040,384÷17.4K；复测臂 MRR64 超配，c64 3 次回退） | 16（池 276,480÷17.4K；复测臂 MRR64 不再是约束） |
-| §4.1 | 1K→128 | 1/8/32/64 | 64（复测臂 MRR；初测 48） | 64（复测臂 MRR；初测 16） |
-| §4.2 | 1K→4K | 1/8/32/64 | 64（复测臂 MRR；初测 48） | ~52（池；复测臂 c64 有 12 条排队；初测 c64 中止 rc=143，复测已补齐） |
-| §5.1 | 64K→512 | 1/4/8 | 15（池） | 4（池） |
-| §5.1 | 128K→512 | 1/2/4 | 7（池） | 2（池） |
-| §5.2 | 256K→1 | 1/2/3 | 3（池） | 1（池贴边） |
-| §5.2 | 512K→1 | 1 | 1 | 结构性不可（ctx） |
-| §5.2 | 896K→1（代 B300"约1M"） | 1 | 1 | 结构性不可（ctx） |
-
-测量协议：冷缓存（shared-frac 0）+ 每点 flush + 服务端命中核验 ≤0.01（超限重试一次）；nreq=max(8, 2×cc)；P95 nearest-rank 对齐 B300；语料耗尽（21.23M/21.30M）下用回收窗口（`--pool-override` 基址映射见 REPORT 附录 A；复测臂为全新容器实例，同基址文本对 hicache 宿主层重新成为处女文本，命中核验全 0.0）。有效测量点 57 个（tp2pp4 24 + pp4mrr64 复测 12 + mrr48ctl 对照 1 + e7b/e7b64 合计 20），全点 0 OOM；唯一 retraction 点 = TP2PP4 主场景 C=64（MRR64 对 16K 池超配，两轮各 3 次，其余全点 0）。
-
-## 判决速览（详见 REPORT.md，两臂数值均为复测值）
-
-- **掉图断崖已修复（E7b 复测核心判决）**：初测 MRR16 + 图 bs1-8 把 E7b 窗口封死 C≤8（16K c16 TPOT 298ms 断崖）；调参 MRR64 + 图桶 1-64 后 C=16 TPOT 降至 228.8ms，16K c16/32/64 输出 92.1/97.6/99.6 tok/s（较初测 +18~33%），窗口扩到 C=64。
-- **TP2PP4 C=64 档全面提升（pp4mrr64 复测核心判决）**：初测 C=64 损失全部来自 MRR48（48 活跃+16 排队；decode 图默认覆盖 bs≤256 从未掉图）。MRR64 后：4.1 c64 输出 341→433（+27%）、4.2 c64 482→572（+19%）、16K c64 TTFT 135→89s（−34%）；4.1/4.2 c64 TTFT 从 20.0/337.9s 塌缩到 7.2/7.3s。16K c64 到达池顶（活跃 ~59、吞吐 −5%、3 次回退为确定性行为，两轮一致）。
-- **MRR64 的 −14% 代价（1K 短输入 c32）与归因对照**：4.1 c32 锚点 276→228（−17%），全新实例 MRR48 对照 266.42 分解 = −3% 实例新鲜度 + −14% MRR64 配置本身（逐请求 TPOT p50 87→102ms 均匀抬高，三实例可复现、机制未定位）；4.2 c32 −6%、16K c32 持平。MRR 按负载形态选：1K 短输入为主且并发 ≤48 可保持 MRR48。
-- **分界负载形态化**：prefill 密集（16K 主场景）仍 TP2PP4 占优——c64 out 201 vs 99.6 = 2.0×，E7b 输出被 prefill 墙（chunk8192+TP8 无 PP 流水）封在 ~100 tok/s 平台；短输入 c32 两方案打平（228 vs 229）、c64 TP2PP4 1.6×（433 vs 272）且 TTFT/TPOT 双优。
-- **decode 密集（1K→4K）E7b 全档反超**：c8/32/64 = 402/676/**826.5 tok/s**，c64 为全场最高输出吞吐（超 TP2PP4 同点 572 达 45%），TPOT 85.1ms 仍优、TTFT 被 TP2PP4 MRR64 反超（7.30 vs 12.18s）；初测排队断崖（c32 TTFT 305s）消除为 6.19s。
-- **E7b C=1 优势不变**：16K out 49.6 vs 16.7=3.0×、TPOT 13.7 vs 50.3ms=3.7×；1K→4K out 135 vs 20.3=6.7×、TPOT 8.7ms。
-- **TP2PP4 甜点 c16+**：16K 爬至 c64 池顶（活跃 ~59 封顶）；边界 256K/512K/896K 只有 TP2PP4 可达（input 7,050/5,662/4,153 tok/s）；E7b ctx 270,336 结构性封顶。
-- **DSA 复现**：C=1 TPOT 对上下文不敏感（50.3/50.0/49.7ms @16/64/128K），并发才是驱动（c8: 70.9→161.8ms）。
-- **vs B300**：定性结构复现（LL/HT 分野一致），绝对差 4-5×，边界 prefill 差距收窄至 ~2×；主场景分界本机更靠前（C8 vs C64-128，容量上限而非算力），decode 密集场景调参后 E7b 全档无交叉（B300 报告未呈现该形态）。
-
-## 关键坑位（复测必读）
-
-1. **hicache 宿主层陷阱**：256K prewarm KV 占池 94.8% 触发宿主层下放，`flush_cache` 清不掉宿主层 → 同文本测量命中 0.9998。冷缓存复测**必须换该实例从未发过的文本**（本战役 E7b 256K C=1 用窗口 9,900,000 重测达标；e7b64 复测臂为全新容器实例，沿用同基址窗口即满足该条件，10 点命中全 0.0）。
-2. 语料已耗尽：回收窗口复用仅在 flush+命中核验协议下有效；TP2PP4 臂 radix 本来就关，零污染。
-3. 在役保全流程：`docker stop` → `docker rename glm53-nvfp4 glm53-nvfp4-insvc`（必须先改名，E7b 部署脚本会 rm -f 同名容器）→ 测毕 `rename` 回 + `start`。docker stop/rm 偶发 "zombie PID" 报错是收尾边界现象，容器终态 exited(137)、显存归零，稍等重试即可。
-
-## 资产与 md5 台账（60.8 执行件 = 本目录 = 60.7 原件 三方一致）
-
-```
-1e34dd8d813ccb7ac8243e8d1573a49e  bench_corpus_v2.py        (scripts/)
-4c126d067d33b5ea27c268f37561634c  extract_summary.py        (scripts/)
-5892b44610b2ce61f533f1721105625e  run_b300_matrix.sh        (scripts/)
-165db732fa3237a80a4d53a963a128e7  deploy_glm53_e7b_hicc.sh  (scripts/，E7b 高并发版：MRR64+图桶1-64，其余与 607_exp 逐字一致)
-fe46da2eb22f5ae23eb963bb9d467922  run_retest_e7b64.sh       (scripts/，10 点复测驱动，run-id 96xx)
-376bedfd42fa29ba9de8bfdf0c67c3b2  gen_retest_compare.py     (scripts/，复测前后对比表生成器)
-533440efc04e89bf5bbec3e0efbb35ed  deploy_glm53_pp4_mrr64.sh (scripts/，TP2PP4 高并发版：仅 MRR 48→64，其余与 deploy_glm53_pp4.sh 逐 token 一致)
-c9b6361d8679c41e59aced298e0f50bf  run_retest_pp4mrr64.sh    (scripts/，pass-1 6 点复测驱动，run-id 966x)
-7d8578169b2e7d3d1c9d461f53c44e87  run_retest_pp4mrr64_v2.sh (scripts/，v2 有序版：回退点 16K c64 排末位，run-id 967x，正文采用值)
-889c242859e46a0d41240a818f754bd1  launch_v2_after_pass1.sh  (scripts/，pass-1→拆台→重部署→QG→v2 全自动衔接 wrapper)
-1ff8a21f12a79a3b2aaa8c2dafb4af32  control_mrr48_41c32.sh    (scripts/，MRR48 归因对照 + 自动链接在役恢复)
-20cea216bce9aeda4ec4ecea4569a3de  gen_retest_compare_pp4.py (scripts/，TP2PP4 复测前后对比表生成器)
-def3c64c5dc19e1d507080e3366c3762  deploy_glm53_pp4.sh       (见 dual_scenario_bench/scripts/，md5 对照一致)
-21db641f1d997e26fcf1ddc97163b87e  deploy_glm53_607_exp.sh   (见 dual_scenario_bench/scripts/，md5 对照一致)
-a8fc9a504c041acc40831a848b4985d8  patches/custom_all_reduce.py
-65a4d22bd87ae343e7d68c4879c14f98  patches/custom_all_reduce_utils.py
-```
-
-gen_report_tables.py 为本地表格生成器（md5 未入台账，60.8 侧执行件同源）。
-
-## 原始数据
-
-- 本目录 `results/{tp2pp4,e7b,e7b64,pp4mrr64}/`：all_results*.jsonl（逐点 SUMMARY + 命中核验）、status*.txt、server_facts*.txt（启动参数+池分配日志摘录）、gpu_inventory_idle/final.csv、vram_timeline.csv（30s 采样全矩阵；csv/log 属 gitignore 中间件，关键事实折叠于 provenance.md）；pp4mrr64/ 含 pass-1、v2（正文采用值）、mrr48ctl（归因对照）三套 jsonl + retest_compare_pp4_v2.md 前后对比表
-- 60.8 侧：`/root/bench_logs/b300eq_tp2pp4_20260910_1138/`、`/root/bench_logs/b300eq_e7b_20260910_1428/`（初测留档基线）、`/root/bench_logs/b300eq_e7b64_20260910_1732/`（E7b 复测，正文采用值）+ `/root/bench_logs/retest_compare.md`（前后对比）、`/root/bench_logs/b300eq_pp4mrr64_20260910_1927/`（TP2PP4 复测 pass-1）、`/root/bench_logs/b300eq_pp4mrr64v2_20260910_2021/`（TP2PP4 复测 v2，正文采用值）+ `/root/bench_logs/retest_compare_pp4_v2.md`、`/root/bench_logs/b300eq_mrr48ctl_20260910_2112/`（MRR48 对照）
-- E7b 256K C=1：all_results.jsonl 同 tag 共 4 条，最后一条为干净重测值（生成器 dict 载入后写覆盖，天然生效）

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/REPORT.md

@@ -1,295 +0,0 @@
-# GLM-5.3-NVFP4 ｜ RTX 6000D ｜ SGLang 双方案 B300 对标场景压测报告
-
-- 测试日期：2026-09-10（单日单机完成两臂；两臂均完成当日调参复测——E7b：MRR 64 + decode 图桶 ≤64，TP2PP4：MRR 48→64——正文一律采用复测值）
-- 测试机：174.1.60.8（6000D，8 卡）
-- 对标基线：飞书《GLM 5.3 ｜ SGLang ｜ Low-latency & High-Throughput 测试结果》（B300 报告，wiki UPB2w4Y5yi65qwkMxJJcZko5nUc）
-- 测后状态：60.8 在役服务（TP4PP2@0.90 口径，KV 池 647,040）已原容器恢复并终验（当日三轮拆台均按 rename→start 流程保全恢复；末轮恢复后 fired up + health 200 + 16K 单发抽测 ok，池 647,040 与显存水位 77.2/82.3 GiB 逐字一致）
-
-## 1. 结论摘要
-
-本轮在单台 8 卡 RTX 6000D 上，用 GLM-5.3-**NVFP4** 完整复刻 B300 报告的场景矩阵，测试了两套在役部署方案。两套方案代表完整部署形态，不是单参数 A/B：**TP2PP4** 为 D 生产口径（吞吐/长上下文形态），**TP8+EAGLE3+AR** 为 E7b 配方（低延迟形态，含 custom allreduce 1stage 补丁）。
-
-- **低并发优先 E7b（TP8+EAGLE3+AR）**：主场景 `16K→512, C=1` 输出 49.6 tok/s、TPOT 13.7 ms，对 TP2PP4（16.7 tok/s、50.3 ms）分别是 **3.0×** 与 **3.7×**；长输出 `1K→4K, C=1` 输出 135 tok/s、TPOT 8.7 ms，对 TP2PP4（20.3、49.8 ms）是 **6.7×** 与 **5.7×**。
-- **prefill 密集场景（16K 输入）高并发优先 TP2PP4**：主场景 C=64 达到 6,419 input tok/s / 201 output tok/s，对 E7b（3,188 / 99.6）为 **2.0×**；短输入 C=32 两方案打平（228 vs 229 tok/s，TP2PP4 的 MRR64 配置代价见下条）、C=64 TP2PP4 拉开（433 vs 272，**+59%**）。
-- **E7b 初测的掉图断崖是配置产物，调参后已消除**：初测 MRR=16 + decode 图仅覆盖 bs 1–8，并发 >8 即掉图（主场景 C=16 TPOT P95 298 ms）。按决策调参为 **MRR=64 + decode 图桶 1–64**（其余配方逐字不变）后复测：C=16 TPOT P95 降至 228.8 ms，16K 场景 C=16/32/64 输出 92.1/97.6/99.6 tok/s（较初测 +18~33%），E7b 可用并发窗口从 C≤8 扩到 **C=64**。
-- **TP2PP4 初测的 C=64 损失全部来自 MRR=48（48 活跃+16 排队），调至 64 后 C=64 档全面提升**：decode 图为栈默认覆盖 bs≤256、本臂从未掉图，MRR 才是唯一瓶颈。MRR64 复测（其余配方逐字不变）：4.1 短输入 C=64 输出 341→433（+27%）、4.2 长输出 C=64 输出 482→572（+19%）、主场景 C=64 TTFT P95 135→89 s（−34%）；4.1/4.2 的 C=64 TTFT P95 分别从 20.0/337.9 s 塌缩到 7.2/7.3 s。
-- **MRR64 的代价与边界**：1K 短输入 C=32 锚点 276→228（−17%；全新实例 MRR48 对照实验定案：−3% 为实例新鲜度、−14% 为 MRR64 配置本身——逐请求 TPOT p50 87→102 ms 均匀抬高，三实例可复现、机制未定位）、1K→4K C=32 −6%、16K C=32 持平（−0.5%）；16K C=64 吞吐 −5%（201 vs 211，池超配 3 次回退）。MRR 按负载形态选，见建议 3。
-- **TP2PP4 甜点在 C=16 之后，MRR64 后主场景 C=64 到达 16K 池顶**：输出吞吐从 C=8 的 101 爬到 C=64 的 201 tok/s（活跃 ~59 条为池所封顶）；其 4.2 长输出 C=64 的 572 tok/s 已被 E7b 反超（826.5），但 16K 主场景仍是本机 prefill 吞吐之王。
-- **decode 密集负载（1K 进、长出）E7b 全并发档最优**：`1K→4K` C=8/32/64 输出 402/676/**826.5 tok/s**——C=64 为全场最高输出吞吐（超 TP2PP4 同点 572 达 45%），TTFT P95 12.2 s、TPOT P95 85.1 ms，EAGLE accept 3.8~4.0。
-- **长上下文与容量边界只有 TP2PP4 可达**：128K C=1 两方案输出打平（11.8 vs 11.9 tok/s）但 TP2PP4 TTFT 减半（18.2 s vs 37.6 s）；256K/512K/896K 边界 E7b 结构性不可测（ctx 270,336 封顶 + KV 池 276,480 贴边），TP2PP4 全部完成（256K C=1/2/3、512K/896K C=1）。
-- **DSA 特性在 6000D 复现**：TP2PP4 C=1 的 TPOT 对上下文长度不敏感（16K/64K/128K = 50.3/50.0/49.7 ms 恒定），并发才是 TPOT 驱动因子（16K 行 C=8→C=64：70.9→261.5 ms，末档含池超配回退的批扰动）。
-- **与 B300 的绝对差距约 4~5×**，边界 prefill 差距收窄到约 2×（256K C=1 input 7,050 vs 17,457；896K 4,153 vs 约1M 行 7,897）。硬件与量化口径不同（B300 报告未写明量化方式），绝对值仅量级可比，两份报告的结构性结论一致（见第 9 章）。
-- 全部有效测量点 **0 OOM**、冷缓存命中核验全部 ≤0.01（E7b 256K C=1 首测触 hicache 宿主层陷阱，用全新文本重测达标，见 5.2 注记；E7b64 复测 10 点、TP2PP4 MRR64 复测两轮 12 点与 MRR48 对照 1 点命中核验全部 0.0）；回退（retraction）仅出现在 TP2PP4 主场景 C=64 一点（MRR64 对 16K 池超配，两轮各 3 次，其余全部点 0 回退）。
-
-## 2. 测试环境与配置
-
-| 项目 | TP2PP4（D 生产口径） | TP8+EAGLE3+AR（E7b 配方） |
-|-|-|-|
-| 硬件 | 单机 8 × NVIDIA RTX 6000D（96 GB GDDR7，nvidia-smi 可见 85,651 MiB/卡） | 同左 |
-| 模型 | GLM-5.3-NVFP4（modelopt 量化，/data/hf_models/GLM-5.3-NVFP4） | 同左 |
-| 镜像 | `nightly-dev-20260828-daf63171` | 同左 |
-| 并行 | TP2 × PP4 | TP8 |
-| 投机解码 | 无 | EAGLE3，num_steps=4，topk=1，draft_tokens=5 |
-| `mem-fraction-static` | 0.85 | 0.90 |
-| 最大活跃请求（MRR） | 64（复测口径；初测 48，见演进注记） | 64 |
-| Chunk Prefill | 16,384 | 8,192 |
-| KV dtype | fp8_e4m3 | fp8_e4m3 |
-| KV 池（服务端实测） | **1,040,384 tokens**（12.6~13.4 GB/rank，无宿主层） | **276,480 tokens GPU**（15.8 GB/rank）+ 分层缓存 hicache×3 宿主层（write_through） |
-| radix cache | 关（`disable_radix_cache=True`） | 开（分层缓存） |
-| 上下文上限 | 1,048,576（config 原生） | 270,336（显存约束下的部署值） |
-| CUDA graph | 常规捕获（decode 图栈默认覆盖 bs≤256，MRR64 全程在图内） | decode/verify 图桶 bs 1–64（1,2,3,4,6,8,12,16,24,32,48,64） |
-| custom allreduce | — | 1stage 补丁注入（`SGLANG_CUSTOM_ALLREDUCE_ALGO=1stage` 环境强制；8 rank `SSKJ_CAR_PATCH_ACTIVE` 日志验证全出现） |
-| `index_topk_freq` | 4（override，等于原生默认，恒等） | 原生默认 4 |
-| 质量门 | 6/7（仅 tool-call 失败：D 口径未配 parser，历史已知；其余全过） | **7/7** |
-
-**E7b 臂配置演进注记**：E7b 初测为 MRR=16 + decode 图桶 bs 1–8，高并发点（C>8）出现 decode 掉图 + MRR 排队双击。按决策将 MRR 调至 64、decode 图桶扩至 1–64（部署脚本 `deploy_glm53_e7b_hicc.sh`，其余配方与在役 E7b 逐字一致），三场景 C=8/16/32/64 共 10 点全部重测，正文一律采用复测值；C=1 各点与 5.1/5.2 长上下文点受 KV 池上限约束（活跃 1~4 条），行为与该调参无关，沿用初测值。C=8 锚点前后偏差 ≤3%（16K 81.4→83.9、1K 152→151、1K→4K 412→402 tok/s），证明两轮环境无漂移。初测原始数据留档于服务器 `b300eq_e7b_20260910_1428/` 与库内 `results/e7b/`。
-
-**TP2PP4 臂配置演进注记**：TP2PP4 初测为 MRR=48，主场景/4.1/4.2 的 C=64 点实际运行为 48 活跃+16 排队（decode 图为栈默认覆盖 bs≤256，本臂从未掉图，MRR 才是 C=64 的活跃上限）。按决策将 MRR 调至 64（部署脚本 `deploy_glm53_pp4_mrr64.sh`，与初测脚本逐 token diff 仅 MRR 一处），三场景 C=32/C=64 共 6 点重测两轮——pass-1 常规顺序，v2 全新实例并把易触发回退的主场景 C=64 排末位以排除状态污染；两轮全部点位对齐 ≤1.8%，证明回退后遗污染假设不成立、结果为 MRR64 的可复现行为。另以全新实例跑原版 MRR48 的 4.1 C=32 单点做归因对照（276 初测 → 266 对照 → 228 复测：−3% 实例新鲜度 + −14% MRR64 配置代价）。正文一律采用 v2 复测值；C≤16 各点与 5.x 长上下文行沿用初测值（MRR 在这些档位不构成约束）。初测原始数据留档于服务器 `b300eq_tp2pp4_20260910_1138/` 与库内 `results/tp2pp4/`，复测两轮与对照留档 `b300eq_pp4mrr64_20260910_1927/`、`b300eq_pp4mrr64v2_20260910_2021/`、`b300eq_mrr48ctl_20260910_2112/`。
-
-因此，下文比较回答的是"两种部署形态谁更适合该负载"，不能把差异单独归因于 EAGLE、PP 流水、chunk、radix 或图覆盖中的某一项（与 B300 报告同款声明）。
-
-**测量协议**（对齐 B300 口径）：
-
-- 冷缓存：`--shared-frac 0`，每点前 `POST /flush_cache`，服务端 Prefill 日志核算命中率，>0.01 重测一次，仍超停点排查；全矩阵命中核验最终全部达标。
-- 指标：Input TPS / Output TPS / TTFT P95 / TPOT P95（P95 为 nearest-rank；Input TPS = 输入 token / 全程墙钟，与 B300 口径一致）。
-- 负载：PG19 真实语料 token 切片（`corpus_ids.json`），input_ids 直打 `/generate`，temperature=0、ignore_eos、流式；nreq = max(8, 2×并发)，边界行 nreq=并发。
-- 并发档位按决策收敛：16K/1K 类封顶 64；64K 封 8、128K 封 4（128K C=2 补一档）；超出活跃上限的档位是**排队观察点**（与 B300 C=256 同性质，保留为有效观察）。
-- 语料已耗尽（21.23M/21.30M），冷缓存口径下用**回收窗口**复用（窗口基址见附录 A，逐记录 `corpus_window` 字段留档）。
-
-**两臂活跃上限**（MRR 与 KV 池决定，解释各行哪些并发是排队观察点）：
-
-| 场景 | TP2PP4 活跃上限 | E7b 活跃上限 |
-|-|-|-|
-| 16K | ~59（池 1,040,384 ÷ 17.4K；MRR64 超配，C=64 有 3 次回退） | 16（池 276,480 ÷ 约17.4K；MRR64 不再是约束） |
-| 1K→128 | 64（MRR） | 64（MRR） |
-| 1K→4K | 64（MRR） | ~52（池；C=64 有 12 条排队） |
-| 64K | 15（池） | 4（池） |
-| 128K | 7（池） | 2（池） |
-| 256K | 3（池） | 1（池 262K KV / 276K 贴边） |
-| 512K / 896K | 1（池） | 结构性不可（ctx 270,336） |
-
-## 3. 主场景：16K 输入、512 输出
-
-B300 跑了 C=1/8/32/64/128/256；本机按 MRR 上限收敛为 C=1/8/16/32/64（C=16 为本机甜点档，B300 无此档；C=128/256 超出两臂 MRR）。
-
-| 并发 | 方案 | Input TPS | Output TPS | TTFT P95 | TPOT P95 |
-|-|-|-|-|-|-|
-| 1 | TP2PP4 | 534 | 16.7 | 5.36 s | 50.3 ms |
-| 1 | TP8+EAGLE3+AR | 1,587 | 49.6 | **3.98 s** | **13.7 ms** |
-| 8 | TP2PP4 | 3,236 | **101** | **14.79 s** | **70.9 ms** |
-| 8 | TP8+EAGLE3+AR | 2,684 | 83.9 | 32.47 s | 136.1 ms |
-| 16 | TP2PP4 | 4,674 | **146** | **25.78 s** | **98.5 ms** |
-| 16 | TP8+EAGLE3+AR | 2,947 | 92.1 | 59.91 s | 228.8 ms |
-| 32 | TP2PP4 | **6,156** | **192** | **46.31 s** | **153.2 ms** |
-| 32 | TP8+EAGLE3+AR | 3,123 | 97.6 | 140.33 s | 213.6 ms |
-| 64 | TP2PP4 | **6,419** | **201** | **89.22 s** | 261.5 ms |
-| 64 | TP8+EAGLE3+AR | 3,188 | 99.6 | 293.21 s | **214.6 ms** |
-
-趋势：
-
-- **分界仍在 C=8，但差距显著收窄**：C=1 E7b 全指标占优；C=8 起 TP2PP4 输出/输入/TTFT 反超。输出吞吐差距 2.0×（201 vs 99.6）；TPOT P95 在 C=64 反被 E7b 反超（261.5 vs 214.6 ms——TP2PP4 该点活跃 ~59 条贴池顶 + 3 次回退，E7b 池限 16 活跃反而批更轻）。
-- **调参消除掉图断崖**：E7b 输出从 C=8 的 83.9 单调爬到 C=64 的 99.6 tok/s（+19%），C=16 TPOT P95 从初测 298 ms 降到 228.8 ms，C=32/64 稳定在 ~214 ms——decode 图全程覆盖运行批。EAGLE accept 随并发从 2.56 爬到 2.95。
-- **prefill 墙成为 E7b 的输出上限**：其 input TPS 从 C=8 的 2,684 到 C=64 仅 +19%（3,188），TP2PP4 同区间 +98%（3,236→6,419）——chunk 8192 + TP8 无 PP 流水 vs chunk 16384 + PP4 摊满。16K 场景 E7b 输出被 prefill 封死在 ~100 tok/s 平台，并发再高也不突破。
-- **两臂的 C=64 都撞各自的容量墙**：E7b 活跃上限 = KV 池（~16 条，C=32/64 为排队观察点，TTFT P95 140/293 s）；TP2PP4 MRR64 后活跃 ~59 条 = 16K 池顶（C=32→64 输出仅 +4%，3 次回退），但 TTFT P95 从 MRR48 排队态的 134.8 s 压到 89.2 s（−34%）——同样需要准入控制。
-
-## 4. 短输入与长输出
-
-### 4.1 `1K -> 128`
-
-| 并发 | 方案 | Input TPS | Output TPS | TTFT P95 | TPOT P95 |
-|-|-|-|-|-|-|
-| 1 | TP2PP4 | 155 | 19.3 | 386 ms | 49.3 ms |
-| 1 | TP8+EAGLE3+AR | **518** | **64.7** | **326 ms** | **14.3 ms** |
-| 8 | TP2PP4 | 815 | 102 | **1.69 s** | 72.9 ms |
-| 8 | TP8+EAGLE3+AR | **1,211** | **151** | 2.17 s | **53.7 ms** |
-| 32 | TP2PP4 | 1,825 | 228 | **5.49 s** | **120.7 ms** |
-| 32 | TP8+EAGLE3+AR | **1,833** | **229** | 7.09 s | 150.6 ms |
-| 64 | TP2PP4 | **3,466** | **433** | **7.16 s** | **116.7 ms** |
-| 64 | TP8+EAGLE3+AR | 2,178 | 272 | 12.15 s | 292.7 ms |
-
-短输入下 E7b 在 C=8 显著占优（151 vs 102，1.5×）；C=32 两方案打平（229 vs 228——TP2PP4 的 MRR64 配置代价吃掉了初测 MRR48 276 的领先，见 2 节演进注记）；C=64 TP2PP4 重新拉开（433 vs 272，**1.6×**）且 TTFT/TPOT 双优（7.16 s/116.7 ms vs 12.15 s/292.7 ms）——MRR64 后 64 条满飞无排队，而 E7b 池上限 ~52 活跃、TPOT 被大稳态批拖高。B300 同场景 Low-Latency 到 C=128 才被反超，本机在 C=8~64 之间，主因是容量/配置而非算力。
-
-### 4.2 `1K -> 4K`
-
-| 并发 | 方案 | Output TPS | TTFT P95 | TPOT P95 |
-|-|-|-|-|-|
-| 1 | TP2PP4 | 20.3 | 395 ms | 49.8 ms |
-| 1 | TP8+EAGLE3+AR | **135** | **320 ms** | **8.7 ms** |
-| 8 | TP2PP4 | 101 | 1.87 s | 79.4 ms |
-| 8 | TP8+EAGLE3+AR | **402** | **1.65 s** | **24.3 ms** |
-| 32 | TP2PP4 | 347 | **4.95 s** | 101.2 ms |
-| 32 | TP8+EAGLE3+AR | **676** | 6.19 s | **54.6 ms** |
-| 64 | TP2PP4 | 572 | **7.30 s** | 112.5 ms |
-| 64 | TP8+EAGLE3+AR | **826.5** | 12.18 s | **85.1 ms** |
-
-长输出放大了两形态的差异——调参后 E7b 在本场景全并发档反超：
-
-- **E7b 全档最优，C=64 = 826.5 tok/s 为全场最高输出吞吐**：C=1/8/32/64 输出 135/402/676/826.5，对 TP2PP4（20.3/101/347/572）为 6.7×/4.0×/1.9×/1.4×；C=64 TPOT 仍优（85.1 vs 112.5 ms），TTFT 被 TP2PP4 的 MRR64 反超（7.30 vs 12.18 s）。EAGLE accept 3.7~4.0——长输出让草稿模型进入"顺笔"状态，显著高于 4.1 短输出行（2.0~2.1）。
-- **初测的排队断崖已消除**：C=32 TTFT P95 从初测 304.75 s（nreq=64 / MRR=16 四波串行）降到 6.19 s；C=64 从未完成变为 12.18 s。MRR64 下 1K→4K 的池上限约 52 条活跃，C=64 仅 12 条排队，请求几乎全程满飞。
-- **TP2PP4 本场景被压，但 MRR64 修复了它的 C=64**：1K 短进长出下既无 prefill 墙可摊、也无投机解码加成；C=64 输出 482→572 tok/s（+19%）、TTFT P95 从 337.92 s 塌缩到 7.30 s（48 活跃+16 排队 → 64 满飞），不再只限离线批处理，但输出仍被 E7b 反超 45%。
-
-## 5. 长上下文观察
-
-### 5.1 `64K/128K -> 512`
-
-（并发档位按用户指示收敛：64K 封 8、128K 封 4，另补 128K C=2；B300 同场景为 64K C=8/32/64、128K C=8/32。）
-
-| 场景 | 并发 | 方案 | Input TPS | Output TPS | TTFT P95 | TPOT P95 |
-|-|-|-|-|-|-|-|
-| 64K→512 | 1 | TP2PP4 | 1,834 | 14.3 | **10.32 s** | 50.0 ms |
-| 64K→512 | 1 | TP8+EAGLE3+AR | **2,877** | **22.5** | 17.17 s | **13.5 ms** |
-| 64K→512 | 4 | TP2PP4 | **4,287** | **33.5** | **28.83 s** | **99.5 ms** |
-| 64K→512 | 4 | TP8+EAGLE3+AR | 3,292 | 25.7 | 68.87 s | 154.6 ms |
-| 64K→512 | 8 | TP2PP4 | **5,650** | **44.1** | **53.38 s** | **161.8 ms** |
-| 64K→512 | 8 | TP8+EAGLE3+AR | 3,280 | 25.6 | 149.45 s | 157.5 ms |
-| 128K→512 | 1 | TP2PP4 | 3,017 | 11.8 | **18.23 s** | 49.7 ms |
-| 128K→512 | 1 | TP8+EAGLE3+AR | **3,054** | **11.9** | 37.60 s | **12.3 ms** |
-| 128K→512 | 2 | TP2PP4 | **4,310** | **16.8** | **32.42 s** | **83.7 ms** |
-| 128K→512 | 2 | TP8+EAGLE3+AR | 3,145 | 12.3 | 75.81 s | 162.2 ms |
-| 128K→512 | 4 | TP2PP4 | **5,626** | **22.0** | **60.35 s** | **146.8 ms** |
-| 128K→512 | 4 | TP8+EAGLE3+AR | 3,133 | 12.2 | 159.50 s | 163.4 ms |
-
-- 64K C=1 E7b 仍占优（22.5 vs 14.3 tok/s），但 128K C=1 两方案输出打平（11.8 vs 11.9）——prefill 逐渐成为长上下文的主导成本，E7b 的 decode 优势被稀释；其 TTFT 反而慢 2×（37.6 vs 18.2 s）。
-- C≥2 起 TP2PP4 全指标占优；E7b 的 output TPS 在 64K/128K 行几乎不随并发变化（22.5→25.6、11.9→12.2）——KV 池把活跃钉在 ~4/~2 条，并发收益被容量封死（与 MRR/图调参无关，故沿用初测值）。
-- DSA 的 TPOT 上下文不变性（TP2PP4 C=1：50.0 ms @64K ≈ 49.7 ms @128K ≈ 50.3 ms @16K）与并发驱动性（C=8：70.9 ms @16K → 161.8 ms @64K）在本组完整呈现。
-
-### 5.2 上下文边界
-
-（OSL=1，只验证容量与 prefill，不比较 Output TPS/TPOT——与 B300 同声明。B300 完成到约 1M；本机以 896K=917,504 tokens 对应 B300"约 1M"档。）
-
-| 输入长度 | 方案 | 已完成并发 | C=1 Input TPS | 最高并发 TTFT P95 |
-|-|-|-|-|-|
-| 256K | TP2PP4 | 1/2/3 | **7,050** | 102.99 s（C=3） |
-| 256K | TP8+EAGLE3+AR | 1 | 2,867 | 91.45 s（C=1） |
-| 512K | TP2PP4 | 1 | **5,662** | 92.60 s（C=1） |
-| 512K | TP8+EAGLE3+AR | 结构性不可（ctx 270,336） | — | — |
-| 896K | TP2PP4 | 1 | **4,153** | 220.91 s（C=1） |
-| 896K | TP8+EAGLE3+AR | 结构性不可（ctx 270,336） | — | — |
-
-- 256K C=1 两臂相差 2.5×（7,050 vs 2,867 tok/s）：TP2PP4 的 chunk 16384 + PP4 流水对超长 prefill 的摊满优势，在边界长度上比 128K 行（几乎打平）进一步放大；E7b 的 chunk 8192 代价随长度累积。
-- TP2PP4 边界 input TPS 随长度衰减平缓（7,050 → 5,662 → 4,153），896K 单条 220.9 s 完成、池 1,040,384 tokens 单条可容（KV 917,504 + 余量）。
-- **E7b 256K C=1 命中核验注记（hicache 宿主层陷阱）**：首测命中率 0.9998、重试仍超——根因是该点 nreq=1 测量文本与预热完全相同，256K 预热 KV（262,160 tokens）占池 94.8% 触发分层缓存宿主层下放，`flush_cache` 只清 GPU radix 树、清不掉宿主层。改用该服务实例从未发过的文本（窗口基址 9,900,000）无预热重测，命中 0.0，数据干净。此为分层缓存运维要点：**宿主层缓存不受 flush_cache 影响，冷测必须换文本**。
-
-## 6. 显存状态
-
-- **TP2PP4**：服务加载后空载 64.6 GiB/卡，矩阵峰值 **84.8 GiB/卡**（MRR64 复测矩阵，主场景 C=64 活跃 ~59 条贴池顶时；初测 MRR48 峰值 85.0 GiB、最紧张卡余量约 0.6 GiB）。mem 0.85 下 KV 池按卡容量贴满分配，属预期；MRR64 已把 16K 场景推到池顶，继续上调并发或上下文没有余量，扩容前必须先降 mem-fraction。
-- **TP8+EAGLE3+AR**：空载 77.5 GiB/卡（EAGLE 草稿权重 + mem 0.90 大池 + 13 档 decode 图，图捕获完成后余 6.11 GB/卡），复测矩阵峰值 **81.7 GiB/卡**（余量约 3.9 GiB；初测臂含 64K/128K/256K 长上下文的 hicache 传输，峰值 83.6 GiB/卡）。
-- 全矩阵 **0 OOM**；唯一出现回退（retraction）的点是 TP2PP4 主场景 C=64（MRR64 对 16K 池超配，两轮复测各 3 次；MRR48 初测同点 0 回退）——B300 未披露该指标，除该点外全部点 retractions_total=0。
-- 显存时间线逐 30 s 采样留档（vram_timeline.csv），可复核任一时刻的卡间分布。
-
-## 7. 建议
-
-1. **低并发交互/agent 长思考（C≤8）用 E7b**：主场景 C=1 TPOT 13.7 ms、长输出 C=8 输出 402 tok/s；该区间对 TP2PP4 的优势最大（3.0~6.7×）。
-2. **prefill 密集/长上下文（16K 主场景、输入 ≥64K）用 TP2PP4**：主场景 C=64 输出 201 tok/s（E7b 99.6）、128K C=1 TTFT 18.2 s、896K 可达；MRR64 后 16K 场景活跃 ~59 条为池封顶，1K 场景吞吐上限即 MRR=64。
-3. **MRR 按负载形态选（本轮单变量结论）**：需要 C=64 档或低 TTFT 时 MRR64 收益确定——4.1 C=64 +27%、4.2 C=64 +19%、三场景 C=64 TTFT P95 全部塌缩（20.0/337.9/134.8 s → 7.2/7.3/89.2 s）；但 1K 短输入 C≤32 档付出 ~14% 配置代价（4.1 C=32 276→228，全新实例 MRR48 对照定案），16K 场景无代价。以 1K 短输入为主且并发 ≤48 的生产口可保持 MRR48；两档并存可按场景路由。
-4. **负载形态分界线（调参后）**：decode 密集（短进长出，1K→4K）任何并发档 E7b 全优（C=64 输出 826 vs 572 tok/s）；prefill 密集（16K 主场景）C≥8 TP2PP4 全优（201 vs 100）；短输入短输出（1K→128）C=8 E7b、C=32 打平、C=64 TP2PP4（1.6×）。选型看输入/输出长度分布，不能只看并发。
-5. **E7b 剩余瓶颈在 prefill 墙与 KV 池，不再在图**：MRR64 + 图桶 1–64 已消掉掉图断崖（本报告即复测值）；16K 主场景输出封顶 ~100 tok/s 是 chunk 8192 + TP8 无 PP 流水所致。扩并发容量的唯一杠杆是 KV 池（hicache 宿主层只救命中场景，不增并发容量）。
-6. **边界与超长上下文只有 TP2PP4 口径可服务**：E7b 若要对标 B300 512K/约1M 行，需要 ctx ≥524,288 与池 ≥52 万 tokens 的部署形态，本版（ctx 270,336 / 池 276,480）结构性不可达。
-7. **不要把两臂差异单归因 EAGLE**：两臂同时差在并行拓扑、chunk、radix、MRR 与图覆盖；单变量消融未做（与 B300 报告建议 3 同款）。MRR64 在 1K 短输入的 −14% 代价同样只有三实例 A/B 事实、机制未定位。
-8. **生产容量同时设吞吐和延迟 SLO**：TP2PP4 主场景 C=64 输出最高但 TTFT P95 仍达 89 s；E7b 主场景 C=64 TTFT P95 293 s（池限 16 活跃的排队）。只看峰值 TPS 会掩盖排队长尾。
-9. **分层缓存运维**：宿主层缓存不受 `flush_cache` 影响，任何冷缓存测量/复测必须更换输入文本（见 5.2 注记）。
-
-## 8. 原始结果与复现
-
-- 服务器原始结果（60.8）：
-  - TP2PP4 臂：`/root/bench_logs/b300eq_tp2pp4_20260910_1138/`（all_results.jsonl 24 点、status.txt、server_facts.txt、gpu_inventory、vram_timeline.csv）
-  - E7b 臂初测（MRR16/图 1–8，留档基线）：`/root/bench_logs/b300eq_e7b_20260910_1428/`（all_results.jsonl 20 条：16 点 OK + 4.2 C=64 用户中止 + 256K C=1 干净重测；status.txt 含 4 个结构性跳过与 HIT_FAIL_FINAL 首测记录）
-  - E7b 臂复测（MRR64/图 1–64，**正文采用值**）：`/root/bench_logs/b300eq_e7b64_20260910_1732/`（all_results.jsonl 10 点全 OK：16K/4.1/4.2 三场景 C=8/16/32/64，命中核验全 0.0、0 retraction）；前后对比表 `/root/bench_logs/retest_compare.md`
-  - TP2PP4 MRR64 复测 pass-1：`/root/bench_logs/b300eq_pp4mrr64_20260910_1927/`（6 点全 OK，与 v2 对齐 ≤1.8%）
-  - TP2PP4 MRR64 复测 v2（**正文采用值**，全新实例 + 回退点排末位）：`/root/bench_logs/b300eq_pp4mrr64v2_20260910_2021/`（6 点全 OK；16K C=64 3 次回退为 MRR64 池超配的确定性行为）；前后对比表 `/root/bench_logs/retest_compare_pp4_v2.md`
-  - TP2PP4 MRR48 对照（归因实验，全新实例单点）：`/root/bench_logs/b300eq_mrr48ctl_20260910_2112/`（4.1 C=32 = 266.42 tok/s）
-  - 资产 md5 台账：`/root/bench_logs/b300eq_md5_ledger.txt`
-- 本地镜像：`D:\sskj\b300eq\{tp2pp4,e7b,e7b64}\`（上述全部文件）、`D:\sskj\b300eq\pp4mrr64_mirror\`（TP2PP4 复测两轮 + 对照全量）、`D:\sskj\b300eq\report_tables.md`（表格生成器输出）、`D:\sskj\b300eq\retest_compare.md`（E7b64 前后对比）、`D:\sskj\b300eq\pp4mrr64_mirror\retest_compare_pp4_v2.md`（TP2PP4 前后对比）
-- 部署脚本：`/root/deploy_glm53_pp4.sh`（md5 def3c64c…，与库内 sskj main 副本一致）、`/root/deploy_glm53_pp4_mrr64.sh`（533440ef…，TP2PP4 高并发版：仅 MRR 48→64 一处差异）、`/root/deploy_glm53_607_exp.sh`（E7b 在役配方）、`/root/deploy_glm53_e7b_hicc.sh`（165db732…，E7b 高并发版：MRR64 + 图桶 1–64，其余与前者逐字一致）；复测驱动 `/root/run_retest_e7b64.sh`（fe46da2e…）、`/root/run_retest_pp4mrr64.sh`（c9b6361d…，pass-1）、`/root/run_retest_pp4mrr64_v2.sh`（7d857816…，v2 有序版）、`/root/launch_v2_after_pass1.sh`（889c2428…，衔接 wrapper）、`/root/control_mrr48_41c32.sh`（1ff8a21f…，对照 + 自动恢复链）；对比生成器 `/root/gen_retest_compare.py`、`/root/gen_retest_compare_pp4.py`；CAR 补丁：`/root/patches/custom_all_reduce.py`（a8fc9a50…）+ `custom_all_reduce_utils.py`（65a4d22b…），三处（60.7 原件/本地/库内）md5 一致
-- 测量工具：`/root/bench_corpus_v2.py`（md5 1e34dd8d…，p95 nearest-rank + 逐请求 dump）、`/root/extract_summary.py`（4c126d06…）、`/root/run_b300_matrix.sh`（5892b446…，矩阵驱动：alive/idle_wait/prewarm/flush/命中核验/重试/VRAM 采样）
-- 复现命令（单点示例）：
-
-```bash
-# 冷缓存压测（E7b 256K C=1 干净版）
-python3 /root/bench_corpus_v2.py --input-len 262144 --output-len 1 --shared-frac 0 \
-  --concurrency 1 --num-requests 1 --run-id 9551 --pool-override 9900000 \
-  --dump-records $L/b52_256k_c1_v2_records.jsonl
-# 全矩阵：nohup bash /root/run_b300_matrix.sh <arm> > <progress.log> 2>&1 &
-# E7b 高并发复测（MRR64/图≤64，10 点）：nohup bash /root/run_retest_e7b64.sh > <progress.log> 2>&1 &
-# TP2PP4 MRR64 复测（v2 有序，回退点排末位）：nohup bash /root/run_retest_pp4mrr64_v2.sh > <progress.log> 2>&1 &
-```
-
-## 9. 与 B300 对比观察
-
-> **口径声明**：B300 报告未写明模型量化方式（若为原始 BF16 权重，则与本机 NVFP4 非同模型形态）；硬件为 8×B300（288 GB HBM3e）vs 本机 8×RTX 6000D（96 GB GDDR7）；镜像 v0.5.18-cu130-dev4 vs nightly-20260828-daf63171。**绝对值仅量级可比，本对比只对结构性结论负责**。
-
-- **定性结构完全复现**：低延迟配方（B300 Low-Latency = TP8+EAGLE vs 本机 E7b = TP8+EAGLE3+AR）在 C=1 占优、吞吐配方（B300 High-Throughput = DP8+DeepEP vs 本机 TP2PP4 = D 生产口径）在高并发占优——两套硬件上"低延迟 vs 高吞吐"的分野方向一致。
-- **分界点本机更靠前，且调参后由负载形态决定**：B300 的交叉点在 C=64~128（主场景 HT C=128 反超 33%）；本机主场景交叉仍在 C=8 附近（容量上限所致：本臂 MRR64/池贴边 16 活跃 vs B300 配方 256/默认，先于算力撞墙），但 decode 密集场景（1K→4K）E7b 调参后全档占优、无交叉——这一形态差异 B300 报告未呈现。
-- **绝对差距 4~5×（主场景）**：C=1 输出 246 vs 49.6 tok/s（5.0×）、input 7,882 vs 1,587（5.0×）；吞吐侧峰值 997 vs 201（5.0×）、31,889 vs 6,419（5.0×）。与显存带宽硬件代差量级一致。
-- **边界 prefill 差距收窄到 ~2×**：256K C=1 input 7,050 vs 17,457（2.5×）→ 512K 5,662 vs 12,421（2.2×）→ 896K/约1M 4,153 vs 7,897（1.9×）。计算密集的超长 prefill 是 6000D 相对最能打的位置（PP 流水摊满 + 带宽占比下降）。
-- **TPOT 差距小于吞吐差距**：B300 LL C=1 4.36 ms vs E7b 13.7 ms（3.1×）；高并发侧 B300 HT C=128 165 ms vs TP2PP4 C=64 261.5 ms（1.6×，本机该点为池顶运行、含 3 次回退的批扰动；MRR48 排队态为 203.7 ms）——NVFP4 + DSA 把 decode 单步成本压得相对不差，差距主要在吞吐面。
-- **饱和形态不同**：B300 LL 在 C=64 后进入 24K input tok/s 平台、HT 在 C=128 达峰后 C=256 回退 19%；本机 TP2PP4 到 C=64 到达 16K 池顶（活跃 ~59 条封顶，C=32→64 输出仅 +4%），E7b 调参后在 16K 场景呈 ~100 tok/s 输出平台（池限 16 活跃 + prefill 墙）、在 1K→4K 场景爬到 826 tok/s 无回退。本机没有一档出现吞吐回退——"甜点=并发上限"由 MRR/池决定而非算力。
-- **EAGLE 配方差异**：B300 LL 为 5 steps/6 draft tokens，本机 E7b 为 4 steps/topk1/5 draft tokens；本机实测 accept 2.0~4.0（短输出 2.0、主场景 2.1~3.0、长输出 3.7~4.0，随 decode 深入上升）。B300 未披露 accept，无法直接对比投机效率。
-- **容量边界差距最大**：B300 两模式都完成约 1M 输入 C=1/2/4；本机仅 TP2PP4 可达 896K 且 C=1 单条（池 1,040,384 刚容一条），E7b 连 512K 都结构性不可测（ctx 270,336）。96 GB 卡上"上下文边界=显存边界"比 B300 严酷得多。
-
-## 附录 A：语料窗口映射（回收窗口）
-
-语料总量 21,296,780 tokens，此前场景一/二战役已消费至 21,235,008。冷缓存协议下回收复用：窗口基址 `--pool-override` 显式指定，每点窗口在基址上顺序推进（逐记录 `corpus_window.start/end` 留档），每点 flush + 命中核验 ≤0.01 保证冷。E7b 复测臂沿用与初测相同的窗口基址（2,300,000 / 4,500,000 / 4,700,000）——复测为全新容器实例，这些文本对 hicache 宿主层重新成为"处女文本"，冷缓存协议成立（复测 10 点命中核验全部 0.0）。TP2PP4 复测两轮（pass-1/v2）与 MRR48 对照同样沿用该三处基址（radix 关 + flush 即冷，13 次测量命中核验全部 0.0）。
-
-| 场景 | 窗口基址 | 备注 |
-|-|-|-|
-| 主场景 16K→512 | 2,300,000 | |
-| 4.1 短输入 1K→128 | 4,500,000 | |
-| 4.2 长输出 1K→4K | 4,700,000 | |
-| 5.1 64K→512 | 5,000,000 | |
-| 5.1 128K→512 | 6,200,000 | |
-| 5.2 256K→1 | 8,400,000 | E7b 干净重测改用 9,900,000（实例首用文本，避 hicache 宿主层残留） |
-| 5.2 512K→1 | 9,300,000 | 仅 TP2PP4 |
-| 5.2 896K→1 | 9,900,000 | 仅 TP2PP4 |
-
-## 附录 B：全量指标（含 mean/p95/max、回退、投机接受长度）
-
-| 场景点 | 方案 | ok/nreq | wall(s) | out tok/s | in tok/s | TTFT mean/p95/max(s) | TPOT mean/p95/max(ms) | retractions | accept |
-|---|---|---|---|---|---|---|---|---|---|
-| b3_16k_c1 | TP2PP4 | 8/8 | 245.55 | 16.68 | 533.8 | 5.35/5.36/5.36 | 49.6/50.3/50.3 | 0 | None |
-| b3_16k_c1 | TP8+EAGLE3+AR | 8/8 | 82.58 | 49.6 | 1587.15 | 3.93/3.98/3.98 | 12.5/13.7/13.7 | 0 | 2.138 |
-| b3_16k_c8 | TP2PP4 | 16/16 | 81.0 | 101.13 | 3236.22 | 10.05/14.79/14.79 | 59.5/70.9/70.9 | 0 | None |
-| b3_16k_c8 | TP8+EAGLE3+AR | 16/16 | 97.66 | 83.88 | 2684.26 | 12.12/32.47/32.47 | 69.2/136.1/136.1 | 0 | 2.556 |
-| b3_16k_c16 | TP2PP4 | 32/32 | 112.16 | 146.08 | 4674.5 | 15.53/25.78/25.84 | 79.2/98.5/101.2 | 0 | None |
-| b3_16k_c16 | TP8+EAGLE3+AR | 32/32 | 177.89 | 92.1 | 2947.27 | 19.76/59.91/63.69 | 129.7/228.8/275.9 | 0 | 2.485 |
-| b3_16k_c32 | TP2PP4 | 64/64 | 170.33 | 192.38 | 6156.1 | 26.48/46.31/47.97 | 114.5/153.2/159.2 | 0 | None |
-| b3_16k_c32 | TP8+EAGLE3+AR | 64/64 | 335.79 | 97.58 | 3122.67 | 82.65/140.33/154.66 | 138.1/213.6/277.5 | 0 | 2.777 |
-| b3_16k_c64 | TP2PP4 | 128/128 | 326.73 | 200.58 | 6418.52 | 50.76/89.22/147.41 | 185.0/261.5/313.1 | 3 | None |
-| b3_16k_c64 | TP8+EAGLE3+AR | 128/128 | 657.79 | 99.63 | 3188.2 | 204.27/293.21/322.84 | 142.5/214.6/276.6 | 0 | 2.954 |
-| b41_1k_c1 | TP2PP4 | 8/8 | 52.95 | 19.34 | 154.7 | 0.38/0.39/0.39 | 49.1/49.3/49.3 | 0 | None |
-| b41_1k_c1 | TP8+EAGLE3+AR | 8/8 | 15.82 | 64.74 | 517.93 | 0.30/0.33/0.33 | 13.2/14.3/14.3 | 0 | 2.028 |
-| b41_1k_c8 | TP2PP4 | 16/16 | 20.1 | 101.91 | 815.3 | 1.31/1.69/1.69 | 68.6/72.9/72.9 | 0 | None |
-| b41_1k_c8 | TP8+EAGLE3+AR | 16/16 | 13.53 | 151.33 | 1210.61 | 1.25/2.17/2.17 | 42.1/53.7/53.7 | 0 | 1.992 |
-| b41_1k_c32 | TP2PP4 | 64/64 | 35.92 | 228.08 | 1824.68 | 4.60/5.49/5.52 | 104.1/120.7/137.5 | 0 | None |
-| b41_1k_c32（MRR48 对照·全新实例） | TP2PP4 | 64/64 | 30.75 | 266.42 | 2131.36 | 4.04/4.92/4.93 | 88.4/103.0/110.0 | 0 | None |
-| b41_1k_c32 | TP8+EAGLE3+AR | 64/64 | 35.75 | 229.18 | 1833.42 | 2.99/7.09/7.10 | 111.9/150.6/179.8 | 0 | 2.035 |
-| b41_1k_c64 | TP2PP4 | 128/128 | 37.82 | 433.2 | 3465.61 | 5.46/7.16/7.24 | 104.3/116.7/147.3 | 0 | None |
-| b41_1k_c64 | TP8+EAGLE3+AR | 128/128 | 60.18 | 272.24 | 2177.88 | 4.76/12.15/13.48 | 191.1/292.7/331.5 | 0 | 2.094 |
-| b42_1k4k_c1 | TP2PP4 | 8/8 | 1614.16 | 20.3 | 5.08 | 0.39/0.40/0.40 | 49.2/49.8/49.8 | 0 | None |
-| b42_1k4k_c1 | TP8+EAGLE3+AR | 8/8 | 242.18 | 135.31 | 33.83 | 0.31/0.32/0.32 | 7.3/8.7/8.7 | 0 | 3.701 |
-| b42_1k4k_c8 | TP2PP4 | 16/16 | 649.69 | 100.87 | 25.22 | 1.35/1.87/1.87 | 79.0/79.4/79.4 | 0 | None |
-| b42_1k4k_c8 | TP8+EAGLE3+AR | 16/16 | 162.84 | 402.47 | 100.62 | 0.95/1.65/1.65 | 18.5/24.3/24.3 | 0 | 3.951 |
-| b42_1k4k_c32 | TP2PP4 | 64/64 | 756.13 | 346.69 | 86.67 | 2.85/4.95/4.97 | 91.0/101.2/101.4 | 0 | None |
-| b42_1k4k_c32 | TP8+EAGLE3+AR | 64/64 | 387.9 | 675.81 | 168.95 | 2.58/6.19/6.20 | 43.0/54.6/60.4 | 0 | 3.82 |
-| b42_1k4k_c64 | TP2PP4 | 128/128 | 916.56 | 572.01 | 143.0 | 3.47/7.30/7.32 | 109.7/112.5/115.4 | 0 | None |
-| b42_1k4k_c64 | TP8+EAGLE3+AR | 128/128 | 634.35 | 826.5 | 206.62 | 4.36/12.18/12.50 | 69.9/85.1/101.5 | 0 | 3.878 |
-| b51_64k_c1 | TP2PP4 | 8/8 | 285.91 | 14.33 | 1833.72 | 10.28/10.32/10.32 | 49.8/50.0/50.0 | 0 | None |
-| b51_64k_c1 | TP8+EAGLE3+AR | 8/8 | 182.23 | 22.48 | 2877.06 | 17.04/17.17/17.17 | 11.2/13.5/13.5 | 0 | 2.468 |
-| b51_64k_c4 | TP2PP4 | 8/8 | 122.3 | 33.49 | 4286.94 | 19.55/28.83/28.83 | 81.4/99.5/99.5 | 0 | None |
-| b51_64k_c4 | TP8+EAGLE3+AR | 8/8 | 159.24 | 25.72 | 3292.45 | 30.55/68.87/68.87 | 94.9/154.6/154.6 | 0 | 2.438 |
-| b51_64k_c8 | TP2PP4 | 16/16 | 185.59 | 44.14 | 5650.11 | 31.83/53.38/53.38 | 119.2/161.8/161.8 | 0 | None |
-| b51_64k_c8 | TP8+EAGLE3+AR | 16/16 | 319.66 | 25.63 | 3280.31 | 90.92/149.45/149.45 | 105.8/157.5/157.5 | 0 | 2.442 |
-| b51_128k_c1 | TP2PP4 | 8/8 | 347.5 | 11.79 | 3017.45 | 18.21/18.23/18.23 | 49.4/49.7/49.7 | 0 | None |
-| b51_128k_c1 | TP8+EAGLE3+AR | 8/8 | 343.36 | 11.93 | 3053.83 | 37.59/37.60/37.60 | 10.4/12.3/12.3 | 0 | 2.728 |
-| b51_128k_c2 | TP2PP4 | 8/8 | 243.3 | 16.84 | 4309.84 | 25.27/32.42/32.42 | 69.6/83.7/83.7 | 0 | None |
-| b51_128k_c2 | TP8+EAGLE3+AR | 8/8 | 333.36 | 12.29 | 3145.44 | 48.21/75.81/75.81 | 68.0/162.2/162.2 | 0 | 2.769 |
-| b51_128k_c4 | TP2PP4 | 8/8 | 186.37 | 21.98 | 5626.35 | 39.29/60.35/60.35 | 105.4/146.8/146.8 | 0 | None |
-| b51_128k_c4 | TP8+EAGLE3+AR | 8/8 | 334.71 | 12.24 | 3132.76 | 110.14/159.50/159.50 | 68.7/163.4/163.4 | 0 | 2.595 |
-| b52_256k_c1 | TP2PP4 | 1/1 | 37.18 | 0.03 | 7050.23 | 37.18/37.18/37.18 | — | 0 | None |
-| b52_256k_c1 | TP8+EAGLE3+AR | 1/1 | 91.45 | 0.01 | 2866.59 | 91.45/91.45/91.45 | — | 0 | None |
-| b52_256k_c2 | TP2PP4 | 2/2 | 70.19 | 0.03 | 7469.22 | 53.69/70.12/70.12 | — | 0 | None |
-| b52_256k_c3 | TP2PP4 | 3/3 | 103.12 | 0.03 | 7626.22 | 70.12/102.99/102.99 | — | 0 | None |
-| b52_512k_c1 | TP2PP4 | 1/1 | 92.6 | 0.01 | 5662.03 | 92.60/92.60/92.60 | — | 0 | None |
-| b52_896k_c1 | TP2PP4 | 1/1 | 220.91 | 0.0 | 4153.24 | 220.91/220.91/220.91 | — | 0 | None |
-
-注：TP2PP4 的 16K/4.1/4.2 场景 C=32/C=64 共 6 点为 MRR64 复测 v2 值（正文采用值；pass-1 两轮对齐 ≤1.8% 留档 `results/pp4mrr64/`），C=1/C=8/C=16 与 5.1/5.2 行沿用初测 MRR48 值（MRR 在这些档位不构成约束）；"MRR48 对照"行是 4.1 C=32 的全新实例归因实验（276 初测 → 266 对照 → 228 复测 = −3% 实例新鲜度 −14% MRR64 配置代价）。E7b 的 16K/4.1/4.2 场景 C=8~C=64 共 10 点为调参后（MRR64/decode 图 1–64）复测值，即正文采用值；C=1 各点与 5.1/5.2 行沿用初测值（活跃数受池上限约束，与调参无关）。E7b 256K/512K/896K C>1 为结构性跳过；256K C=1 为全新文本干净重测值（窗口 9,900,000）。初测全量原始数据留档于 `results/tp2pp4/`（MRR48）与 `results/e7b/`（MRR16/图 1–8）。

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/patches/custom_all_reduce.py

@@ -1,424 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-# Adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/distributed/device_communicators/custom_all_reduce.py
-
-import ctypes
-import logging
-import os
-from contextlib import contextmanager
-from functools import partial
-from typing import Any, List, Optional, Union
-
-import torch
-import torch.distributed as dist
-from torch.distributed import ProcessGroup
-
-import sglang.srt.distributed.device_communicators.custom_all_reduce_ops as ops
-from sglang.srt.distributed.device_communicators.cuda_wrapper import CudaRTLibrary
-from sglang.srt.distributed.device_communicators.custom_all_reduce_utils import (
-    can_use_custom_all_reduce_with_nvlink,
-    is_weak_contiguous,
-)
-from sglang.srt.environ import envs
-from sglang.srt.model_executor.runner_backend_utils.tc_piecewise_cuda_graph import (
-    is_in_tc_piecewise_cuda_graph,
-)
-from sglang.srt.utils import (
-    get_bool_env_var,
-    is_cuda,
-    is_hip,
-    is_musa,
-    log_info_on_rank0,
-)
-
-_is_cuda = is_cuda()
-_is_hip = is_hip()
-_is_musa = is_musa()
-
-logger = logging.getLogger(__name__)
-os.environ.setdefault("SGLANG_CUSTOM_ALLREDUCE_ALGO", "1stage")  # SSKJ-PATCH: C++ dispatch no-ops when full_nvlink=False; force kernel launch
-
-
-class CustomAllreduce:
-    _SUPPORTED_WORLD_SIZES = [2, 4, 6, 8]
-    _MAX_CAR_SIZE = 8192 * 1024
-    if _is_hip:
-        # crossover is at 16MB buffer size for ROCm
-        _MAX_CAR_SIZE = 2 * 8192 * 1024
-    if _is_musa:
-        # crossover is at 128MB buffer size for MUSA
-        _MAX_CAR_SIZE = 16 * 8196 * 1024
-
-    # max_size: max supported allreduce size
-    def __init__(
-        self,
-        group: ProcessGroup,
-        device: Union[int, str, torch.device],
-        max_size=_MAX_CAR_SIZE,
-    ) -> None:
-        """
-        Args:
-            group: the process group to work on. If None, it will use the
-                default process group.
-            device: the device to bind the CustomAllreduce to. If None,
-                it will be bind to f"cuda:{local_rank}".
-        It is the caller's responsibility to make sure each communicator
-        is bind to a unique device, and all communicators in this group
-        are in the same node.
-        """
-        self._IS_CAPTURING = False
-        self.disabled = True  # This can be modified in-place by context manager in piecewise cuda graph runner
-        self.original_disabled = True  # To store the original state
-        self.use_amd_deterministic_impl = _use_amd_deterministic_impl()
-
-        if not ops.IS_CUSTOM_AR_AVAILABLE:
-            # disable because of missing custom allreduce library
-            # e.g. in a non-cuda environment
-            return
-
-        rank = dist.get_rank(group=group)
-        world_size = dist.get_world_size(group=group)
-
-        if isinstance(device, int):
-            device = torch.device(f"cuda:{device}")
-        elif isinstance(device, str):
-            device = torch.device(device)
-        # now `device` is a `torch.device` object
-        assert isinstance(device, torch.device)
-        self.device = device
-        full_nvlink = can_use_custom_all_reduce_with_nvlink(
-            group=group,
-            device=device,
-            supported_world_size=self._SUPPORTED_WORLD_SIZES,
-            cls_name="CustomAllreduce",
-        )
-        if full_nvlink is None:
-            return  # fail to get nvlink status
-
-        self.group = group
-        self.max_size = max_size
-        self.rank = rank
-        self.world_size = world_size
-        self.full_nvlink = full_nvlink
-
-        if not _is_hip:
-            # Buffers memory are owned by this Python class and passed to C++.
-            # Meta data composes of two parts: meta data for synchronization and a
-            # temporary buffer for storing intermediate allreduce results.
-            self.meta_ptrs = self.create_shared_buffer(
-                ops.meta_size() + max_size, group=group
-            )
-            # This is a pre-registered IPC buffer. In eager mode, input tensors
-            # are first copied into this buffer before allreduce is performed
-            self.buffer_ptrs = self.create_shared_buffer(max_size, group=group)
-            # This is a buffer for storing the tuples of pointers pointing to
-            # IPC buffers from all ranks. Each registered tuple has size of
-            # 8*world_size bytes where world_size is at most 8. Allocating 8MB
-            # is enough for 131072 such tuples. The largest model I've seen only
-            # needs less than 10000 of registered tuples.
-            self.rank_data = torch.empty(
-                max_size, dtype=torch.uint8, device=self.device
-            )
-            self._ptr = ops.init_custom_ar(
-                self.meta_ptrs, self.rank_data, rank, self.full_nvlink
-            )
-            ops.register_buffer(self._ptr, self.buffer_ptrs)
-        else:
-            # meta data buffers need to be "uncached" for signal on MI200
-            self.meta = ops.allocate_meta_buffer(ops.meta_size() + max_size)
-            self.buffer = torch.empty(max_size, dtype=torch.uint8, device=self.device)
-            handle = ops.get_meta_buffer_ipc_handle(self.meta)
-            shard_data = (
-                bytes(handle),  # ipc handle to base ptr
-                0,  # offset of base ptr
-            )
-            handles, offsets = self._gather_ipc_meta(shard_data)
-            self.rank_data = torch.empty(
-                max_size, dtype=torch.uint8, device=self.device
-            )
-            self._ptr = ops.init_custom_ar(
-                self.meta, self.rank_data, handles, offsets, rank, self.full_nvlink
-            )
-            self.register_buffer(self.buffer)
-
-        self.disabled = False
-        self.original_disabled = False  # Ensure original_disabled == disabled
-        logger.warning(f"SSKJ_CAR_PATCH_ACTIVE ws={self.world_size} full_nvlink={self.full_nvlink}")
-        self.tms_cudagraph = envs.SGLANG_MEMORY_SAVER_CUDA_GRAPH.get()
-
-    @staticmethod
-    def create_shared_buffer(
-        size_in_bytes: int, group: Optional[ProcessGroup] = None
-    ) -> List[int]:
-        """
-        Creates a shared buffer and returns a list of pointers
-        representing the buffer on all processes in the group.
-        """
-        lib = CudaRTLibrary()
-        pointer = lib.cudaMalloc(size_in_bytes)
-        if _is_musa:
-            lib.cudaMemset(pointer, 0, size_in_bytes)
-        handle = lib.cudaIpcGetMemHandle(pointer)
-        world_size = dist.get_world_size(group=group)
-        rank = dist.get_rank(group=group)
-        handles = [None] * world_size
-        dist.all_gather_object(handles, handle, group=group)
-
-        pointers: List[int] = []
-        for i, h in enumerate(handles):
-            if i == rank:
-                pointers.append(pointer.value)  # type: ignore
-            else:
-                pointers.append(lib.cudaIpcOpenMemHandle(h).value)  # type: ignore
-
-        return pointers
-
-    @staticmethod
-    def free_shared_buffer(
-        pointers: List[int], group: Optional[ProcessGroup] = None
-    ) -> None:
-        rank = dist.get_rank(group=group)
-        lib = CudaRTLibrary()
-        lib.cudaFree(ctypes.c_void_p(pointers[rank]))
-
-    @contextmanager
-    def capture(self):
-        """
-        The main responsibility of this context manager is the
-        `register_graph_buffers` call at the end of the context.
-        It records all the buffer addresses used in the CUDA graph.
-        """
-        try:
-            self._IS_CAPTURING = True
-            yield
-        finally:
-            self._IS_CAPTURING = False
-            if not self.disabled:
-                self.register_graph_buffers()
-
-    def _get_ipc_meta(self, inp: torch.Tensor):
-        # _share_cuda_() doesn't accept meta buffer not allocated from
-        # PyTorch cache allocator, use direct HIP call to get IPC handle
-        handle = ops.get_meta_buffer_ipc_handle(inp)
-        shard_data = (
-            bytes(handle),  # ipc handle to base ptr
-            0,  # offset of base ptr
-        )
-        return self._gather_ipc_meta(shard_data)
-
-    def _gather_ipc_meta(self, shard_data):
-        # Note: don't use `[[None]] * self.world_size` here
-        # because it will create a list of the same reference
-        all_data: List[Optional[Any]] = [[None] for i in range(self.world_size)]
-        all_data[self.rank][0] = shard_data
-
-        ranks = dist.get_process_group_ranks(group=self.group)
-        ranks.sort()
-        for i, rank in enumerate(ranks):
-            dist.broadcast_object_list(
-                all_data[i], src=rank, group=self.group, device="cpu"
-            )
-
-        # we cannot directly use `dist.all_gather_object` here
-        # because it is incompatible with `gloo` backend under inference mode.
-        # see https://github.com/pytorch/pytorch/issues/126032 for details.
-
-        handles = []
-        offsets = []
-        for i in range(len(all_data)):
-            handles.append(all_data[i][0][0])  # type: ignore
-            offsets.append(all_data[i][0][1])  # type: ignore
-        return handles, offsets
-
-    def register_buffer(self, inp: torch.Tensor):
-        handles, offsets = self._get_ipc_meta(inp)
-        ops.register_buffer(self._ptr, inp, handles, offsets)
-
-    def register_graph_buffers(self):
-        if _is_hip:
-            handle, offset = ops.get_graph_buffer_ipc_meta(self._ptr)
-            handles, offsets = self._gather_ipc_meta((bytes(handle), offset))
-            log_info_on_rank0(logger, f"Registering {len(offset)} cuda graph addresses")
-            ops.register_graph_buffers(self._ptr, handles, offsets)
-        else:
-            handle, offset = ops.get_graph_buffer_ipc_meta(self._ptr)
-            log_info_on_rank0(logger, f"Registering {len(offset)} cuda graph addresses")
-            # We cannot directly use `dist.all_gather_object` here
-            # because it is incompatible with `gloo` backend under inference mode.
-            # see https://github.com/pytorch/pytorch/issues/126032 for details.
-            all_data = [
-                [None, None] for _ in range(dist.get_world_size(group=self.group))
-            ]
-            all_data[self.rank] = [handle, offset]
-            ranks = sorted(dist.get_process_group_ranks(group=self.group))
-            for i, rank in enumerate(ranks):
-                dist.broadcast_object_list(
-                    all_data[i], src=rank, group=self.group, device="cpu"
-                )
-            # Unpack list of tuples to tuple of lists.
-            handles = [d[0] for d in all_data]  # type: ignore
-            offsets = [d[1] for d in all_data]  # type: ignore
-            ops.register_graph_buffers(self._ptr, handles, offsets)
-
-    def should_custom_ar(self, inp: torch.Tensor):
-        if self.disabled:
-            return False
-        inp_size = inp.numel() * inp.element_size()
-        # custom allreduce requires input byte size to be multiples of 16
-        if inp_size % 16 != 0:
-            return False
-        if not is_weak_contiguous(inp):
-            return False
-        # for 4 or more non NVLink-capable GPUs, custom allreduce provides
-        # little performance improvement over NCCL.
-        if not _is_hip:
-            if True:
-                return inp_size <= self.max_size
-            return False
-
-        if _is_hip:
-            if self.use_amd_deterministic_impl:
-                return True
-            if self.full_nvlink:
-                return inp_size <= self.max_size
-            return False
-
-        return False
-
-    def _all_reduce_impl(self, inp: torch.Tensor, registered: bool):
-        out = torch.empty_like(inp)
-        if not _is_hip:  # CUDA-like
-            if registered:
-                ops.all_reduce(self._ptr, inp, out, 0, 0)
-            else:
-                ops.all_reduce(
-                    self._ptr, inp, out, self.buffer_ptrs[self.rank], self.max_size
-                )
-        elif self.use_amd_deterministic_impl:
-            inp_size = inp.numel() * inp.element_size()
-            if inp_size < self.max_size:
-                reg_buffer = self.buffer.view(inp.dtype)[: inp.numel()]
-                ops.deterministic_all_reduce_unreg(self._ptr, inp, reg_buffer, out)
-            else:
-                self.register_buffer(inp)
-                ops.deterministic_all_reduce_reg(self._ptr, inp, out)
-        else:  # normal AMD ROCm path
-            if registered:
-                ops.all_reduce_reg(self._ptr, inp, out)
-            else:
-                ops.all_reduce_unreg(self._ptr, inp, self.buffer, out)
-        return out
-
-    def custom_all_reduce(self, input: torch.Tensor) -> Optional[torch.Tensor]:
-        """The main allreduce API that provides support for cuda graph."""
-        # When custom allreduce is disabled, this will be None.
-        if self.disabled or not self.should_custom_ar(input):
-            return None
-        if self._IS_CAPTURING:
-            if torch.cuda.is_current_stream_capturing():
-                return self._all_reduce_impl(input, registered=not self.tms_cudagraph)
-            else:
-                # Could be warmup OR piecewise cuda graph split op execution.
-                # In piecewise cuda graph, split ops run eagerly outside the graph
-                # but _IS_CAPTURING is still True. We need to do real all-reduce.
-                if is_in_tc_piecewise_cuda_graph():
-                    # Split op execution - do real all-reduce
-                    return self._all_reduce_impl(input, registered=False)
-                else:
-                    # True warmup - mimic the allocation pattern since custom
-                    # allreduce is out-of-place.
-                    return torch.zeros_like(input)
-        else:
-            return self._all_reduce_impl(input, registered=False)
-
-    def close(self):
-        if not self.disabled and self._ptr:
-            if ops is not None:
-                ops.dispose(self._ptr)
-            if _is_cuda:
-                self.free_shared_buffer(self.meta_ptrs)
-                self.free_shared_buffer(self.buffer_ptrs)
-            self._ptr = 0
-
-    def __del__(self):
-        self.close()
-
-
-def dispatch_custom_allreduce(
-    group: ProcessGroup,
-    device: torch.device,
-):
-    """Return the CustomAllreduce class to use (aiter on ROCm if enabled).
-
-    On AMD with 1-stage AR enabled, use sglang's CustomAllreduce.
-    Otherwise use AiterCustomAllreduce if available.
-
-    On CUDA, the JIT-compiled v2 implementation is used by default.
-    Set SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2=0 to fall back to the legacy CustomAllreduce.
-    Multi-node v2 is admitted only for a single NVLink clique (see
-    can_use_custom_all_reduce_v2); other cross-node groups fall back to NCCL.
-    """
-    if _is_cuda and envs.SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2.get():
-        from .custom_all_reduce_v2 import (
-            CustomAllReduceV2,
-            can_use_custom_all_reduce_v2,
-        )
-
-        if can_use_custom_all_reduce_v2(group=group, device=device):
-            logger.debug("[AR] Using CustomAllReduceV2 (JIT-compiled)")
-            return CustomAllReduceV2
-
-    if _is_cuda or _is_musa:
-        return CustomAllreduce
-
-    assert _is_hip
-
-    if envs.SGLANG_USE_1STAGE_ALLREDUCE.is_set():
-        if envs.SGLANG_USE_1STAGE_ALLREDUCE.get():
-            logger.debug(
-                "[AR] All-reduce: 1-stage kernel (SGLANG_USE_1STAGE_ALLREDUCE=1)"
-            )
-        else:
-            logger.debug("[AR] All-reduce: default (SGLANG_USE_1STAGE_ALLREDUCE=0)")
-    elif envs.SGLANG_ENABLE_DETERMINISTIC_INFERENCE.get():
-        logger.debug(
-            "[AR] All-reduce: 1-stage kernel (deterministic inference enabled)"
-        )
-    else:
-        logger.debug("[AR] All-reduce: default")
-
-    # On AMD with 1-stage AR, use sglang's CustomAllreduce
-    # (AiterCustomAllreduce doesn't have deterministic_all_reduce method)
-    if _use_amd_deterministic_impl():
-        return CustomAllreduce
-
-    if get_bool_env_var("SGLANG_USE_AITER_AR", default="true"):
-        try:
-            from aiter.dist.device_communicators.custom_all_reduce import (
-                CustomAllreduce as AiterCustomAllreduce,
-            )
-
-            logger.info("[AR] Using AiterCustomAllreduce (AMD default)")
-            tms_cudagraph = envs.SGLANG_MEMORY_SAVER_CUDA_GRAPH.get()
-            return partial(
-                AiterCustomAllreduce,
-                enable_register_for_capturing=not tms_cudagraph,
-            )
-        except ImportError as e:
-            logger.warning(
-                "[AR] Aiter custom all-reduce not available; "
-                "falling back to sglang CustomAllreduce. Details: %s",
-                e,
-            )
-            return CustomAllreduce
-
-    return CustomAllreduce
-
-
-def _use_amd_deterministic_impl() -> bool:
-    if not _is_hip:  # CUDA is always deterministic
-        return False
-    if envs.SGLANG_USE_1STAGE_ALLREDUCE.is_set():
-        return envs.SGLANG_USE_1STAGE_ALLREDUCE.get()
-    else:
-        return envs.SGLANG_ENABLE_DETERMINISTIC_INFERENCE.get()

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/patches/custom_all_reduce_utils.py

@@ -1,519 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-# Adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/distributed/device_communicators/custom_all_reduce_utils.py
-
-import ctypes
-import json
-import logging
-import os
-import pickle
-import subprocess
-import sys
-import tempfile
-from functools import wraps
-from itertools import product
-from typing import Callable, Dict, List, Optional, Sequence, TypeVar
-
-import torch
-import torch.distributed as dist
-import torch.multiprocessing as mp
-from typing_extensions import ParamSpec
-
-from sglang.srt.distributed.device_communicators.cuda_wrapper import CudaRTLibrary
-from sglang.srt.distributed.parallel_state import in_the_same_node_as
-from sglang.srt.environ import envs as sglang_envs
-from sglang.srt.utils import is_cuda, is_hip, is_musa
-from sglang.srt.utils.cuda_vmm_utils import _gpu_fabric_clique
-
-logger = logging.getLogger(__name__)
-
-_is_cuda = is_cuda()
-_is_hip = is_hip()
-_is_musa = is_musa()
-
-if _is_cuda:
-    try:
-        import pynvml
-    except ImportError as e:
-        logger.warning("Failed to import pynvml with %r", e)
-
-if _is_musa:
-    try:
-        import pymtml as pynvml
-    except ImportError as e:
-        logger.warning("Failed to import pymtml with %r", e)
-
-if _is_hip:
-    try:
-        from amdsmi import (
-            AmdSmiException,
-            amdsmi_get_processor_handles,
-            amdsmi_init,
-            amdsmi_shut_down,
-            amdsmi_topo_get_link_type,
-        )
-    except ImportError as e:
-        logger.warning("Failed to import amdsmi with %r", e)
-
-_P = ParamSpec("_P")
-_R = TypeVar("_R")
-
-
-def update_environment_variables(envs: Dict[str, str]):
-    for k, v in envs.items():
-        if k in os.environ and os.environ[k] != v:
-            logger.warning(
-                "Overwriting environment variable %s " "from '%s' to '%s'",
-                k,
-                os.environ[k],
-                v,
-            )
-        os.environ[k] = v
-
-
-def producer(
-    batch_src: Sequence[int],
-    producer_queue,
-    consumer_queue,
-    result_queue,
-    cuda_visible_devices: Optional[str] = None,
-):
-    if cuda_visible_devices is not None:
-        update_environment_variables({"CUDA_VISIBLE_DEVICES": cuda_visible_devices})
-
-    lib = CudaRTLibrary()
-    for i in batch_src:
-        lib.cudaSetDevice(i)
-        pointer = lib.cudaMalloc(1024)
-        lib.cudaMemset(pointer, 1, 1024)
-        lib.cudaDeviceSynchronize()
-        handle = lib.cudaIpcGetMemHandle(pointer)
-        producer_queue.put(handle)
-        open_success = consumer_queue.get()
-        if open_success:
-            # use two queues to simulate barrier
-            producer_queue.put(0)
-            consumer_queue.get()
-            # check if the memory is modified
-            host_data = (ctypes.c_char * 1024)()
-            lib.cudaMemcpy(host_data, pointer, 1024)  # type: ignore
-            for i in range(1024):
-                if ord(host_data[i]) != 2:
-                    open_success = False
-                    break
-        result_queue.put(open_success)
-        lib.cudaDeviceReset()
-
-
-def consumer(
-    batch_tgt: Sequence[int],
-    producer_queue,
-    consumer_queue,
-    result_queue,
-    cuda_visible_devices: Optional[str] = None,
-):
-    if cuda_visible_devices is not None:
-        update_environment_variables({"CUDA_VISIBLE_DEVICES": cuda_visible_devices})
-
-    lib = CudaRTLibrary()
-    for j in batch_tgt:
-        lib.cudaSetDevice(j)
-        handle = producer_queue.get()
-        open_success = False
-        try:
-            pointer = lib.cudaIpcOpenMemHandle(handle)  # type: ignore
-            open_success = True
-        except RuntimeError:
-            # cannot error out here, because the producer process
-            # is still waiting for the response.
-            pass
-        consumer_queue.put(open_success)
-        if open_success:
-            # modify the memory
-            lib.cudaMemset(pointer, 2, 1024)
-            lib.cudaDeviceSynchronize()
-            # use two queues to simulate barrier
-            producer_queue.get()
-            consumer_queue.put(0)
-            # check if the memory is modified
-            host_data = (ctypes.c_char * 1024)()
-            lib.cudaMemcpy(host_data, pointer, 1024)  # type: ignore
-            for i in range(1024):
-                if ord(host_data[i]) != 2:
-                    open_success = False
-                    break
-        result_queue.put(open_success)
-        lib.cudaDeviceReset()
-
-
-def can_actually_p2p(
-    batch_src: Sequence[int],
-    batch_tgt: Sequence[int],
-) -> Sequence[bool]:
-    """
-    Usually, checking if P2P access is enabled can be done by
-    `torch.cuda.can_device_access_peer(src, tgt)`. However, sometimes
-    the driver might be broken, and `torch.cuda.can_device_access_peer(src, tgt)`
-    returns `True` even if P2P access is not actually possible.
-    See https://github.com/vllm-project/vllm/issues/2728 and
-    https://forums.developer.nvidia.com/t/direct-gpu-gpu-communication-does-not-seem-to-work-properly/283264/10
-    Therefore, we have to perform a real P2P access to check if it is actually
-    possible.
-
-    Note on p2p and cuda IPC:
-    Usually, one process uses one GPU:
-    GPU src --> cuda context src --> tensor src --> process src
-
-    We need to combine p2p and cuda IPC, so that:
-    GPU src --> cuda context src --> tensor src --> process src
-                                      |shared|
-    GPU tgt --> cuda context tgt --> tensor tgt --> process tgt
-    That is to say, process src creates a tensor in GPU src, passes IPC handle to
-    process tgt, and process tgt accesses the tensor in GPU tgt. Any operation on the
-    tensor in process tgt will be reflected in the tensor in process src, because
-    they are the same memory segment.
-    It is important to note that process tgt accesses the tensor in GPU tgt, not
-    GPU src. That's why we need p2p access.
-
-    The most time-consuming part is the process creation. To avoid creating
-    processes for every pair of GPUs, we use batched testing. We create two
-    processes for testing all pairs of GPUs in batch. The trick is to reset
-    the device after each test (which is not available in PyTorch).
-    """  # noqa
-    cuda_visible_devices = os.environ.get("CUDA_VISIBLE_DEVICES", None)
-    # pass the CUDA_VISIBLE_DEVICES to the child process
-    # to make sure they see the same set of GPUs
-
-    # make sure the processes are spawned
-    smp = mp.get_context("spawn")
-    producer_queue = smp.Queue()
-    consumer_queue = smp.Queue()
-    result_queue = smp.Queue()
-    p_src = smp.Process(
-        target=producer,
-        args=(
-            batch_src,
-            producer_queue,
-            consumer_queue,
-            result_queue,
-            cuda_visible_devices,
-        ),
-    )
-    p_tgt = smp.Process(
-        target=consumer,
-        args=(
-            batch_tgt,
-            producer_queue,
-            consumer_queue,
-            result_queue,
-            cuda_visible_devices,
-        ),
-    )
-    p_src.start()
-    p_tgt.start()
-    p_src.join()
-    p_tgt.join()
-    assert p_src.exitcode == 0 and p_tgt.exitcode == 0
-    result: List[bool] = []
-    for src, tgt in zip(batch_src, batch_tgt):
-        a = result_queue.get()
-        b = result_queue.get()
-        if a != b:
-            logger.warning(
-                "Two processes do not agree on the P2P access"
-                " status on %d -> %d, treat as disabled.",
-                src,
-                tgt,
-            )
-            result.append(False)
-        else:
-            result.append(a)
-    return result
-
-
-# why do we need this cache?
-# we are testing peer-to-peer (p2p) access between GPUs,across processes.
-# if we test it every time, it will be very slow, because we need to create
-#  N * N * 2 processes, where N is the world size. This is very slow.
-# to reduce the time, we use a cache file to store the p2p access status.
-# the cache file is generated by the master process if it does not exist.
-# then all the processes can read the cache file to check the p2p access status.
-# Note that the cache file is suffixed by the CUDA_VISIBLE_DEVICES, so that we
-#  can have different cache files for different CUDA_VISIBLE_DEVICES settings,
-#  e.g. used by different vllm engines. The device id in the cache file is a
-#  **local** device id, i.e. from 0 to num_dev-1, where num_dev is the number
-#  of visible devices in the vllm engine.
-_gpu_p2p_access_cache: Optional[Dict[str, bool]] = None
-
-
-def gpu_p2p_access_check(src: int, tgt: int) -> bool:
-    """Check if GPU src can access GPU tgt."""
-
-    # if the cache variable is already calculated,
-    # read from the cache instead of checking it again
-    global _gpu_p2p_access_cache
-    if _gpu_p2p_access_cache is not None:
-        return _gpu_p2p_access_cache[f"{src}->{tgt}"]
-
-    is_distributed = dist.is_initialized()
-
-    num_dev = torch.cuda.device_count()
-    cuda_visible_devices = os.environ.get("CUDA_VISIBLE_DEVICES", None)
-    if cuda_visible_devices is None:
-        cuda_visible_devices = ",".join(str(i) for i in range(num_dev))
-
-    # VLLM_CACHE_ROOT -> SGLANG_CACHE_ROOT
-    # "~/.cache/vllm" -> envs.SGLANG_CACHE_DIR
-    SGLANG_CACHE_ROOT = os.path.expanduser(sglang_envs.SGLANG_CACHE_DIR.get())
-    path = os.path.join(
-        SGLANG_CACHE_ROOT, f"gpu_p2p_access_cache_for_{cuda_visible_devices}.json"
-    )
-    cache_dir = os.path.dirname(path)
-    try:
-        os.makedirs(cache_dir, exist_ok=True)
-    except (FileExistsError, NotADirectoryError):
-        if not os.path.isdir(cache_dir):
-            # Path exists as a file (stale cache/lock). Remove and retry.
-            try:
-                os.remove(cache_dir)
-            except OSError:
-                pass
-            os.makedirs(cache_dir, exist_ok=True)
-    from sglang.srt.distributed.parallel_state import get_world_group
-
-    if (not is_distributed or get_world_group().local_rank == 0) and (
-        not os.path.exists(path)
-    ):
-        # only the local master process (with local_rank == 0) can
-        #  enter this block to calculate the cache
-        logger.info("generating GPU P2P access cache in %s", path)
-        cache: Dict[str, bool] = {}
-        ids = list(range(num_dev))
-        # batch of all pairs of GPUs
-        batch_src, batch_tgt = zip(*list(product(ids, ids)))
-        # NOTE: we use `subprocess` rather than `multiprocessing` here
-        # because the caller might not have `if __name__ == "__main__":`,
-        # in that case we cannot use spawn method in multiprocessing.
-        # However, `can_actually_p2p` requires spawn method.
-        # The fix is, we use `subprocess` to call the function,
-        # where we have `if __name__ == "__main__":` in this file.
-
-        # use a temporary file to store the result
-        # we don't use the output of the subprocess directly,
-        # because the subprocess might produce logging output
-        with tempfile.NamedTemporaryFile() as output_file:
-            input_bytes = pickle.dumps((batch_src, batch_tgt, output_file.name))
-            returned = subprocess.run(
-                [sys.executable, __file__], input=input_bytes, capture_output=True
-            )
-            # check if the subprocess is successful
-            try:
-                returned.check_returncode()
-            except Exception as e:
-                # wrap raised exception to provide more information
-                raise RuntimeError(
-                    f"Error happened when batch testing "
-                    f"peer-to-peer access from {batch_src} to {batch_tgt}:\n"
-                    f"{returned.stderr.decode()}"
-                ) from e
-            with open(output_file.name, "rb") as f:
-                result = pickle.load(f)
-        for _i, _j, r in zip(batch_src, batch_tgt, result):
-            cache[f"{_i}->{_j}"] = r
-        with open(path, "w") as f:
-            json.dump(cache, f, indent=4)
-    if is_distributed:
-        get_world_group().barrier()
-    logger.info("reading GPU P2P access cache from %s", path)
-    with open(path) as f:
-        cache = json.load(f)
-    _gpu_p2p_access_cache = cache
-    return _gpu_p2p_access_cache[f"{src}->{tgt}"]
-
-
-def with_nvml_context(fn: Callable[_P, _R]) -> Callable[_P, _R]:
-    @wraps(fn)
-    def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _R:
-        if _is_hip:
-            try:
-                amdsmi_init()
-                return fn(*args, **kwargs)
-            finally:
-                amdsmi_shut_down()
-        else:
-            pynvml.nvmlInit()
-            try:
-                return fn(*args, **kwargs)
-            finally:
-                pynvml.nvmlShutdown()
-
-    return wrapper
-
-
-@with_nvml_context
-def is_full_nvlink(physical_device_ids: List[int], world_size: int) -> bool:
-    if _is_hip:
-        """
-        query if the set of gpus are fully connected by xgmi (1 hop)
-        """
-        handles = [amdsmi_get_processor_handles()[i] for i in physical_device_ids]
-        for i, handle in enumerate(handles):
-            for j, peer_handle in enumerate(handles):
-                if i < j:
-                    try:
-                        link_type = amdsmi_topo_get_link_type(handle, peer_handle)
-                        # type is 2 for XGMI
-                        if link_type["hops"] != 1 or link_type["type"] != 2:
-                            return False
-                    except AmdSmiException as error:
-                        logger.error("AMD 1 hop XGMI detection failed.", exc_info=error)
-                        return False
-        return True
-    else:
-        """
-        query if the set of gpus are fully connected by nvlink (1 hop)
-        """
-        handles = [pynvml.nvmlDeviceGetHandleByIndex(i) for i in physical_device_ids]
-        for i, handle in enumerate(handles):
-            for j, peer_handle in enumerate(handles):
-                if i < j:
-                    try:
-                        p2p_status = pynvml.nvmlDeviceGetP2PStatus(
-                            handle, peer_handle, pynvml.NVML_P2P_CAPS_INDEX_NVLINK
-                        )
-                        if p2p_status != pynvml.NVML_P2P_STATUS_OK:
-                            return False
-                    except pynvml.NVMLError:
-                        logger.exception(
-                            "NVLink detection failed. This is normal if your"
-                            " machine has no NVLink equipped."
-                        )
-                        return False
-        return True
-
-
-@with_nvml_context
-def is_one_nvlink_clique(
-    group: torch.distributed.ProcessGroup, device: torch.device
-) -> bool:
-    """True iff every rank's GPU is in the same NVLink fabric clique (one NVL72 /
-    MNNVL domain). Such a clique shares a single NVLink address space even across
-    nodes, so custom-AR v2's symm-mem storage + fabric peer VAs are valid group-wide."""
-    if _is_hip:
-        return False
-    try:
-        clique = _gpu_fabric_clique(device)
-    except Exception as e:
-        logger.warning(
-            "GPU fabric clique query failed (%r); custom-AR stays intra-node.", e
-        )
-        clique = None
-    # Always all-gather (every rank calls it once) so a failed query on any rank
-    # resolves to a clean False rather than a collective mismatch.
-    world_size = dist.get_world_size(group=group)
-    gathered: List[object] = [None] * world_size
-    dist.all_gather_object(gathered, clique, group=group)
-    if any(c is None for c in gathered):
-        return False
-    return len(set(gathered)) == 1
-
-
-def is_weak_contiguous(inp: torch.Tensor):
-    return inp.is_contiguous() or (
-        inp.storage().nbytes() - inp.storage_offset() * inp.element_size()
-        == inp.numel() * inp.element_size()
-    )
-
-
-def can_p2p(rank: int, world_size: int) -> bool:
-    # SGLANG_SKIP_P2P_CHECK can be set to False in sglang
-    SGLANG_SKIP_P2P_CHECK = os.getenv("SGLANG_SKIP_P2P_CHECK", "0") == "1"
-    for i in range(world_size):
-        if i == rank:
-            continue
-        if SGLANG_SKIP_P2P_CHECK:
-            logger.info("Skipping P2P check and trusting the driver's P2P report.")
-            return torch.cuda.can_device_access_peer(rank, i)
-        if not gpu_p2p_access_check(rank, i):
-            return False
-    return True
-
-
-def can_use_custom_all_reduce_with_nvlink(
-    group: torch.distributed.ProcessGroup,
-    device: torch.device,
-    supported_world_size: List[int],
-    cls_name: str,
-) -> Optional[bool]:  # None if fail; otherwise return whether NVLink is available
-    assert (
-        dist.get_backend(group) != dist.Backend.NCCL
-    ), f"{cls_name} should be attached to a non-NCCL group."
-
-    rank = dist.get_rank(group=group)
-    world_size = dist.get_world_size(group=group)
-
-    # No need to initialize custom allreduce for single GPU case.
-    if world_size == 1:
-        return
-
-    # No need to initialize custom allreduce for multi-node case.
-    if not all(in_the_same_node_as(group, source_rank=0)):
-        logger.warning(
-            f"{cls_name} is disabled because this process group" " spans across nodes."
-        )
-        return
-
-    # For not supported world size, we disable custom allreduce.
-    if world_size not in supported_world_size:
-        logger.warning(
-            f"{cls_name} is disabled due to an unsupported world"
-            f" size: {world_size}. Supported world sizes: {supported_world_size}. "
-            "To silence this warning, specify disable_custom_all_reduce=True explicitly.",
-        )
-        return
-
-    cuda_visible_devices = os.environ.get("CUDA_VISIBLE_DEVICES", None)
-    if cuda_visible_devices:
-        device_ids = list(map(int, cuda_visible_devices.split(",")))
-    else:
-        device_ids = list(range(torch.cuda.device_count()))
-    physical_device_id = device_ids[device.index]
-    tensor = torch.tensor([physical_device_id], dtype=torch.int, device="cpu")
-    gather_list = [
-        torch.tensor([0], dtype=torch.int, device="cpu") for _ in range(world_size)
-    ]
-    dist.all_gather(gather_list, tensor, group=group)
-    physical_device_ids = [int(t) for t in gather_list]
-    full_nvlink = is_full_nvlink(physical_device_ids, world_size)
-
-    # test nvlink first, this will filter out most of the cases
-    # where custom allreduce is not supported
-    # this checks hardware and driver support for NVLink
-    if False:
-        logger.warning(
-            f"{cls_name} is disabled because it's not supported on"
-            " more than two PCIe-only GPUs. To silence this warning, "
-            "specify disable_custom_all_reduce=True explicitly."
-        )
-        return
-
-    # test P2P capability, this checks software/cudaruntime support
-    # this is expensive to compute at the first time
-    # then we cache the result
-    # On AMD GPU, p2p is always enabled between XGMI connected GPUs
-    if not _is_hip and not can_p2p(rank, world_size):
-        logger.warning(
-            f"{cls_name} is disabled because your platform lacks "
-            "GPU P2P capability or P2P test failed. To silence this "
-            "warning, specify disable_custom_all_reduce=True explicitly."
-        )
-        return
-
-    return full_nvlink
-
-
-if __name__ == "__main__":
-    batch_src, batch_tgt, output_file = pickle.loads(sys.stdin.buffer.read())
-    result = can_actually_p2p(batch_src, batch_tgt)
-    with open(output_file, "wb") as f:
-        f.write(pickle.dumps(result))

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b/all_results.jsonl

@@ -1,20 +0,0 @@
-{"tag": "b3_16k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9501, "corpus_window": {"start": 2300000, "end": 2431072}, "ok": 8, "failed": 0, "wall_s": 82.58, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 49.6, "input_throughput_tok_s": 1587.15, "ttft_s": {"mean": 3.9252, "p50": 3.9266, "p95": 3.9787, "max": 3.9787, "min": 3.8899}, "tpot_s": {"mean": 0.0125, "p50": 0.0128, "p95": 0.0137, "max": 0.0137, "min": 0.0112}, "e2e_s": {"mean": 10.3227, "p50": 10.4215, "p95": 10.9871, "max": 10.9871, "min": 9.6445}, "per_req_out_tok_s_e2e": {"mean": 49.6617, "p50": 49.8845, "p95": 53.0871, "max": 53.0871, "min": 46.6001}, "per_req_decode_tok_s": {"mean": 80.2896, "p50": 80.4694, "p95": 89.5483, "max": 89.5483, "min": 72.9492}, "spec_accept_length_mean": 2.138, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 16, "new_tokens": 131072, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c8", "arm": "e7b", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9502, "corpus_window": {"start": 2300000, "end": 2562144}, "ok": 16, "failed": 0, "wall_s": 100.63, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 8192, "output_throughput_tok_s": 81.41, "input_throughput_tok_s": 2605.13, "ttft_s": {"mean": 11.7518, "p50": 6.5296, "p95": 32.5233, "max": 32.5233, "min": 3.9869}, "tpot_s": {"mean": 0.0716, "p50": 0.0731, "p95": 0.1351, "max": 0.1351, "min": 0.0266}, "e2e_s": {"mean": 48.3305, "p50": 50.9027, "p95": 78.2404, "max": 78.2404, "min": 17.5929}, "per_req_out_tok_s_e2e": {"mean": 12.9171, "p50": 10.4857, "p95": 29.1027, "max": 29.1027, "min": 6.5439}, "per_req_decode_tok_s": {"mean": 16.8775, "p50": 15.7396, "p95": 37.6553, "max": 37.6553, "min": 7.4162}, "spec_accept_length_mean": 2.375, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c16", "arm": "e7b", "summary": {"concurrency": 16, "num_requests": 32, "run_id": 9503, "corpus_window": {"start": 2300000, "end": 2824288}, "ok": 32, "failed": 0, "wall_s": 236.44, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 16384, "output_throughput_tok_s": 69.29, "input_throughput_tok_s": 2217.4, "ttft_s": {"mean": 23.5964, "p50": 13.173, "p95": 60.864, "max": 90.3879, "min": 4.2969}, "tpot_s": {"mean": 0.1733, "p50": 0.1784, "p95": 0.2978, "max": 0.3055, "min": 0.0668}, "e2e_s": {"mean": 112.1663, "p50": 105.5499, "p95": 170.4271, "max": 183.3467, "min": 52.7259}, "per_req_out_tok_s_e2e": {"mean": 5.0312, "p50": 4.8797, "p95": 8.0542, "max": 9.7106, "min": 2.7925}, "per_req_decode_tok_s": {"mean": 6.7269, "p50": 5.8271, "p95": 12.9785, "max": 15.0103, "min": 3.2802}, "spec_accept_length_mean": 2.517, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c32", "arm": "e7b", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9504, "corpus_window": {"start": 2300000, "end": 3348576}, "ok": 64, "failed": 0, "wall_s": 414.96, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 32768, "output_throughput_tok_s": 78.97, "input_throughput_tok_s": 2526.91, "ttft_s": {"mean": 100.2552, "p50": 109.5841, "p95": 169.6812, "max": 190.7286, "min": 5.2261}, "tpot_s": {"mean": 0.1698, "p50": 0.1728, "p95": 0.2599, "max": 0.3191, "min": 0.0328}, "e2e_s": {"mean": 187.0411, "p50": 190.6599, "p95": 271.5134, "max": 292.7775, "min": 86.5615}, "per_req_out_tok_s_e2e": {"mean": 2.9412, "p50": 2.6906, "p95": 4.5951, "max": 5.9149, "min": 1.7488}, "per_req_decode_tok_s": {"mean": 7.1251, "p50": 5.846, "p95": 15.414, "max": 30.5614, "min": 3.1404}, "spec_accept_length_mean": 2.767, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c64", "arm": "e7b", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9505, "corpus_window": {"start": 2300000, "end": 4397152}, "ok": 128, "failed": 0, "wall_s": 777.55, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 65536, "output_throughput_tok_s": 84.29, "input_throughput_tok_s": 2697.14, "ttft_s": {"mean": 240.0965, "p50": 287.3833, "p95": 345.6933, "max": 380.9581, "min": 5.2288}, "tpot_s": {"mean": 0.168, "p50": 0.1702, "p95": 0.2315, "max": 0.3119, "min": 0.0404}, "e2e_s": {"mean": 325.934, "p50": 364.8118, "p95": 432.439, "max": 474.7226, "min": 83.7008}, "per_req_out_tok_s_e2e": {"mean": 1.8253, "p50": 1.4064, "p95": 4.196, "max": 6.117, "min": 1.0785}, "per_req_decode_tok_s": {"mean": 6.6174, "p50": 5.8876, "p95": 12.5326, "max": 24.7977, "min": 3.2126}, "spec_accept_length_mean": 2.986, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9511, "corpus_window": {"start": 4500000, "end": 4508192}, "ok": 8, "failed": 0, "wall_s": 15.82, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 1024, "output_throughput_tok_s": 64.74, "input_throughput_tok_s": 517.93, "ttft_s": {"mean": 0.3034, "p50": 0.3015, "p95": 0.3258, "max": 0.3258, "min": 0.2959}, "tpot_s": {"mean": 0.0132, "p50": 0.0135, "p95": 0.0143, "max": 0.0143, "min": 0.0119}, "e2e_s": {"mean": 1.9769, "p50": 2.0115, "p95": 2.1177, "max": 2.1177, "min": 1.8091}, "per_req_out_tok_s_e2e": {"mean": 64.9178, "p50": 66.8893, "p95": 70.753, "max": 70.753, "min": 60.442}, "per_req_decode_tok_s": {"mean": 76.7983, "p50": 79.4095, "p95": 84.9204, "max": 84.9204, "min": 70.4319}, "spec_accept_length_mean": 2.028, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 8, "new_tokens": 8192, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c8", "arm": "e7b", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9512, "corpus_window": {"start": 4500000, "end": 4516384}, "ok": 16, "failed": 0, "wall_s": 13.44, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 2048, "output_throughput_tok_s": 152.4, "input_throughput_tok_s": 1219.24, "ttft_s": {"mean": 1.21, "p50": 1.7505, "p95": 2.1601, "max": 2.1601, "min": 0.387}, "tpot_s": {"mean": 0.0414, "p50": 0.042, "p95": 0.0537, "max": 0.0537, "min": 0.0319}, "e2e_s": {"mean": 6.4682, "p50": 6.5412, "p95": 8.9777, "max": 8.9777, "min": 4.4513}, "per_req_out_tok_s_e2e": {"mean": 20.4798, "p50": 20.2955, "p95": 28.7556, "max": 28.7556, "min": 14.2576}, "per_req_decode_tok_s": {"mean": 24.8432, "p50": 24.8024, "p95": 31.5592, "max": 31.5592, "min": 18.7758}, "spec_accept_length_mean": 2.048, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 10, "new_tokens": 16384, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c32", "arm": "e7b", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9513, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 81.8, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 100.14, "input_throughput_tok_s": 801.15, "ttft_s": {"mean": 17.5622, "p50": 21.7567, "p95": 25.6003, "max": 27.8932, "min": 1.5226}, "tpot_s": {"mean": 0.1466, "p50": 0.1501, "p95": 0.1905, "max": 0.1952, "min": 0.088}, "e2e_s": {"mean": 36.1812, "p50": 40.1766, "p95": 46.8967, "max": 47.1947, "min": 18.2249}, "per_req_out_tok_s_e2e": {"mean": 3.8358, "p50": 3.1917, "p95": 6.6145, "max": 7.0234, "min": 2.7122}, "per_req_decode_tok_s": {"mean": 7.1041, "p50": 6.7208, "p95": 10.1791, "max": 11.4499, "min": 5.164}, "spec_accept_length_mean": 2.054, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 38, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c64", "arm": "e7b", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9514, "corpus_window": {"start": 4500000, "end": 4631072}, "ok": 128, "failed": 0, "wall_s": 158.73, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 16384, "output_throughput_tok_s": 103.22, "input_throughput_tok_s": 825.75, "ttft_s": {"mean": 46.7369, "p50": 57.65, "p95": 65.8132, "max": 70.7061, "min": 1.5163}, "tpot_s": {"mean": 0.1445, "p50": 0.144, "p95": 0.1777, "max": 0.2125, "min": 0.0759}, "e2e_s": {"mean": 65.0893, "p50": 74.2949, "p95": 84.7332, "max": 89.7485, "min": 18.3747}, "per_req_out_tok_s_e2e": {"mean": 2.4029, "p50": 1.7238, "p95": 6.272, "max": 6.9661, "min": 1.4262}, "per_req_decode_tok_s": {"mean": 7.1513, "p50": 7.0024, "p95": 9.0245, "max": 13.2811, "min": 4.7422}, "spec_accept_length_mean": 2.087, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 85, "new_tokens": 131072, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9521, "corpus_window": {"start": 4700000, "end": 4708192}, "ok": 8, "failed": 0, "wall_s": 242.18, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 32768, "output_throughput_tok_s": 135.31, "input_throughput_tok_s": 33.83, "ttft_s": {"mean": 0.3142, "p50": 0.3133, "p95": 0.3204, "max": 0.3204, "min": 0.3097}, "tpot_s": {"mean": 0.0073, "p50": 0.0078, "p95": 0.0087, "max": 0.0087, "min": 0.0059}, "e2e_s": {"mean": 30.2716, "p50": 32.1753, "p95": 35.9141, "max": 35.9141, "min": 24.509}, "per_req_out_tok_s_e2e": {"mean": 137.6058, "p50": 136.5088, "p95": 167.1221, "max": 167.1221, "min": 114.0498}, "per_req_decode_tok_s": {"mean": 139.1062, "p50": 137.9513, "p95": 169.3421, "max": 169.3421, "min": 115.0748}, "spec_accept_length_mean": 3.701, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 8, "new_tokens": 8192, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c8", "arm": "e7b", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9522, "corpus_window": {"start": 4700000, "end": 4716384}, "ok": 16, "failed": 0, "wall_s": 159.24, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 65536, "output_throughput_tok_s": 411.54, "input_throughput_tok_s": 102.89, "ttft_s": {"mean": 0.9638, "p50": 1.2709, "p95": 1.6805, "max": 1.6805, "min": 0.378}, "tpot_s": {"mean": 0.0181, "p50": 0.0175, "p95": 0.0225, "max": 0.0225, "min": 0.0153}, "e2e_s": {"mean": 75.1101, "p50": 73.2342, "p95": 93.9711, "max": 93.9711, "min": 64.3187}, "per_req_out_tok_s_e2e": {"mean": 55.3003, "p50": 58.2448, "p95": 63.6829, "max": 63.6829, "min": 43.5879}, "per_req_decode_tok_s": {"mean": 55.9977, "p50": 58.5719, "p95": 65.3811, "max": 65.3811, "min": 44.3769}, "spec_accept_length_mean": 4.003, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 10, "new_tokens": 16384, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c32", "arm": "e7b", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9523, "corpus_window": {"start": 4700000, "end": 4765536}, "ok": 64, "failed": 0, "wall_s": 1040.79, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 262144, "output_throughput_tok_s": 251.87, "input_throughput_tok_s": 62.97, "ttft_s": {"mean": 195.07, "p50": 240.7323, "p95": 304.7528, "max": 334.6933, "min": 1.3472}, "tpot_s": {"mean": 0.0607, "p50": 0.0581, "p95": 0.0746, "max": 0.0923, "min": 0.0465}, "e2e_s": {"mean": 443.6795, "p50": 478.2794, "p95": 587.5712, "max": 597.8883, "min": 219.3532}, "per_req_out_tok_s_e2e": {"mean": 10.0455, "p50": 8.6106, "p95": 17.4441, "max": 18.6731, "min": 6.8508}, "per_req_decode_tok_s": {"mean": 16.784, "p50": 17.4276, "p95": 20.0358, "max": 21.5315, "min": 10.8381}, "spec_accept_length_mean": 3.852, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 51, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9531, "corpus_window": {"start": 5000000, "end": 5524288}, "ok": 8, "failed": 0, "wall_s": 182.23, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 22.48, "input_throughput_tok_s": 2877.06, "ttft_s": {"mean": 17.0384, "p50": 17.022, "p95": 17.1703, "max": 17.1703, "min": 17.0028}, "tpot_s": {"mean": 0.0112, "p50": 0.0117, "p95": 0.0135, "max": 0.0135, "min": 0.008}, "e2e_s": {"mean": 22.7786, "p50": 23.0017, "p95": 23.9301, "max": 23.9301, "min": 21.118}, "per_req_out_tok_s_e2e": {"mean": 22.5108, "p50": 22.5486, "p95": 24.2447, "max": 24.2447, "min": 21.3957}, "per_req_decode_tok_s": {"mean": 91.4983, "p50": 90.0463, "p95": 125.007, "max": 125.007, "min": 74.0091}, "spec_accept_length_mean": 2.468, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c4", "arm": "e7b", "summary": {"concurrency": 4, "num_requests": 8, "run_id": 9532, "corpus_window": {"start": 5000000, "end": 5524288}, "ok": 8, "failed": 0, "wall_s": 159.24, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 25.72, "input_throughput_tok_s": 3292.45, "ttft_s": {"mean": 30.5492, "p50": 18.5951, "p95": 68.8675, "max": 68.8675, "min": 17.0513}, "tpot_s": {"mean": 0.0949, "p50": 0.1145, "p95": 0.1546, "max": 0.1546, "min": 0.02}, "e2e_s": {"mean": 79.0616, "p50": 80.1169, "p95": 131.9282, "max": 131.9282, "min": 27.2645}, "per_req_out_tok_s_e2e": {"mean": 8.1791, "p50": 6.6417, "p95": 18.779, "max": 18.779, "min": 3.8809}, "per_req_decode_tok_s": {"mean": 16.1959, "p50": 11.2172, "p95": 50.1581, "max": 50.1581, "min": 6.4814}, "spec_accept_length_mean": 2.438, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c8", "arm": "e7b", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9533, "corpus_window": {"start": 5000000, "end": 6048576}, "ok": 16, "failed": 0, "wall_s": 319.66, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 8192, "output_throughput_tok_s": 25.63, "input_throughput_tok_s": 3280.31, "ttft_s": {"mean": 90.9186, "p50": 96.5085, "p95": 149.4465, "max": 149.4465, "min": 18.5948}, "tpot_s": {"mean": 0.1058, "p50": 0.1221, "p95": 0.1575, "max": 0.1575, "min": 0.0172}, "e2e_s": {"mean": 144.9847, "p50": 155.6435, "p95": 211.845, "max": 211.845, "min": 75.7166}, "per_req_out_tok_s_e2e": {"mean": 3.7942, "p50": 3.6883, "p95": 6.7621, "max": 6.7621, "min": 2.4169}, "per_req_decode_tok_s": {"mean": 13.0455, "p50": 8.4248, "p95": 58.0944, "max": 58.0944, "min": 6.3604}, "spec_accept_length_mean": 2.442, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9541, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 343.36, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 11.93, "input_throughput_tok_s": 3053.83, "ttft_s": {"mean": 37.5917, "p50": 37.5968, "p95": 37.6012, "max": 37.6012, "min": 37.5564}, "tpot_s": {"mean": 0.0104, "p50": 0.0104, "p95": 0.0123, "max": 0.0123, "min": 0.009}, "e2e_s": {"mean": 42.9201, "p50": 42.9071, "p95": 43.9018, "max": 43.9018, "min": 42.1336}, "per_req_out_tok_s_e2e": {"mean": 11.9312, "p50": 11.9694, "p95": 12.1518, "max": 12.1518, "min": 11.6624}, "per_req_decode_tok_s": {"mean": 97.1304, "p50": 98.8664, "p95": 111.8659, "max": 111.8659, "min": 81.2653}, "spec_accept_length_mean": 2.728, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c2", "arm": "e7b", "summary": {"concurrency": 2, "num_requests": 8, "run_id": 9542, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 333.36, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 12.29, "input_throughput_tok_s": 3145.44, "ttft_s": {"mean": 48.2081, "p50": 39.1403, "p95": 75.8082, "max": 75.8082, "min": 37.5834}, "tpot_s": {"mean": 0.068, "p50": 0.0858, "p95": 0.1622, "max": 0.1622, "min": 0.0099}, "e2e_s": {"mean": 82.9494, "p50": 84.1258, "p95": 122.0311, "max": 122.0311, "min": 42.6804}, "per_req_out_tok_s_e2e": {"mean": 7.0763, "p50": 6.2562, "p95": 11.9961, "max": 11.9961, "min": 4.1957}, "per_req_decode_tok_s": {"mean": 37.0402, "p50": 12.1918, "p95": 100.963, "max": 100.963, "min": 6.1768}, "spec_accept_length_mean": 2.769, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c4", "arm": "e7b", "summary": {"concurrency": 4, "num_requests": 8, "run_id": 9543, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 334.71, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 12.24, "input_throughput_tok_s": 3132.76, "ttft_s": {"mean": 110.1385, "p50": 120.8044, "p95": 159.4963, "max": 159.4963, "min": 39.1664}, "tpot_s": {"mean": 0.0687, "p50": 0.0152, "p95": 0.1634, "max": 0.1634, "min": 0.0097}, "e2e_s": {"mean": 145.2581, "p50": 130.4964, "p95": 204.1758, "max": 204.1758, "min": 83.563}, "per_req_out_tok_s_e2e": {"mean": 3.8065, "p50": 4.04, "p95": 6.1271, "max": 6.1271, "min": 2.5076}, "per_req_decode_tok_s": {"mean": 54.2853, "p50": 73.1655, "p95": 103.6085, "max": 103.6085, "min": 6.1313}, "spec_accept_length_mean": 2.595, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_256k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 1, "run_id": 9551, "corpus_window": {"start": 8400000, "end": 8662144}, "ok": 1, "failed": 0, "wall_s": 0.5, "input_len": 262144, "shared_len": 0, "unique_len": 262144, "output_len": 1, "output_tokens_total": 1, "output_throughput_tok_s": 2.01, "input_throughput_tok_s": 527883.82, "ttft_s": {"mean": 0.4955, "p50": 0.4955, "p95": 0.4955, "max": 0.4955, "min": 0.4955}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 0.4957, "p50": 0.4957, "p95": 0.4957, "max": 0.4957, "min": 0.4957}, "per_req_out_tok_s_e2e": {"mean": 2.0172, "p50": 2.0172, "p95": 2.0172, "max": 2.0172, "min": 2.0172}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 1, "new_tokens": 64, "cached_tokens": 262080, "hit_rate": 0.9998}}}
-{"tag": "b52_256k_c1", "arm": "e7b", "summary": {"concurrency": 1, "num_requests": 1, "run_id": 9551, "corpus_window": {"start": 9900000, "end": 10162144}, "ok": 1, "failed": 0, "wall_s": 91.45, "input_len": 262144, "shared_len": 0, "unique_len": 262144, "output_len": 1, "output_tokens_total": 1, "output_throughput_tok_s": 0.01, "input_throughput_tok_s": 2866.59, "ttft_s": {"mean": 91.4465, "p50": 91.4465, "p95": 91.4465, "max": 91.4465, "min": 91.4465}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 91.4468, "p50": 91.4468, "p95": 91.4468, "max": 91.4468, "min": 91.4468}, "per_req_out_tok_s_e2e": {"mean": 0.0109, "p50": 0.0109, "p95": 0.0109, "max": 0.0109, "min": 0.0109}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b/md5_assets.txt

@@ -1,3 +0,0 @@
-a8f3909211adb92501cf007ab71d3b29  /root/corpus_ids.json
-1e34dd8d813ccb7ac8243e8d1573a49e  /root/bench_corpus_v2.py
-4c126d067d33b5ea27c268f37561634c  /root/extract_summary.py

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b/status.txt

@@ -1,20 +0,0 @@
-b3_16k_c1 OK
-b3_16k_c8 OK
-b3_16k_c16 OK
-b3_16k_c32 OK
-b3_16k_c64 OK
-b41_1k_c1 OK
-b41_1k_c8 OK
-b41_1k_c32 OK
-b41_1k_c64 OK
-b42_1k4k_c1 OK
-b42_1k4k_c8 OK
-b42_1k4k_c32 OK
-b42_1k4k_c64 BENCH_FAIL rc=143
-b51_64k_c1 OK
-b51_64k_c4 OK
-b51_64k_c8 OK
-b51_128k_c1 OK
-b51_128k_c2 OK
-b51_128k_c4 OK
-b52_256k_c1 HIT_FAIL_FINAL

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b64/all_results.jsonl

@@ -1,10 +0,0 @@
-{"tag": "b3_16k_c8", "arm": "e7b64", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9602, "corpus_window": {"start": 2300000, "end": 2562144}, "ok": 16, "failed": 0, "wall_s": 97.66, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 8192, "output_throughput_tok_s": 83.88, "input_throughput_tok_s": 2684.26, "ttft_s": {"mean": 12.1187, "p50": 8.3735, "p95": 32.4651, "max": 32.4651, "min": 3.9862}, "tpot_s": {"mean": 0.0692, "p50": 0.0688, "p95": 0.1361, "max": 0.1361, "min": 0.0264}, "e2e_s": {"mean": 47.483, "p50": 50.5394, "p95": 78.7413, "max": 78.7413, "min": 18.1949}, "per_req_out_tok_s_e2e": {"mean": 13.2102, "p50": 11.2272, "p95": 28.1397, "max": 28.1397, "min": 6.5023}, "per_req_decode_tok_s": {"mean": 18.5895, "p50": 14.9416, "p95": 37.9333, "max": 37.9333, "min": 7.3633}, "spec_accept_length_mean": 2.556, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c16", "arm": "e7b64", "summary": {"concurrency": 16, "num_requests": 32, "run_id": 9603, "corpus_window": {"start": 2300000, "end": 2824288}, "ok": 32, "failed": 0, "wall_s": 177.89, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 16384, "output_throughput_tok_s": 92.1, "input_throughput_tok_s": 2947.27, "ttft_s": {"mean": 19.7632, "p50": 7.8805, "p95": 59.91, "max": 63.6881, "min": 4.062}, "tpot_s": {"mean": 0.1297, "p50": 0.134, "p95": 0.2288, "max": 0.2759, "min": 0.0334}, "e2e_s": {"mean": 86.0176, "p50": 83.7193, "p95": 145.8411, "max": 150.1856, "min": 22.7897}, "per_req_out_tok_s_e2e": {"mean": 7.4324, "p50": 6.5209, "p95": 16.0562, "max": 22.4663, "min": 3.4091}, "per_req_decode_tok_s": {"mean": 10.0542, "p50": 7.5804, "p95": 27.3398, "max": 29.9645, "min": 3.6318}, "spec_accept_length_mean": 2.485, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c32", "arm": "e7b64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9604, "corpus_window": {"start": 2300000, "end": 3348576}, "ok": 64, "failed": 0, "wall_s": 335.79, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 32768, "output_throughput_tok_s": 97.58, "input_throughput_tok_s": 3122.67, "ttft_s": {"mean": 82.6459, "p50": 87.8342, "p95": 140.3273, "max": 154.661, "min": 5.2339}, "tpot_s": {"mean": 0.1381, "p50": 0.1443, "p95": 0.2136, "max": 0.2775, "min": 0.026}, "e2e_s": {"mean": 153.2127, "p50": 153.6215, "p95": 227.6964, "max": 243.4494, "min": 78.5488}, "per_req_out_tok_s_e2e": {"mean": 3.581, "p50": 3.3906, "p95": 5.2619, "max": 6.5182, "min": 2.1031}, "per_req_decode_tok_s": {"mean": 9.3726, "p50": 7.1484, "p95": 24.3778, "max": 38.5186, "min": 3.6109}, "spec_accept_length_mean": 2.777, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c64", "arm": "e7b64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9605, "corpus_window": {"start": 2300000, "end": 4397152}, "ok": 128, "failed": 0, "wall_s": 657.79, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 65536, "output_throughput_tok_s": 99.63, "input_throughput_tok_s": 3188.2, "ttft_s": {"mean": 204.2735, "p50": 244.6246, "p95": 293.2111, "max": 322.84, "min": 5.231}, "tpot_s": {"mean": 0.1425, "p50": 0.1465, "p95": 0.2146, "max": 0.2766, "min": 0.0297}, "e2e_s": {"mean": 277.1081, "p50": 309.1898, "p95": 368.9412, "max": 410.789, "min": 77.0243}, "per_req_out_tok_s_e2e": {"mean": 2.133, "p50": 1.6583, "p95": 4.7327, "max": 6.6473, "min": 1.2464}, "per_req_decode_tok_s": {"mean": 8.0518, "p50": 6.8676, "p95": 16.3762, "max": 33.7377, "min": 3.6219}, "spec_accept_length_mean": 2.954, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c8", "arm": "e7b64", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9612, "corpus_window": {"start": 4500000, "end": 4516384}, "ok": 16, "failed": 0, "wall_s": 13.53, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 2048, "output_throughput_tok_s": 151.33, "input_throughput_tok_s": 1210.61, "ttft_s": {"mean": 1.2544, "p50": 1.7591, "p95": 2.1654, "max": 2.1654, "min": 0.3904}, "tpot_s": {"mean": 0.0421, "p50": 0.041, "p95": 0.0537, "max": 0.0537, "min": 0.0336}, "e2e_s": {"mean": 6.6071, "p50": 6.6204, "p95": 8.6618, "max": 8.6618, "min": 4.8636}, "per_req_out_tok_s_e2e": {"mean": 19.981, "p50": 19.5317, "p95": 26.3178, "max": 26.3178, "min": 14.7775}, "per_req_decode_tok_s": {"mean": 24.481, "p50": 24.6734, "p95": 29.9848, "max": 29.9848, "min": 18.7591}, "spec_accept_length_mean": 1.992, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 9, "new_tokens": 16384, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c32", "arm": "e7b64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9613, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 35.75, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 229.18, "input_throughput_tok_s": 1833.42, "ttft_s": {"mean": 2.9919, "p50": 2.2946, "p95": 7.0943, "max": 7.1032, "min": 0.5556}, "tpot_s": {"mean": 0.1119, "p50": 0.1092, "p95": 0.1506, "max": 0.1798, "min": 0.0728}, "e2e_s": {"mean": 17.1982, "p50": 16.6865, "p95": 23.6063, "max": 25.9273, "min": 9.7993}, "per_req_out_tok_s_e2e": {"mean": 7.8468, "p50": 7.6792, "p95": 11.3781, "max": 13.0622, "min": 4.9369}, "per_req_decode_tok_s": {"mean": 9.4487, "p50": 9.2601, "p95": 12.987, "max": 13.852, "min": 5.6066}, "spec_accept_length_mean": 2.035, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 20, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c64", "arm": "e7b64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9614, "corpus_window": {"start": 4500000, "end": 4631072}, "ok": 128, "failed": 0, "wall_s": 60.18, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 16384, "output_throughput_tok_s": 272.24, "input_throughput_tok_s": 2177.88, "ttft_s": {"mean": 4.7575, "p50": 2.3032, "p95": 12.1513, "max": 13.4791, "min": 0.6534}, "tpot_s": {"mean": 0.1911, "p50": 0.1882, "p95": 0.2927, "max": 0.3315, "min": 0.1063}, "e2e_s": {"mean": 29.0341, "p50": 28.6144, "p95": 42.031, "max": 44.989, "min": 14.7114}, "per_req_out_tok_s_e2e": {"mean": 4.7331, "p50": 4.4822, "p95": 7.5631, "max": 8.7007, "min": 2.8451}, "per_req_decode_tok_s": {"mean": 5.6253, "p50": 5.3567, "p95": 8.1286, "max": 9.481, "min": 3.0404}, "spec_accept_length_mean": 2.094, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 27, "new_tokens": 131072, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c8", "arm": "e7b64", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9622, "corpus_window": {"start": 4700000, "end": 4716384}, "ok": 16, "failed": 0, "wall_s": 162.84, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 65536, "output_throughput_tok_s": 402.47, "input_throughput_tok_s": 100.62, "ttft_s": {"mean": 0.9469, "p50": 1.3161, "p95": 1.6549, "max": 1.6549, "min": 0.3801}, "tpot_s": {"mean": 0.0185, "p50": 0.0181, "p95": 0.0243, "max": 0.0243, "min": 0.0149}, "e2e_s": {"mean": 76.5237, "p50": 75.395, "p95": 101.2464, "max": 101.2464, "min": 61.5722}, "per_req_out_tok_s_e2e": {"mean": 54.4322, "p50": 55.1122, "p95": 66.5236, "max": 66.5236, "min": 40.4557}, "per_req_decode_tok_s": {"mean": 55.085, "p50": 56.3616, "p95": 66.9683, "max": 66.9683, "min": 41.1259}, "spec_accept_length_mean": 3.951, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 10, "new_tokens": 16384, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c32", "arm": "e7b64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9623, "corpus_window": {"start": 4700000, "end": 4765536}, "ok": 64, "failed": 0, "wall_s": 387.9, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 262144, "output_throughput_tok_s": 675.81, "input_throughput_tok_s": 168.95, "ttft_s": {"mean": 2.5762, "p50": 1.5202, "p95": 6.1895, "max": 6.2023, "min": 0.4797}, "tpot_s": {"mean": 0.043, "p50": 0.042, "p95": 0.0546, "max": 0.0604, "min": 0.0317}, "e2e_s": {"mean": 178.7057, "p50": 174.8683, "p95": 227.0262, "max": 253.2517, "min": 130.3185}, "per_req_out_tok_s_e2e": {"mean": 23.3694, "p50": 23.4457, "p95": 28.0936, "max": 31.4307, "min": 16.1736}, "per_req_decode_tok_s": {"mean": 23.6957, "p50": 23.8798, "p95": 28.3919, "max": 31.568, "min": 16.5524}, "spec_accept_length_mean": 3.82, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 36, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c64", "arm": "e7b64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9624, "corpus_window": {"start": 4700000, "end": 4831072}, "ok": 128, "failed": 0, "wall_s": 634.35, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 524288, "output_throughput_tok_s": 826.5, "input_throughput_tok_s": 206.62, "ttft_s": {"mean": 4.359, "p50": 2.4011, "p95": 12.1834, "max": 12.4977, "min": 0.536}, "tpot_s": {"mean": 0.0699, "p50": 0.0687, "p95": 0.0851, "max": 0.1015, "min": 0.0521}, "e2e_s": {"mean": 290.7916, "p50": 283.6317, "p95": 358.4567, "max": 420.1138, "min": 213.9936}, "per_req_out_tok_s_e2e": {"mean": 14.2904, "p50": 14.4588, "p95": 16.8051, "max": 19.1408, "min": 9.7497}, "per_req_decode_tok_s": {"mean": 14.5029, "p50": 14.5947, "p95": 16.9633, "max": 19.2102, "min": 9.8594}, "spec_accept_length_mean": 3.878, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 69, "new_tokens": 131072, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b64/md5_assets.txt

@@ -1,3 +0,0 @@
-a8f3909211adb92501cf007ab71d3b29  /root/corpus_ids.json
-1e34dd8d813ccb7ac8243e8d1573a49e  /root/bench_corpus_v2.py
-4c126d067d33b5ea27c268f37561634c  /root/extract_summary.py

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b64/retest_compare.md

@@ -1,27 +0,0 @@
-| 场景 | 并发 | Output TPS 旧→新 | Δ | TPOT P95 旧→新 | TTFT P95 旧→新 | accept 旧→新 |
-|---|---|---|---|---|---|---|
-| 主场景 16K→512 | 8 | 81.4 → **83.9** | +3% | 135.1 ms → 136.1 ms | 32.52 s → 32.47 s | 2.375 → 2.556 |
-| 主场景 16K→512 | 16 | 69.3 → **92.1** | +33% | 297.8 ms → 228.8 ms | 60.86 s → 59.91 s | 2.517 → 2.485 |
-| 主场景 16K→512 | 32 | 79.0 → **97.6** | +24% | 259.9 ms → 213.6 ms | 169.68 s → 140.33 s | 2.767 → 2.777 |
-| 主场景 16K→512 | 64 | 84.3 → **99.6** | +18% | 231.5 ms → 214.6 ms | 345.69 s → 293.21 s | 2.986 → 2.954 |
-| 4.1 短输入 1K→128 | 8 | 152 → **151** | -1% | 53.7 ms → 53.7 ms | 2.16 s → 2.17 s | 2.048 → 1.992 |
-| 4.1 短输入 1K→128 | 32 | 100 → **229** | +129% | 190.5 ms → 150.6 ms | 25.60 s → 7.09 s | 2.054 → 2.035 |
-| 4.1 短输入 1K→128 | 64 | 103 → **272** | +164% | 177.7 ms → 292.7 ms | 65.81 s → 12.15 s | 2.087 → 2.094 |
-| 4.2 长输出 1K→4K | 8 | 412 → **402** | -2% | 22.5 ms → 24.3 ms | 1.68 s → 1.65 s | 4.003 → 3.951 |
-| 4.2 长输出 1K→4K | 32 | 252 → **676** | +168% | 74.6 ms → 54.6 ms | 304.75 s → 6.19 s | 3.852 → 3.82 |
-| 4.2 长输出 1K→4K | 64 | (无旧数据) |  | 826 (新) | 85.1 ms | 3.878 |
-
-## 附录：E7b64 复测全量指标
-
-| 场景点 | ok/nreq | wall(s) | out tok/s | in tok/s | TTFT mean/p95/max(s) | TPOT mean/p95/max(ms) | retractions | accept | hit |
-|---|---|---|---|---|---|---|---|---|---|
-| b3_16k_c8 | 16/16 | 97.66 | 83.88 | 2684.26 | 12.12/32.47/32.47 | 69.2/136.1/136.1 | 0 | 2.556 | 0.0 |
-| b3_16k_c16 | 32/32 | 177.89 | 92.1 | 2947.27 | 19.76/59.91/63.69 | 129.7/228.8/275.9 | 0 | 2.485 | 0.0 |
-| b3_16k_c32 | 64/64 | 335.79 | 97.58 | 3122.67 | 82.65/140.33/154.66 | 138.1/213.6/277.5 | 0 | 2.777 | 0.0 |
-| b3_16k_c64 | 128/128 | 657.79 | 99.63 | 3188.2 | 204.27/293.21/322.84 | 142.5/214.6/276.6 | 0 | 2.954 | 0.0 |
-| b41_1k_c8 | 16/16 | 13.53 | 151.33 | 1210.61 | 1.25/2.17/2.17 | 42.1/53.7/53.7 | 0 | 1.992 | 0.0 |
-| b41_1k_c32 | 64/64 | 35.75 | 229.18 | 1833.42 | 2.99/7.09/7.10 | 111.9/150.6/179.8 | 0 | 2.035 | 0.0 |
-| b41_1k_c64 | 128/128 | 60.18 | 272.24 | 2177.88 | 4.76/12.15/13.48 | 191.1/292.7/331.5 | 0 | 2.094 | 0.0 |
-| b42_1k4k_c8 | 16/16 | 162.84 | 402.47 | 100.62 | 0.95/1.65/1.65 | 18.5/24.3/24.3 | 0 | 3.951 | 0.0 |
-| b42_1k4k_c32 | 64/64 | 387.9 | 675.81 | 168.95 | 2.58/6.19/6.20 | 43.0/54.6/60.4 | 0 | 3.82 | 0.0 |
-| b42_1k4k_c64 | 128/128 | 634.35 | 826.5 | 206.62 | 4.36/12.18/12.50 | 69.9/85.1/101.5 | 0 | 3.878 | 0.0 |

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/e7b64/status.txt

@@ -1,10 +0,0 @@
-b3_16k_c8 OK
-b3_16k_c16 OK
-b3_16k_c32 OK
-b3_16k_c64 OK
-b41_1k_c8 OK
-b41_1k_c32 OK
-b41_1k_c64 OK
-b42_1k4k_c8 OK
-b42_1k4k_c32 OK
-b42_1k4k_c64 OK

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/md5_ledger.txt

@@ -1,7 +0,0 @@
-1e34dd8d813ccb7ac8243e8d1573a49e  bench_corpus_v2.py
-4c126d067d33b5ea27c268f37561634c  extract_summary.py
-5892b44610b2ce61f533f1721105625e  run_b300_matrix.sh
-def3c64c5dc19e1d507080e3366c3762  deploy_glm53_pp4.sh
-21db641f1d997e26fcf1ddc97163b87e  deploy_glm53_607_exp.sh
-a8fc9a504c041acc40831a848b4985d8  patches/custom_all_reduce.py
-65a4d22bd87ae343e7d68c4879c14f98  patches/custom_all_reduce_utils.py

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/all_results_mrr48ctl.jsonl

@@ -1 +0,0 @@
-{"tag": "b41_1k_c32", "arm": "pp4mrr48ctl", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9681, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 30.75, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 266.42, "input_throughput_tok_s": 2131.36, "ttft_s": {"mean": 4.0402, "p50": 4.0687, "p95": 4.916, "max": 4.9315, "min": 1.6735}, "tpot_s": {"mean": 0.0884, "p50": 0.0878, "p95": 0.103, "max": 0.11, "min": 0.0809}, "e2e_s": {"mean": 15.2685, "p50": 15.3453, "p95": 15.5575, "max": 15.6425, "min": 15.0932}, "per_req_out_tok_s_e2e": {"mean": 8.3842, "p50": 8.4491, "p95": 8.4775, "max": 8.4806, "min": 8.1828}, "per_req_decode_tok_s": {"mean": 11.465, "p50": 11.4809, "p95": 12.363, "max": 12.4651, "min": 9.1633}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 36, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/all_results_pass1.jsonl

@@ -1,6 +0,0 @@
-{"tag": "b3_16k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9661, "corpus_window": {"start": 2300000, "end": 3348576}, "ok": 64, "failed": 0, "wall_s": 170.68, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 32768, "output_throughput_tok_s": 191.98, "input_throughput_tok_s": 6143.37, "ttft_s": {"mean": 26.5048, "p50": 27.0902, "p95": 46.4142, "max": 47.902, "min": 5.226}, "tpot_s": {"mean": 0.1148, "p50": 0.1149, "p95": 0.1529, "max": 0.1584, "min": 0.0713}, "e2e_s": {"mean": 85.1644, "p50": 85.6885, "p95": 86.1924, "max": 86.3287, "min": 84.3109}, "per_req_out_tok_s_e2e": {"mean": 6.0124, "p50": 6.0573, "p95": 6.0716, "max": 6.0728, "min": 5.9308}, "per_req_decode_tok_s": {"mean": 9.1855, "p50": 8.7353, "p95": 13.0741, "max": 14.0525, "min": 6.3271}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9662, "corpus_window": {"start": 2300000, "end": 4397152}, "ok": 128, "failed": 0, "wall_s": 326.79, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 65536, "output_throughput_tok_s": 200.55, "input_throughput_tok_s": 6417.5, "ttft_s": {"mean": 50.7799, "p50": 49.8304, "p95": 89.0393, "max": 146.8059, "min": 5.3643}, "tpot_s": {"mean": 0.1847, "p50": 0.1857, "p95": 0.2618, "max": 0.313, "min": 0.0538}, "e2e_s": {"mean": 145.1904, "p50": 145.9843, "p95": 149.0379, "max": 286.567, "min": 39.4665}, "per_req_out_tok_s_e2e": {"mean": 3.6818, "p50": 3.6238, "p95": 3.6472, "max": 12.973, "min": 1.7867}, "per_req_decode_tok_s": {"mean": 6.0216, "p50": 5.4382, "p95": 9.4602, "max": 18.6186, "min": 3.201}, "spec_accept_length_mean": null, "retractions_total": 3, "cache_hit_from_logs": {"prefill_batches": 532, "new_tokens": 8589312, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9663, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 35.58, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 230.24, "input_throughput_tok_s": 1841.95, "ttft_s": {"mean": 4.4841, "p50": 4.6833, "p95": 5.3728, "max": 5.3824, "min": 0.4799}, "tpot_s": {"mean": 0.1035, "p50": 0.1018, "p95": 0.1191, "max": 0.1346, "min": 0.0962}, "e2e_s": {"mean": 17.6355, "p50": 17.7518, "p95": 18.0249, "max": 18.1344, "min": 17.3667}, "per_req_out_tok_s_e2e": {"mean": 7.2594, "p50": 7.2851, "p95": 7.3661, "max": 7.3704, "min": 7.0584}, "per_req_decode_tok_s": {"mean": 9.7882, "p50": 9.8997, "p95": 10.4804, "max": 10.4815, "min": 7.4906}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9664, "corpus_window": {"start": 4500000, "end": 4631072}, "ok": 128, "failed": 0, "wall_s": 38.17, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 16384, "output_throughput_tok_s": 429.24, "input_throughput_tok_s": 3433.95, "ttft_s": {"mean": 5.5975, "p50": 5.8971, "p95": 7.303, "max": 7.3229, "min": 0.3871}, "tpot_s": {"mean": 0.105, "p50": 0.1022, "p95": 0.1191, "max": 0.1498, "min": 0.0899}, "e2e_s": {"mean": 18.9379, "p50": 19.1349, "p95": 19.3038, "max": 19.5071, "min": 18.5527}, "per_req_out_tok_s_e2e": {"mean": 6.7607, "p50": 6.8239, "p95": 6.8916, "max": 6.8993, "min": 6.5617}, "per_req_decode_tok_s": {"mean": 9.6961, "p50": 9.8648, "p95": 10.9748, "max": 11.2171, "min": 6.7276}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 44, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9665, "corpus_window": {"start": 4700000, "end": 4765536}, "ok": 64, "failed": 0, "wall_s": 769.61, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 262144, "output_throughput_tok_s": 340.62, "input_throughput_tok_s": 85.15, "ttft_s": {"mean": 2.8816, "p50": 2.3118, "p95": 4.8777, "max": 4.8873, "min": 0.3856}, "tpot_s": {"mean": 0.0925, "p50": 0.0992, "p95": 0.1004, "max": 0.1005, "min": 0.0837}, "e2e_s": {"mean": 381.5542, "p50": 411.1625, "p95": 414.2572, "max": 415.1497, "min": 344.0864}, "per_req_out_tok_s_e2e": {"mean": 10.8071, "p50": 11.4442, "p95": 11.809, "max": 11.904, "min": 9.8663}, "per_req_decode_tok_s": {"mean": 10.8859, "p50": 11.4629, "p95": 11.8826, "max": 11.9505, "min": 9.948}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 60, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9666, "corpus_window": {"start": 4700000, "end": 4831072}, "ok": 128, "failed": 0, "wall_s": 917.54, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 524288, "output_throughput_tok_s": 571.4, "input_throughput_tok_s": 142.85, "ttft_s": {"mean": 3.3071, "p50": 1.6982, "p95": 7.249, "max": 7.27, "min": 0.3884}, "tpot_s": {"mean": 0.1102, "p50": 0.1104, "p95": 0.1133, "max": 0.1185, "min": 0.1}, "e2e_s": {"mean": 454.4814, "p50": 455.407, "p95": 465.1982, "max": 486.5096, "min": 415.9375}, "per_req_out_tok_s_e2e": {"mean": 9.0166, "p50": 8.9946, "p95": 9.3778, "max": 9.8476, "min": 8.4192}, "per_req_decode_tok_s": {"mean": 9.084, "p50": 9.0613, "p95": 9.467, "max": 9.9975, "min": 8.439}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 228, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/all_results_v2.jsonl

@@ -1,6 +0,0 @@
-{"tag": "b41_1k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9671, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 35.92, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 228.08, "input_throughput_tok_s": 1824.68, "ttft_s": {"mean": 4.6017, "p50": 4.71, "p95": 5.4921, "max": 5.5164, "min": 0.4391}, "tpot_s": {"mean": 0.1041, "p50": 0.1035, "p95": 0.1207, "max": 0.1375, "min": 0.0971}, "e2e_s": {"mean": 17.8172, "p50": 17.8119, "p95": 18.0286, "max": 18.187, "min": 17.6927}, "per_req_out_tok_s_e2e": {"mean": 7.1844, "p50": 7.1868, "p95": 7.2269, "max": 7.2346, "min": 7.038}, "per_req_decode_tok_s": {"mean": 9.737, "p50": 9.7435, "p95": 10.3684, "max": 10.3817, "min": 7.3313}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9672, "corpus_window": {"start": 4500000, "end": 4631072}, "ok": 128, "failed": 0, "wall_s": 37.82, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 16384, "output_throughput_tok_s": 433.2, "input_throughput_tok_s": 3465.61, "ttft_s": {"mean": 5.4604, "p50": 5.8986, "p95": 7.1577, "max": 7.2407, "min": 0.3928}, "tpot_s": {"mean": 0.1043, "p50": 0.0999, "p95": 0.1167, "max": 0.1473, "min": 0.0898}, "e2e_s": {"mean": 18.7041, "p50": 18.7087, "p95": 19.063, "max": 19.1341, "min": 18.5506}, "per_req_out_tok_s_e2e": {"mean": 6.8437, "p50": 6.8424, "p95": 6.8924, "max": 6.9, "min": 6.6896}, "per_req_decode_tok_s": {"mean": 9.7799, "p50": 10.0872, "p95": 11.0923, "max": 11.2174, "min": 6.8403}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 44, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9673, "corpus_window": {"start": 4700000, "end": 4765536}, "ok": 64, "failed": 0, "wall_s": 756.13, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 262144, "output_throughput_tok_s": 346.69, "input_throughput_tok_s": 86.67, "ttft_s": {"mean": 2.8487, "p50": 2.2062, "p95": 4.9526, "max": 4.9666, "min": 0.3857}, "tpot_s": {"mean": 0.091, "p50": 0.1, "p95": 0.1012, "max": 0.1014, "min": 0.0798}, "e2e_s": {"mean": 375.3459, "p50": 414.2945, "p95": 416.7195, "max": 417.0268, "min": 328.3758}, "per_req_out_tok_s_e2e": {"mean": 11.0403, "p50": 12.0491, "p95": 12.4404, "max": 12.4735, "min": 9.8219}, "per_req_decode_tok_s": {"mean": 11.1193, "p50": 12.0715, "p95": 12.4793, "max": 12.5369, "min": 9.8601}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 68, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9674, "corpus_window": {"start": 4700000, "end": 4831072}, "ok": 128, "failed": 0, "wall_s": 916.56, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 524288, "output_throughput_tok_s": 572.01, "input_throughput_tok_s": 143.0, "ttft_s": {"mean": 3.4732, "p50": 2.1089, "p95": 7.2969, "max": 7.3202, "min": 0.387}, "tpot_s": {"mean": 0.1097, "p50": 0.1102, "p95": 0.1125, "max": 0.1154, "min": 0.1013}, "e2e_s": {"mean": 452.9018, "p50": 454.1397, "p95": 465.3856, "max": 473.7285, "min": 416.137}, "per_req_out_tok_s_e2e": {"mean": 9.0488, "p50": 9.0208, "p95": 9.4307, "max": 9.8429, "min": 8.6463}, "per_req_decode_tok_s": {"mean": 9.1186, "p50": 9.0811, "p95": 9.5525, "max": 9.8755, "min": 8.6658}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 196, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c32", "arm": "pp4mrr64", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9675, "corpus_window": {"start": 2300000, "end": 3348576}, "ok": 64, "failed": 0, "wall_s": 170.33, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 32768, "output_throughput_tok_s": 192.38, "input_throughput_tok_s": 6156.1, "ttft_s": {"mean": 26.475, "p50": 27.0154, "p95": 46.313, "max": 47.9669, "min": 5.2614}, "tpot_s": {"mean": 0.1145, "p50": 0.1156, "p95": 0.1532, "max": 0.1592, "min": 0.0695}, "e2e_s": {"mean": 84.9718, "p50": 86.193, "p95": 86.7916, "max": 86.8312, "min": 83.4017}, "per_req_out_tok_s_e2e": {"mean": 6.0273, "p50": 6.1226, "p95": 6.1376, "max": 6.139, "min": 5.8965}, "per_req_decode_tok_s": {"mean": 9.2192, "p50": 8.8289, "p95": 13.2592, "max": 14.4106, "min": 6.2935}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c64", "arm": "pp4mrr64", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9676, "corpus_window": {"start": 2300000, "end": 4397152}, "ok": 128, "failed": 0, "wall_s": 326.73, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 65536, "output_throughput_tok_s": 200.58, "input_throughput_tok_s": 6418.52, "ttft_s": {"mean": 50.7606, "p50": 49.8362, "p95": 89.2249, "max": 147.4077, "min": 5.3842}, "tpot_s": {"mean": 0.185, "p50": 0.1854, "p95": 0.2615, "max": 0.3131, "min": 0.0533}, "e2e_s": {"mean": 145.3021, "p50": 145.6727, "p95": 148.6968, "max": 286.8298, "min": 39.2402}, "per_req_out_tok_s_e2e": {"mean": 3.6798, "p50": 3.6103, "p95": 3.6332, "max": 13.0478, "min": 1.785}, "per_req_decode_tok_s": {"mean": 6.0154, "p50": 5.4114, "p95": 9.3655, "max": 18.8041, "min": 3.1997}, "spec_accept_length_mean": null, "retractions_total": 3, "cache_hit_from_logs": {"prefill_batches": 532, "new_tokens": 8589312, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/retest_compare_pp4_v2.md

@@ -1,19 +0,0 @@
-| 场景 | 并发 | 点型 | Output TPS 旧→新 | Δ | TPOT P95 旧→新 | TTFT P95 旧→新 | E7b64 同点 out 参考 |
-|---|---|---|---|---|---|---|---|
-| 主场景 16K→512 | 32 | 锚点 | 193 → **192** | -0% | 152.0 ms → 153.2 ms | 46.43 s → 46.31 s | 97.6 |
-| 主场景 16K→512 | 64 | 目标 | 211 → **201** | -5% | 203.7 ms → 261.5 ms | 134.78 s → 89.22 s | 99.6 |
-| 4.1 短输入 1K→128 | 32 | 锚点 | 276 → **228** | -17% | 98.5 ms → 120.7 ms | 4.87 s → 5.49 s | 229 |
-| 4.1 短输入 1K→128 | 64 | 目标 | 341 → **433** | +27% | 101.8 ms → 116.7 ms | 19.96 s → 7.16 s | 272 |
-| 4.2 长输出 1K→4K | 32 | 锚点 | 367 → **347** | -6% | 90.5 ms → 101.2 ms | 4.14 s → 4.95 s | 676 |
-| 4.2 长输出 1K→4K | 64 | 目标 | 482 → **572** | +19% | 95.5 ms → 112.5 ms | 337.92 s → 7.30 s | 826 |
-
-## 附录：PP4MRR64 复测全量指标
-
-| 场景点 | ok/nreq | wall(s) | out tok/s | in tok/s | TTFT mean/p95/max(s) | TPOT mean/p95/max(ms) | retractions | hit |
-|---|---|---|---|---|---|---|---|---|
-| b3_16k_c32 | 64/64 | 170.33 | 192.38 | 6156.1 | 26.48/46.31/47.97 | 114.5/153.2/159.2 | 0 | 0.0 |
-| b3_16k_c64 | 128/128 | 326.73 | 200.58 | 6418.52 | 50.76/89.22/147.41 | 185.0/261.5/313.1 | 3 | 0.0 |
-| b41_1k_c32 | 64/64 | 35.92 | 228.08 | 1824.68 | 4.60/5.49/5.52 | 104.1/120.7/137.5 | 0 | 0.0 |
-| b41_1k_c64 | 128/128 | 37.82 | 433.2 | 3465.61 | 5.46/7.16/7.24 | 104.3/116.7/147.3 | 0 | 0.0 |
-| b42_1k4k_c32 | 64/64 | 756.13 | 346.69 | 86.67 | 2.85/4.95/4.97 | 91.0/101.2/101.4 | 0 | 0.0 |
-| b42_1k4k_c64 | 128/128 | 916.56 | 572.01 | 143.0 | 3.47/7.30/7.32 | 109.7/112.5/115.4 | 0 | 0.0 |

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/status_pass1.txt

@@ -1,6 +0,0 @@
-b3_16k_c32 OK
-b3_16k_c64 OK
-b41_1k_c32 OK
-b41_1k_c64 OK
-b42_1k4k_c32 OK
-b42_1k4k_c64 OK

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/pp4mrr64/status_v2.txt

@@ -1,6 +0,0 @@
-b41_1k_c32 OK
-b41_1k_c64 OK
-b42_1k4k_c32 OK
-b42_1k4k_c64 OK
-b3_16k_c32 OK
-b3_16k_c64 OK

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/provenance.md

@@ -1,66 +0,0 @@
-# 数据来源与核验记录（provenance）
-
-原始 csv/log 按仓库惯例不入库（.gitignore `*.csv`/`*.log`），完整文件在 60.8
-`/root/bench_logs/b300eq_{tp2pp4_20260910_1138,e7b_20260910_1428,e7b64_20260910_1732}/` 与本地镜像
-`D:/sskj/b300eq/`。本文件固化其中的关键事实。
-
-## GPU 清单（nvidia-smi，8×RTX 6000D，总 85,651 MiB/卡）
-
-### TP2PP4 臂（mem0.85，加载后空载 → 矩阵结束）
-
-| GPU | 空载 MiB | 结束 MiB |
-|---|---|---|
-| 0/1 | 64,613 | 79,391 |
-| 2/3 | 70,867 | 81,751 |
-| 4/5 | 74,499 | 84,439 / 84,631 |
-| 6/7 | 75,361 | 84,491 |
-
-vram_timeline.csv（30s 采样）全程峰值 **85,013 MiB**（主场景 C=64，最紧张卡余量 ~638 MiB）。
-
-### E7b 臂（mem0.90 + EAGLE 草稿权重，空载更高）
-
-| GPU | 空载 MiB | 结束 MiB |
-|---|---|---|
-| 0 | 77,861 | 83,477 |
-| 1/2/5/6 | 77,955 | 83,551 / 83,553 |
-| 3/4/7 | 77,859 | 83,477 / 83,479 |
-
-全程峰值 **83,553 MiB**（余量 ~2.1 GiB）。
-
-### E7b64 复测臂（MRR64 + decode 图桶 1–64，20260910_1732）
-
-- 部署：`deploy_glm53_e7b_hicc.sh`（md5 165db732…），与 E7b 初测唯一差异 = MRR 16→64 + 图参数 `--cuda-graph-max-bs-decode 64 --cuda-graph-bs-decode "1 2 3 4 6 8 12 16 24 32 48 64"`；KV 池 276,480 不变，server_args 核验 `max_running_requests=64`、图桶 13 档
-- 启动计时（09:30:27）：load_weight=217.07 s；cuda_graph={prefill=94.48, target_verify=35.95, draft_decode=13.40, draft_extend=1.85}；捕获后 avail_gpu_mem=6.11 GB
-- 空载 79,317~79,411 MiB/卡；复测矩阵（16K/1K 场景，vram_timeline 30s 采样 116 帧）全程峰值 **83,627 MiB**（余量 ~3.9 GiB）
-- 10 点全部 OK、命中核验全 0.0（全新容器实例=回收窗口重新处女文本）、0 retraction；run-id 9601-9610
-- C=8 锚点 vs 初测偏差：16K 81.4→83.9（+3%）、1K 152→151（−1%）、1K→4K 412→402（−2%）→ 两轮环境无漂移
-- 前后对比表：`retest_compare.md`（本目录镜像 = 60.8 `/root/bench_logs/retest_compare.md`）
-
-## 质量门判决
-
-- TP2PP4 臂：`PASS=6 FAIL=1`（唯一失败 = tool-call，D 口径无 parser，历史已知；GSM8K×5 + 中文推理全过）
-- E7b 臂：`PASS=7 FAIL=0`（含 tool-call `get_weather{"city": "北京"}`）
-- E7b64 复测臂：`PASS=7 FAIL=0`（部署后以 "The server is fired up" 真就绪信号判定后跑门，7/7）
-
-## 在役容器保全与恢复（60.8，TP4PP2-nomtp@0.90 口径）
-
-- 停役流程：`docker stop glm53-nvfp4` → `docker rename glm53-nvfp4 glm53-nvfp4-insvc`（先改名，防 E7b 部署脚本 rm -f 同名容器）；inspect/启动命令/挂载/镜像归档于 60.8 `/root/bench_logs/b300eq_meta/`
-- 镜像：`lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729`（sha256:eb090e39…）
-- 停役前显存：82,221~82,395 MiB/卡
-- 恢复流程：E7b 测试容器 rm（显存排干 0 MiB）→ `docker rename glm53-nvfp4-insvc glm53-nvfp4 && docker start`
-- 恢复核验第一轮（09-10 午，初测后）：health 200（启动后 ~4 min）；16K/16tok 冷抽测 ok=1/1、wall 3.59 s；显存 GPU4-7 与停役前持平（82,2xx MiB）、GPU0-3 低 ~5 GiB（重启后 radix 池未回填，正常）；容器口径未变
-- 恢复核验第二轮（09-10 晚，e7b64 复测拆台后，`restore_insvc_e7b64.sh` 自动化）：fired up（start 后 ~4 min）→ health 200 → 16K/32tok 抽测 ok + C=4×16K/256tok 抽测 ok；KV 池 647,040 tokens（8 rank 一致）+ server_args 与归档启动命令逐字一致（TP4PP2/mem0.90/MRR16/cps8192/hicache×3/ctx 1,048,576）；显存 GPU4-7 82,2xx MiB 持平、GPU0-3 77.2 GiB = mem0.90 静态预算水位（较停役前热稳态多 ~5 GiB 余量，与第一轮同象）
-- 僵尸 PID 现象记录：`docker stop`/`rm` 偶发 "container PID xxx is zombie and can not be killed"，实为收尾边界现象（容器终态 exited 137、显存归零），等待 ~20s 重试即成功
-
-## 执行资产 md5（60.8 = 本目录 = 60.7 原件，三方一致）
-
-见 `md5_ledger.txt`。corpus 语料：`/root/corpus_ids.json`（21,296,780 tokens，消费至 21,235,008，回收窗口协议见 REPORT.md 附录 A）。
-
-### PP4MRR64 复测臂（MRR 48→64；pass-1 20260910_1927 + v2 20260910_2021 + MRR48 对照 20260910_2112）
-
-- 部署：`deploy_glm53_pp4_mrr64.sh`（md5 533440ef…），与 `deploy_glm53_pp4.sh`（def3c64c）逐 token diff 仅 `MRR=48→64` 一处；KV 池 1,040,384 / mem0.85 / cps16384 / radix 关全部不变；decode 图保持栈默认（max_bs 256、桶含 56/64）——本臂从未掉图，MRR 才是初测 C=64 点的活跃上限（48 活跃+16 排队）
-- pass-1（19:27–20:14，run-id 9661–9666）：6 点全 OK、命中核验全 0.0；16K c64 出现 3 次 retraction（64×17.4K≈1.11M tokens > 池 1,040,384，活跃封顶 ~59 条）
-- v2（20:21–21:07，run-id 9671–9676，**正文采用值**）：全新实例 + 易回退点 16K c64 排末位，用于排除"retraction 后遗污染后续点"假设；6 点与 pass-1 全部对齐 ≤1.8%（4.1 c32 228.08 vs 230.2、4.1 c64 433.2 vs 429.2、4.2 c32 346.69 vs 340.6、4.2 c64 572.01 vs 571.4、16K c32 192.38 vs 192.0、16K c64 200.58 vs 200.6）→ 污染假设否证，1K 锚点回退为 MRR64 部署的可复现行为；16K c64 再次 3 次 retraction（确定性行为）
-- v2 矩阵 VRAM 峰值 84,791 MiB（GPU2，16K c64 池顶运行时采样）；QG 复验 PASS=6 FAIL=1（tool-call 已知项）
-- MRR48 归因对照（21:12–21:14，run-id 9681）：原版 `deploy_glm53_pp4.sh` 全新实例跑 4.1 c32 单点 = **266.42 tok/s**（TPOT p50 87.8ms，复现初测 87ms 水平）→ −17% 锚点分解：275.6（初测，第 9 点热实例）→ 266.4（新鲜度 −3.3%）→ 228.1（MRR64 配置代价 −14.4%）；逐请求 TPOT p50 87→102ms 均匀抬高、三实例可复现、机制未定位
-- 恢复核验第三轮（21:13–21:17，`control_mrr48_41c32.sh` 自动链）：rm glm53-pp4 → VRAM 排干 → rename 回 + start → fired up → health 200 → 16K/32tok 抽测 ok；KV 池 647,040（8 rank 一致）；显存 GPU0-3 77.2 GiB / GPU4-7 82.3 GiB 与停役前一致

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/tp2pp4/all_results.jsonl

@@ -1,24 +0,0 @@
-{"tag": "b3_16k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9501, "corpus_window": {"start": 2300000, "end": 2431072}, "ok": 8, "failed": 0, "wall_s": 245.55, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 16.68, "input_throughput_tok_s": 533.8, "ttft_s": {"mean": 5.3493, "p50": 5.3474, "p95": 5.3628, "max": 5.3628, "min": 5.3438}, "tpot_s": {"mean": 0.0496, "p50": 0.0495, "p95": 0.0503, "max": 0.0503, "min": 0.0493}, "e2e_s": {"mean": 30.6931, "p50": 30.657, "p95": 31.0364, "max": 31.0364, "min": 30.5593}, "per_req_out_tok_s_e2e": {"mean": 16.6817, "p50": 16.7233, "p95": 16.7543, "max": 16.7543, "min": 16.4968}, "per_req_decode_tok_s": {"mean": 20.2031, "p50": 20.2689, "p95": 20.3081, "max": 20.3081, "min": 19.9299}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c8", "arm": "tp2pp4", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9502, "corpus_window": {"start": 2300000, "end": 2562144}, "ok": 16, "failed": 0, "wall_s": 81.0, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 8192, "output_throughput_tok_s": 101.13, "input_throughput_tok_s": 3236.22, "ttft_s": {"mean": 10.0539, "p50": 10.6041, "p95": 14.7948, "max": 14.7948, "min": 5.2557}, "tpot_s": {"mean": 0.0595, "p50": 0.0602, "p95": 0.0709, "max": 0.0709, "min": 0.0481}, "e2e_s": {"mean": 40.4682, "p50": 41.4728, "p95": 41.6589, "max": 41.6589, "min": 39.2731}, "per_req_out_tok_s_e2e": {"mean": 12.662, "p50": 13.0063, "p95": 13.0369, "max": 13.0369, "min": 12.2903}, "per_req_decode_tok_s": {"mean": 17.0379, "p50": 17.0019, "p95": 20.8395, "max": 20.8395, "min": 14.1226}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c16", "arm": "tp2pp4", "summary": {"concurrency": 16, "num_requests": 32, "run_id": 9503, "corpus_window": {"start": 2300000, "end": 2824288}, "ok": 32, "failed": 0, "wall_s": 112.16, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 16384, "output_throughput_tok_s": 146.08, "input_throughput_tok_s": 4674.5, "ttft_s": {"mean": 15.5283, "p50": 16.1378, "p95": 25.7815, "max": 25.8389, "min": 5.267}, "tpot_s": {"mean": 0.0792, "p50": 0.0797, "p95": 0.0985, "max": 0.1012, "min": 0.0571}, "e2e_s": {"mean": 55.9934, "p50": 56.8023, "p95": 57.0853, "max": 57.1347, "min": 54.9678}, "per_req_out_tok_s_e2e": {"mean": 9.1467, "p50": 9.2952, "p95": 9.3139, "max": 9.3145, "min": 8.9613}, "per_req_decode_tok_s": {"mean": 12.9884, "p50": 12.7194, "p95": 16.7725, "max": 17.5544, "min": 9.9045}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c32", "arm": "tp2pp4", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9504, "corpus_window": {"start": 2300000, "end": 3348576}, "ok": 64, "failed": 0, "wall_s": 169.54, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 32768, "output_throughput_tok_s": 193.27, "input_throughput_tok_s": 6184.73, "ttft_s": {"mean": 26.5164, "p50": 27.1445, "p95": 46.4256, "max": 47.8906, "min": 5.2618}, "tpot_s": {"mean": 0.1137, "p50": 0.114, "p95": 0.152, "max": 0.1573, "min": 0.07}, "e2e_s": {"mean": 84.6344, "p50": 85.322, "p95": 85.7436, "max": 85.8459, "min": 83.6335}, "per_req_out_tok_s_e2e": {"mean": 6.0502, "p50": 6.108, "p95": 6.1205, "max": 6.1219, "min": 5.9642}, "per_req_decode_tok_s": {"mean": 9.2806, "p50": 8.8387, "p95": 13.3026, "max": 14.3169, "min": 6.3708}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b3_16k_c64", "arm": "tp2pp4", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9505, "corpus_window": {"start": 2300000, "end": 4397152}, "ok": 128, "failed": 0, "wall_s": 310.79, "input_len": 16384, "shared_len": 0, "unique_len": 16384, "output_len": 512, "output_tokens_total": 65536, "output_throughput_tok_s": 210.87, "input_throughput_tok_s": 6747.91, "ttft_s": {"mean": 63.0522, "p50": 49.1362, "p95": 134.7794, "max": 139.9931, "min": 5.3652}, "tpot_s": {"mean": 0.1392, "p50": 0.1358, "p95": 0.2037, "max": 0.2143, "min": 0.0707}, "e2e_s": {"mean": 134.1654, "p50": 114.4018, "p95": 226.099, "max": 226.1544, "min": 84.0644}, "per_req_out_tok_s_e2e": {"mean": 4.1916, "p50": 4.476, "p95": 6.0862, "max": 6.0906, "min": 2.2639}, "per_req_decode_tok_s": {"mean": 7.7966, "p50": 7.3815, "p95": 11.9051, "max": 14.1753, "min": 4.6761}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 512, "new_tokens": 8388608, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9511, "corpus_window": {"start": 4500000, "end": 4508192}, "ok": 8, "failed": 0, "wall_s": 52.95, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 1024, "output_throughput_tok_s": 19.34, "input_throughput_tok_s": 154.7, "ttft_s": {"mean": 0.3848, "p50": 0.3852, "p95": 0.3865, "max": 0.3865, "min": 0.3825}, "tpot_s": {"mean": 0.0491, "p50": 0.0491, "p95": 0.0493, "max": 0.0493, "min": 0.0488}, "e2e_s": {"mean": 6.619, "p50": 6.62, "p95": 6.6447, "max": 6.6447, "min": 6.5863}, "per_req_out_tok_s_e2e": {"mean": 19.3385, "p50": 19.3379, "p95": 19.4343, "max": 19.4343, "min": 19.2635}, "per_req_decode_tok_s": {"mean": 20.5327, "p50": 20.5305, "p95": 20.6346, "max": 20.6346, "min": 20.4442}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 32768, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c8", "arm": "tp2pp4", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9512, "corpus_window": {"start": 4500000, "end": 4516384}, "ok": 16, "failed": 0, "wall_s": 20.1, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 2048, "output_throughput_tok_s": 101.91, "input_throughput_tok_s": 815.3, "ttft_s": {"mean": 1.3131, "p50": 1.2998, "p95": 1.6901, "max": 1.6901, "min": 0.6481}, "tpot_s": {"mean": 0.0686, "p50": 0.0684, "p95": 0.0729, "max": 0.0729, "min": 0.0669}, "e2e_s": {"mean": 10.024, "p50": 10.1096, "p95": 10.2887, "max": 10.2887, "min": 9.7907}, "per_req_out_tok_s_e2e": {"mean": 12.7741, "p50": 12.9718, "p95": 13.0736, "max": 13.0736, "min": 12.4409}, "per_req_decode_tok_s": {"mean": 14.7032, "p50": 14.7329, "p95": 15.0683, "max": 15.0683, "min": 13.8323}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 24, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c32", "arm": "tp2pp4", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9513, "corpus_window": {"start": 4500000, "end": 4565536}, "ok": 64, "failed": 0, "wall_s": 29.73, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 8192, "output_throughput_tok_s": 275.55, "input_throughput_tok_s": 2204.4, "ttft_s": {"mean": 3.8882, "p50": 3.9615, "p95": 4.8674, "max": 4.8759, "min": 1.5565}, "tpot_s": {"mean": 0.0856, "p50": 0.0869, "p95": 0.0985, "max": 0.1068, "min": 0.0785}, "e2e_s": {"mean": 14.758, "p50": 14.8384, "p95": 15.0559, "max": 15.1157, "min": 14.5917}, "per_req_out_tok_s_e2e": {"mean": 8.6742, "p50": 8.7413, "p95": 8.7701, "max": 8.7721, "min": 8.468}, "per_req_decode_tok_s": {"mean": 11.8377, "p50": 11.6015, "p95": 12.822, "max": 12.8317, "min": 9.4403}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 36, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b41_1k_c64", "arm": "tp2pp4", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9514, "corpus_window": {"start": 4500000, "end": 4631072}, "ok": 128, "failed": 0, "wall_s": 48.11, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 128, "output_tokens_total": 16384, "output_throughput_tok_s": 340.53, "input_throughput_tok_s": 2724.25, "ttft_s": {"mean": 8.1393, "p50": 4.8485, "p95": 19.9553, "max": 20.0096, "min": 0.3836}, "tpot_s": {"mean": 0.0933, "p50": 0.0924, "p95": 0.1018, "max": 0.1251, "min": 0.0818}, "e2e_s": {"mean": 19.9938, "p50": 16.0958, "p95": 32.0105, "max": 32.0464, "min": 15.7372}, "per_req_out_tok_s_e2e": {"mean": 6.9903, "p50": 7.9525, "p95": 8.1268, "max": 8.1336, "min": 3.9942}, "per_req_decode_tok_s": {"mean": 10.8849, "p50": 10.9086, "p95": 12.2874, "max": 12.3245, "min": 8.0552}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 52, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9521, "corpus_window": {"start": 4700000, "end": 4708192}, "ok": 8, "failed": 0, "wall_s": 1614.16, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 32768, "output_throughput_tok_s": 20.3, "input_throughput_tok_s": 5.08, "ttft_s": {"mean": 0.3926, "p50": 0.394, "p95": 0.3952, "max": 0.3952, "min": 0.3852}, "tpot_s": {"mean": 0.0492, "p50": 0.0491, "p95": 0.0498, "max": 0.0498, "min": 0.049}, "e2e_s": {"mean": 201.7695, "p50": 201.6506, "p95": 204.4013, "max": 204.4013, "min": 200.9309}, "per_req_out_tok_s_e2e": {"mean": 20.3009, "p50": 20.3439, "p95": 20.3851, "max": 20.3851, "min": 20.039}, "per_req_decode_tok_s": {"mean": 20.3407, "p50": 20.3837, "p95": 20.4254, "max": 20.4254, "min": 20.078}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 32, "new_tokens": 32768, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c8", "arm": "tp2pp4", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9522, "corpus_window": {"start": 4700000, "end": 4716384}, "ok": 16, "failed": 0, "wall_s": 649.69, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 65536, "output_throughput_tok_s": 100.87, "input_throughput_tok_s": 25.22, "ttft_s": {"mean": 1.352, "p50": 1.4385, "p95": 1.8726, "max": 1.8726, "min": 0.4364}, "tpot_s": {"mean": 0.079, "p50": 0.0791, "p95": 0.0794, "max": 0.0794, "min": 0.0788}, "e2e_s": {"mean": 324.8285, "p50": 325.5653, "p95": 325.6331, "max": 325.6331, "min": 324.0447}, "per_req_out_tok_s_e2e": {"mean": 12.6098, "p50": 12.6389, "p95": 12.6402, "max": 12.6402, "min": 12.5786}, "per_req_decode_tok_s": {"mean": 12.6626, "p50": 12.6734, "p95": 12.7003, "max": 12.7003, "min": 12.5955}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 20, "new_tokens": 65536, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c32", "arm": "tp2pp4", "summary": {"concurrency": 32, "num_requests": 64, "run_id": 9523, "corpus_window": {"start": 4700000, "end": 4765536}, "ok": 64, "failed": 0, "wall_s": 714.14, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 262144, "output_throughput_tok_s": 367.08, "input_throughput_tok_s": 91.77, "ttft_s": {"mean": 1.9772, "p50": 1.1755, "p95": 4.1378, "max": 4.1467, "min": 0.3867}, "tpot_s": {"mean": 0.0861, "p50": 0.0887, "p95": 0.0905, "max": 0.0911, "min": 0.0805}, "e2e_s": {"mean": 354.393, "p50": 367.2171, "p95": 373.3199, "max": 374.0319, "min": 330.2086}, "per_req_out_tok_s_e2e": {"mean": 11.5833, "p50": 11.8878, "p95": 12.2953, "max": 12.4043, "min": 10.9509}, "per_req_decode_tok_s": {"mean": 11.6457, "p50": 11.9132, "p95": 12.3308, "max": 12.4283, "min": 10.9854}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 262144, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b42_1k4k_c64", "arm": "tp2pp4", "summary": {"concurrency": 64, "num_requests": 128, "run_id": 9524, "corpus_window": {"start": 4700000, "end": 4831072}, "ok": 128, "failed": 0, "wall_s": 1088.63, "input_len": 1024, "shared_len": 0, "unique_len": 1024, "output_len": 4096, "output_tokens_total": 524288, "output_throughput_tok_s": 481.6, "input_throughput_tok_s": 120.4, "ttft_s": {"mean": 80.7415, "p50": 4.7569, "p95": 337.9195, "max": 338.4726, "min": 0.3985}, "tpot_s": {"mean": 0.0912, "p50": 0.0914, "p95": 0.0955, "max": 0.0963, "min": 0.0852}, "e2e_s": {"mean": 454.4024, "p50": 385.4538, "p95": 718.7004, "max": 732.2026, "min": 351.072}, "per_req_out_tok_s_e2e": {"mean": 9.6298, "p50": 10.6531, "p95": 11.4339, "max": 11.6671, "min": 5.5941}, "per_req_decode_tok_s": {"mean": 10.9736, "p50": 10.9486, "p95": 11.4764, "max": 11.7362, "min": 10.3887}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 200, "new_tokens": 524288, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9531, "corpus_window": {"start": 5000000, "end": 5524288}, "ok": 8, "failed": 0, "wall_s": 285.91, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 14.33, "input_throughput_tok_s": 1833.72, "ttft_s": {"mean": 10.2811, "p50": 10.2794, "p95": 10.3216, "max": 10.3216, "min": 10.2634}, "tpot_s": {"mean": 0.0498, "p50": 0.0499, "p95": 0.05, "max": 0.05, "min": 0.0496}, "e2e_s": {"mean": 35.739, "p50": 35.7693, "p95": 35.838, "max": 35.838, "min": 35.6073}, "per_req_out_tok_s_e2e": {"mean": 14.3261, "p50": 14.3345, "p95": 14.3791, "max": 14.3791, "min": 14.2865}, "per_req_decode_tok_s": {"mean": 20.112, "p50": 20.1156, "p95": 20.2096, "max": 20.2096, "min": 20.052}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c4", "arm": "tp2pp4", "summary": {"concurrency": 4, "num_requests": 8, "run_id": 9532, "corpus_window": {"start": 5000000, "end": 5524288}, "ok": 8, "failed": 0, "wall_s": 122.3, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 33.49, "input_throughput_tok_s": 4286.94, "ttft_s": {"mean": 19.5548, "p50": 22.5526, "p95": 28.8262, "max": 28.8262, "min": 10.2823}, "tpot_s": {"mean": 0.0814, "p50": 0.0873, "p95": 0.0995, "max": 0.0995, "min": 0.0632}, "e2e_s": {"mean": 61.1314, "p50": 61.2078, "p95": 61.2548, "max": 61.2548, "min": 61.002}, "per_req_out_tok_s_e2e": {"mean": 8.3754, "p50": 8.3851, "p95": 8.3932, "max": 8.3932, "min": 8.3585}, "per_req_decode_tok_s": {"mean": 12.6679, "p50": 13.2834, "p95": 15.8466, "max": 15.8466, "min": 10.0675}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_64k_c8", "arm": "tp2pp4", "summary": {"concurrency": 8, "num_requests": 16, "run_id": 9533, "corpus_window": {"start": 5000000, "end": 6048576}, "ok": 16, "failed": 0, "wall_s": 185.59, "input_len": 65536, "shared_len": 0, "unique_len": 65536, "output_len": 512, "output_tokens_total": 8192, "output_throughput_tok_s": 44.14, "input_throughput_tok_s": 5650.11, "ttft_s": {"mean": 31.833, "p50": 34.8143, "p95": 53.3841, "max": 53.3841, "min": 10.2802}, "tpot_s": {"mean": 0.1192, "p50": 0.1246, "p95": 0.1618, "max": 0.1618, "min": 0.0765}, "e2e_s": {"mean": 92.7517, "p50": 92.8482, "p95": 92.9902, "max": 92.9902, "min": 92.4957}, "per_req_out_tok_s_e2e": {"mean": 5.5201, "p50": 5.5226, "p95": 5.5354, "max": 5.5354, "min": 5.506}, "per_req_decode_tok_s": {"mean": 8.9039, "p50": 8.8159, "p95": 13.0909, "max": 13.0909, "min": 6.1915}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 8, "run_id": 9541, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 347.5, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 11.79, "input_throughput_tok_s": 3017.45, "ttft_s": {"mean": 18.2074, "p50": 18.2043, "p95": 18.2252, "max": 18.2252, "min": 18.1922}, "tpot_s": {"mean": 0.0494, "p50": 0.0496, "p95": 0.0497, "max": 0.0497, "min": 0.0485}, "e2e_s": {"mean": 43.4375, "p50": 43.5633, "p95": 43.6252, "max": 43.6252, "min": 43.0007}, "per_req_out_tok_s_e2e": {"mean": 11.7874, "p50": 11.7541, "p95": 11.9068, "max": 11.9068, "min": 11.7363}, "per_req_decode_tok_s": {"mean": 20.2951, "p50": 20.1994, "p95": 20.6444, "max": 20.6444, "min": 20.1578}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c2", "arm": "tp2pp4", "summary": {"concurrency": 2, "num_requests": 8, "run_id": 9542, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 243.3, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 16.84, "input_throughput_tok_s": 4309.84, "ttft_s": {"mean": 25.2658, "p50": 32.1952, "p95": 32.4188, "max": 32.4188, "min": 18.173}, "tpot_s": {"mean": 0.0696, "p50": 0.0831, "p95": 0.0837, "max": 0.0837, "min": 0.0556}, "e2e_s": {"mean": 60.8183, "p50": 60.7483, "p95": 61.2374, "max": 61.2374, "min": 60.6134}, "per_req_out_tok_s_e2e": {"mean": 8.4186, "p50": 8.4352, "p95": 8.447, "max": 8.447, "min": 8.3609}, "per_req_decode_tok_s": {"mean": 14.9825, "p50": 17.8153, "p95": 18.0089, "max": 18.0089, "min": 11.965}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b51_128k_c4", "arm": "tp2pp4", "summary": {"concurrency": 4, "num_requests": 8, "run_id": 9543, "corpus_window": {"start": 6200000, "end": 7248576}, "ok": 8, "failed": 0, "wall_s": 186.37, "input_len": 131072, "shared_len": 0, "unique_len": 131072, "output_len": 512, "output_tokens_total": 4096, "output_throughput_tok_s": 21.98, "input_throughput_tok_s": 5626.35, "ttft_s": {"mean": 39.2939, "p50": 46.1927, "p95": 60.3496, "max": 60.3496, "min": 18.2774}, "tpot_s": {"mean": 0.1054, "p50": 0.1187, "p95": 0.1468, "max": 0.1468, "min": 0.0638}, "e2e_s": {"mean": 93.1534, "p50": 93.2237, "p95": 93.3139, "max": 93.3139, "min": 92.97}, "per_req_out_tok_s_e2e": {"mean": 5.4963, "p50": 5.497, "p95": 5.5072, "max": 5.5072, "min": 5.4869}, "per_req_decode_tok_s": {"mean": 10.4415, "p50": 10.8799, "p95": 15.6959, "max": 15.6959, "min": 6.8234}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 256, "new_tokens": 4194304, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_256k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 1, "run_id": 9551, "corpus_window": {"start": 8400000, "end": 8662144}, "ok": 1, "failed": 0, "wall_s": 37.18, "input_len": 262144, "shared_len": 0, "unique_len": 262144, "output_len": 1, "output_tokens_total": 1, "output_throughput_tok_s": 0.03, "input_throughput_tok_s": 7050.23, "ttft_s": {"mean": 37.1811, "p50": 37.1811, "p95": 37.1811, "max": 37.1811, "min": 37.1811}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 37.1814, "p50": 37.1814, "p95": 37.1814, "max": 37.1814, "min": 37.1814}, "per_req_out_tok_s_e2e": {"mean": 0.0269, "p50": 0.0269, "p95": 0.0269, "max": 0.0269, "min": 0.0269}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 64, "new_tokens": 1048576, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_256k_c2", "arm": "tp2pp4", "summary": {"concurrency": 2, "num_requests": 2, "run_id": 9552, "corpus_window": {"start": 8400000, "end": 8924288}, "ok": 2, "failed": 0, "wall_s": 70.19, "input_len": 262144, "shared_len": 0, "unique_len": 262144, "output_len": 1, "output_tokens_total": 2, "output_throughput_tok_s": 0.03, "input_throughput_tok_s": 7469.22, "ttft_s": {"mean": 53.6945, "p50": 70.1229, "p95": 70.1229, "max": 70.1229, "min": 37.2662}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 53.6948, "p50": 70.1232, "p95": 70.1232, "max": 70.1232, "min": 37.2664}, "per_req_out_tok_s_e2e": {"mean": 0.0205, "p50": 0.0268, "p95": 0.0268, "max": 0.0268, "min": 0.0143}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_256k_c3", "arm": "tp2pp4", "summary": {"concurrency": 3, "num_requests": 3, "run_id": 9553, "corpus_window": {"start": 8400000, "end": 9186432}, "ok": 3, "failed": 0, "wall_s": 103.12, "input_len": 262144, "shared_len": 0, "unique_len": 262144, "output_len": 1, "output_tokens_total": 3, "output_throughput_tok_s": 0.03, "input_throughput_tok_s": 7626.22, "ttft_s": {"mean": 70.1194, "p50": 70.1196, "p95": 102.986, "max": 102.986, "min": 37.2526}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 70.1196, "p50": 70.1198, "p95": 102.9863, "max": 102.9863, "min": 37.2529}, "per_req_out_tok_s_e2e": {"mean": 0.0169, "p50": 0.0143, "p95": 0.0268, "max": 0.0268, "min": 0.0097}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 192, "new_tokens": 3145728, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_512k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 1, "run_id": 9554, "corpus_window": {"start": 9300000, "end": 9824288}, "ok": 1, "failed": 0, "wall_s": 92.6, "input_len": 524288, "shared_len": 0, "unique_len": 524288, "output_len": 1, "output_tokens_total": 1, "output_throughput_tok_s": 0.01, "input_throughput_tok_s": 5662.03, "ttft_s": {"mean": 92.5959, "p50": 92.5959, "p95": 92.5959, "max": 92.5959, "min": 92.5959}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 92.5962, "p50": 92.5962, "p95": 92.5962, "max": 92.5962, "min": 92.5962}, "per_req_out_tok_s_e2e": {"mean": 0.0108, "p50": 0.0108, "p95": 0.0108, "max": 0.0108, "min": 0.0108}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 128, "new_tokens": 2097152, "cached_tokens": 0, "hit_rate": 0.0}}}
-{"tag": "b52_896k_c1", "arm": "tp2pp4", "summary": {"concurrency": 1, "num_requests": 1, "run_id": 9555, "corpus_window": {"start": 9900000, "end": 10817504}, "ok": 1, "failed": 0, "wall_s": 220.91, "input_len": 917504, "shared_len": 0, "unique_len": 917504, "output_len": 1, "output_tokens_total": 1, "output_throughput_tok_s": 0.0, "input_throughput_tok_s": 4153.24, "ttft_s": {"mean": 220.9117, "p50": 220.9117, "p95": 220.9117, "max": 220.9117, "min": 220.9117}, "tpot_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "e2e_s": {"mean": 220.912, "p50": 220.912, "p95": 220.912, "max": 220.912, "min": 220.912}, "per_req_out_tok_s_e2e": {"mean": 0.0045, "p50": 0.0045, "p95": 0.0045, "max": 0.0045, "min": 0.0045}, "per_req_decode_tok_s": {"mean": null, "p50": null, "p95": null, "max": null, "min": null}, "spec_accept_length_mean": null, "retractions_total": 0, "cache_hit_from_logs": {"prefill_batches": 224, "new_tokens": 3670016, "cached_tokens": 0, "hit_rate": 0.0}}}

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/tp2pp4/md5_assets.txt

@@ -1,3 +0,0 @@
-a8f3909211adb92501cf007ab71d3b29  /root/corpus_ids.json
-1e34dd8d813ccb7ac8243e8d1573a49e  /root/bench_corpus_v2.py
-4c126d067d33b5ea27c268f37561634c  /root/extract_summary.py

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/results/tp2pp4/status.txt

@@ -1,24 +0,0 @@
-b3_16k_c1 OK
-b3_16k_c8 OK
-b3_16k_c16 OK
-b3_16k_c32 OK
-b3_16k_c64 OK
-b41_1k_c1 OK
-b41_1k_c8 OK
-b41_1k_c32 OK
-b41_1k_c64 OK
-b42_1k4k_c1 OK
-b42_1k4k_c8 OK
-b42_1k4k_c32 OK
-b42_1k4k_c64 OK
-b51_64k_c1 OK
-b51_64k_c4 OK
-b51_64k_c8 OK
-b51_128k_c1 OK
-b51_128k_c2 OK
-b51_128k_c4 OK
-b52_256k_c1 OK
-b52_256k_c2 OK
-b52_256k_c3 OK
-b52_512k_c1 OK
-b52_896k_c1 OK

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/scripts/bench_corpus_v2.py

@@ -1,319 +0,0 @@
-#!/usr/bin/env python3
-"""Real-corpus (PG19) benchmark for sglang GLM-5.3-NVFP4.
-
-Same methodology as bench_hit90.py (input_ids direct to /generate, temp 0,
-ignore_eos, stream, server-side completion_tokens counting, hit-rate verified
-from scheduler logs), but prompts are token slices of REAL book text tokenized
-with the served model's own tokenizer, replacing random ids.
-
-Corpus file: JSON {"ids": [flat token ids], "books": [[start, end), ...]}
-
-Fixed token-offset layout into the flat id array:
-  [0, 117968)      shared prefix for 128k points (90% of 131072)
-  [0, 58976)       shared prefix for 64k points (same region, shorter cut)
-  pool A [131072, +4*8*13104)   128k unique suffixes, run-ids 9301-9304
-  pool B [550400, +4*8*6560)    64k unique suffixes, run-ids 9305-9308
-  pool C [760320, 262144+524288+524288)  16k fully-unique prompts, run-ids 9311-9313
-  spare [2071040, end)          warmup slices / re-run margin
-
-Each run-id maps to one non-overlapping window (one window = one bench point);
-re-running a point with fresh text = bump --pool-override past the spare base.
-
-v2 changes (b300-equivalent campaign, 2026-09-10):
-  - stats(): p95 added (nearest-rank) to match the B300 report metric contract
-    (TTFT P95 / TPOT P95).
-  - --dump-records PATH: per-request records (ttft/e2e/tpot/n_out/retractions/
-    spec_accept_len) written as JSONL for post-hoc percentile checks.
-  - With --shared-frac 0 + --pool-override, any input length is supported
-    (1024 / 16384 / 65536 / 131072 / 262144 / 524288 / 917504).
-
-Usage:
-  python3 bench_corpus_v2.py --corpus /root/corpus_ids.json --input-len 16384 \
-      --concurrency 64 --num-requests 128 --run-id 9505 --shared-frac 0 \
-      --pool-override 2300000 --output-len 512 \
-      --dump-records /root/bench_logs/xx/point_records.jsonl
-"""
-import argparse
-import datetime
-import json
-import math
-import re
-import statistics
-import subprocess
-import time
-from concurrent.futures import ThreadPoolExecutor
-
-import requests
-
-OUTPUT_LEN_DEFAULT = 512
-CORPUS_DEFAULT = "/root/corpus_ids.json"
-
-# fixed pool layout (see docstring)
-S1_128K_RID0, S1_64K_RID0, S2_RID0 = 9301, 9305, 9311
-POOL_A_BASE, POOL_A_PER = 131072, 8 * 13104          # 128k suffix windows
-POOL_B_BASE = POOL_A_BASE + 4 * POOL_A_PER           # 550400
-POOL_B_PER = 8 * 6560                                 # 64k suffix windows
-POOL_C_BASE = POOL_B_BASE + 4 * POOL_B_PER           # 760320
-POOL_C_SIZES = [16 * 16384, 32 * 16384, 32 * 16384]  # cc8 / cc16 / cc32
-SPARE_BASE = POOL_C_BASE + sum(POOL_C_SIZES)         # 2071040
-
-sess = requests.Session()
-sess.trust_env = False  # bypass any proxy env on the host
-
-
-def split_lens(input_len, shared_frac):
-    # unique suffix = (1 - shared_frac) of the prompt, page-16 aligned
-    # (128k @0.9 -> 13104 unique; 16k @0.0 -> fully unique prompts)
-    unique = round(input_len * (1.0 - shared_frac) / 16) * 16
-    return input_len - unique, unique
-
-
-def pool_start_for(input_len, shared_frac, run_id, override):
-    if override is not None:
-        return override
-    if shared_frac > 0:
-        if input_len == 131072:
-            idx = run_id - S1_128K_RID0
-            if not 0 <= idx < 4:
-                sys_exit_bad_runid(run_id, "128k points use run-ids 9301-9304")
-            return POOL_A_BASE + idx * POOL_A_PER
-        if input_len == 65536:
-            idx = run_id - S1_64K_RID0
-            if not 0 <= idx < 4:
-                sys_exit_bad_runid(run_id, "64k points use run-ids 9305-9308")
-            return POOL_B_BASE + idx * POOL_B_PER
-        sys_exit_bad_runid(run_id, "shared-frac>0 supports 131072/65536 only")
-    idx = run_id - S2_RID0
-    if not 0 <= idx < 3:
-        sys_exit_bad_runid(run_id, "16k unique points use run-ids 9311-9313")
-    return POOL_C_BASE + sum(POOL_C_SIZES[:idx])
-
-
-def sys_exit_bad_runid(run_id, msg):
-    raise SystemExit(f"[pool] run-id {run_id} outside expected set: {msg}")
-
-
-def build_prompts(ids, shared_len, unique_len, num_requests, pool_start):
-    if shared_len:
-        shared = ids[0:shared_len]
-    else:
-        shared = []
-    end = pool_start + num_requests * unique_len
-    if end > len(ids):
-        raise SystemExit(
-            f"[pool] window [{pool_start}, {end}) exceeds corpus ({len(ids)} ids); "
-            f"use --pool-override or a larger corpus")
-    prompts = []
-    for i in range(num_requests):
-        s = pool_start + i * unique_len
-        prompts.append(shared + ids[s:s + unique_len])
-    return shared, prompts, (pool_start, end)
-
-
-def warmup(url, ids, shared_len):
-    # primes the radix cache with the shared prefix (same role as in bench_hit90);
-    # warm slice comes from the spare region so it never collides with a pool window
-    if len(ids) >= SPARE_BASE + 64:
-        warm_slice = ids[SPARE_BASE:SPARE_BASE + 64]
-    else:
-        warm_slice = ids[-64:]
-    payload = {
-        "input_ids": ids[0:shared_len] + warm_slice if shared_len else warm_slice,
-        "sampling_params": {"max_new_tokens": 8, "temperature": 0.0, "ignore_eos": True},
-    }
-    t0 = time.perf_counter()
-    r = sess.post(url, json=payload, timeout=1800)
-    dt = time.perf_counter() - t0
-    print(f"[warmup] http={r.status_code} wall={dt:.2f}s", flush=True)
-
-
-def bench_one(url, prompt, output_len, idx, results):
-    payload = {
-        "input_ids": prompt,
-        "sampling_params": {"max_new_tokens": output_len, "temperature": 0.0, "ignore_eos": True},
-        "stream": True,
-    }
-    rec = {"idx": idx}
-    t0 = time.perf_counter()
-    first = last = None
-    first_ct = None
-    final_meta = None
-    max_ct = 0
-    try:
-        with sess.post(url, json=payload, stream=True, timeout=3600) as resp:
-            for raw in resp.iter_lines():
-                if not raw or not raw.startswith(b"data:"):
-                    continue
-                body = raw[5:].strip()
-                if body == b"[DONE]":
-                    continue
-                now = time.perf_counter()
-                try:
-                    d = json.loads(body)
-                except Exception:
-                    continue
-                mi = d.get("meta_info") or {}
-                ct = mi.get("completion_tokens") or 0
-                if ct:
-                    max_ct = max(max_ct, ct)
-                if first is None:
-                    first = now
-                    first_ct = ct
-                last = now
-                if mi.get("finish_reason"):
-                    final_meta = mi
-        t_end = time.perf_counter()
-        n_out = max_ct or ((final_meta or {}).get("completion_tokens") or 0)
-        decode_span = (last - first) if (first and last and last > first) else 0.0
-        rec.update(
-            ok=n_out > 0,
-            ttft=(first - t0) if first else None,
-            e2e=t_end - t0,
-            n_out=n_out,
-            first_chunk_tokens=first_ct,
-            decode_span=decode_span,
-            tpot=(decode_span / (n_out - 1)) if n_out > 1 else None,
-            per_req_decode_tok_s=(n_out / decode_span) if decode_span > 0 else None,
-            retractions=(final_meta or {}).get("num_retractions"),
-            spec_accept_len=(final_meta or {}).get("spec_accept_length"),
-        )
-    except Exception as e:
-        rec.update(ok=False, error=repr(e))
-    results[idx] = rec
-
-
-def verify_hit_rate(container, t_start, t_end):
-    def rfc3339(epoch):
-        return (datetime.datetime.fromtimestamp(epoch, tz=datetime.timezone.utc)
-                .isoformat().replace("+00:00", "Z"))
-
-    try:
-        # No margin before t_start: warmup's prefill lines end strictly before it,
-        # and catching them would deflate the measured hit rate.
-        p = subprocess.run(
-            ["docker", "logs", container, "--since", rfc3339(t_start), "--until", rfc3339(t_end + 2)],
-            capture_output=True, text=True, timeout=120)
-        text = p.stdout + p.stderr
-    except Exception as e:
-        return {"error": repr(e)}
-    pat = re.compile(r"#new-token: (\d+).*?#cached-token: (\d+)")
-    n_batches = new_tok = cached_tok = 0
-    for line in text.splitlines():
-        if "TP0]" not in line or "Prefill batch" not in line:
-            continue
-        m = pat.search(line)
-        if m:
-            n_batches += 1
-            new_tok += int(m.group(1))
-            cached_tok += int(m.group(2))
-    total = new_tok + cached_tok
-    return {
-        "prefill_batches": n_batches,
-        "new_tokens": new_tok,
-        "cached_tokens": cached_tok,
-        "hit_rate": round(cached_tok / total, 4) if total else None,
-    }
-
-
-def stats(vals):
-    vals = [v for v in vals if v is not None]
-    if not vals:
-        return {"mean": None, "p50": None, "p95": None, "max": None, "min": None}
-    s = sorted(vals)
-    # nearest-rank p95: smallest value >= 95th percentile
-    p95_idx = max(0, math.ceil(0.95 * len(s)) - 1)
-    return {
-        "mean": round(statistics.fmean(vals), 4),
-        "p50": round(s[len(s) // 2], 4),
-        "p95": round(s[p95_idx], 4),
-        "max": round(s[-1], 4),
-        "min": round(s[0], 4),
-    }
-
-
-def main():
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--concurrency", type=int, required=True)
-    ap.add_argument("--num-requests", type=int, required=True)
-    ap.add_argument("--run-id", type=int, required=True)
-    ap.add_argument("--input-len", type=int, required=True, help="token length of each prompt")
-    ap.add_argument("--output-len", type=int, default=OUTPUT_LEN_DEFAULT)
-    ap.add_argument("--shared-frac", type=float, default=0.9)
-    ap.add_argument("--corpus", default=CORPUS_DEFAULT)
-    ap.add_argument("--pool-override", type=int, default=None,
-                    help="explicit corpus offset for the unique-suffix window (re-runs)")
-    ap.add_argument("--url", default="http://127.0.0.1:30000/generate")
-    ap.add_argument("--container", default="glm53-nvfp4")
-    ap.add_argument("--dump-records", default=None,
-                    help="write per-request records as JSONL to this path")
-    args = ap.parse_args()
-
-    with open(args.corpus) as f:
-        corpus = json.load(f)
-    ids = corpus["ids"]
-
-    shared_len, unique_len = split_lens(args.input_len, args.shared_frac)
-    pool_start = pool_start_for(args.input_len, args.shared_frac, args.run_id, args.pool_override)
-    shared, prompts, window = build_prompts(ids, shared_len, unique_len, args.num_requests, pool_start)
-    print(f"[pool] window={window} shared_len={shared_len} unique_len={unique_len} "
-          f"corpus_total={len(ids)}", flush=True)
-    warmup(args.url, ids, shared_len)
-
-    results = {}
-    t_start = time.time()
-    t0 = time.perf_counter()
-    with ThreadPoolExecutor(max_workers=args.concurrency) as ex:
-        futs = [ex.submit(bench_one, args.url, p, args.output_len, i, results)
-                for i, p in enumerate(prompts)]
-        for f in futs:
-            f.result()
-    wall = time.perf_counter() - t0
-    t_end = time.time()
-
-    if args.dump_records:
-        with open(args.dump_records, "w") as f:
-            for i in sorted(results):
-                f.write(json.dumps(results[i]) + "\n")
-
-    hit = verify_hit_rate(args.container, t_start, t_end)
-
-    ok = [r for r in results.values() if r.get("ok")]
-    n_out_total = sum(r["n_out"] for r in ok)
-    out_tps = [r["n_out"] / r["e2e"] for r in ok if r.get("e2e")]
-    ttft = stats([r.get("ttft") for r in ok])
-    tpot = stats([r.get("tpot") for r in ok])
-    e2e = stats([r.get("e2e") for r in ok])
-    dec = stats([r.get("per_req_decode_tok_s") for r in ok])
-    spec = [v for v in (r.get("spec_accept_len") for r in ok) if v is not None]
-    retr = sum(r.get("retractions") or 0 for r in ok)
-
-    summary = {
-        "concurrency": args.concurrency,
-        "num_requests": args.num_requests,
-        "run_id": args.run_id,
-        "corpus_window": {"start": window[0], "end": window[1]},
-        "ok": len(ok),
-        "failed": args.num_requests - len(ok),
-        "wall_s": round(wall, 2),
-        "input_len": args.input_len,
-        "shared_len": shared_len,
-        "unique_len": unique_len,
-        "output_len": args.output_len,
-        "output_tokens_total": n_out_total,
-        "output_throughput_tok_s": round(n_out_total / wall, 2) if wall else None,
-        "input_throughput_tok_s": round(args.input_len * len(ok) / wall, 2) if wall else None,
-        "ttft_s": ttft,
-        "tpot_s": tpot,
-        "e2e_s": e2e,
-        "per_req_out_tok_s_e2e": stats(out_tps),
-        "per_req_decode_tok_s": dec,
-        "spec_accept_length_mean": round(statistics.fmean(spec), 3) if spec else None,
-        "retractions_total": retr,
-        "cache_hit_from_logs": hit,
-    }
-    print("\n===== SUMMARY =====")
-    print(json.dumps(summary, indent=2), flush=True)
-
-
-if __name__ == "__main__":
-    main()

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/scripts/control_mrr48_41c32.sh

@@ -1,122 +0,0 @@
-#!/bin/bash
-# control_mrr48_41c32.sh — single-variable control for the 4.1 c32 anchor finding
-#
-# Finding to attribute: pp4mrr64 (MRR 48->64, everything else identical) measures
-# 4.1 c32 = 230.2 (pass-1) and 228.1 (v2, fresh instance, retraction-free) vs
-# 275.6 on the original MRR48 arm (-16~17%). Retraction contamination is ruled
-# out (v2). Remaining confound: the original 275.6 was the 9th point on a
-# long-warmed instance. This control redeploys the ORIGINAL MRR48 config
-# (deploy_glm53_pp4.sh, md5 def3c64c*) as a FRESH instance and measures the
-# same 4.1 c32 point first. ~276 => the regression is a real MRR64 cost;
-# ~230 => the old value was an instance-age artifact and MRR64 is free.
-# Afterwards chains restore_insvc_pp4mrr64.sh (in-service restoration runs
-# regardless of the control outcome).
-#
-# Usage: nohup bash /root/control_mrr48_41c32.sh \
-#          > /root/bench_logs/control_mrr48.log 2>&1 &
-set -u
-
-V2DIR=/root/bench_logs/b300eq_pp4mrr64v2_20260910_2021
-DEPLOY=/root/deploy_glm53_pp4.sh
-RESTORE=/root/restore_insvc_pp4mrr64.sh
-CORPUS=/root/corpus_ids.json
-BENCH=/root/bench_corpus_v2.py
-EXTRACT=/root/extract_summary.py
-URL=http://127.0.0.1:30000
-
-CTLFAIL=0
-echo "=== [ctl] waiting for v2 matrix completion $(date +%T) ==="
-ok=0
-for i in $(seq 1 240); do
-  n=$(wc -l < "$V2DIR/status.txt" 2>/dev/null || echo 0)
-  if [ "$n" -ge 6 ]; then ok=1; break; fi
-  sleep 30
-done
-if [ "$ok" != "1" ]; then
-  echo "=== [ctl] ABORT: v2 did not reach 6 status lines in 2h — going straight to restore ==="
-  CTLFAIL=1
-else
-  echo "--- v2 final status:"; cat "$V2DIR/status.txt"
-
-  echo "=== [ctl] md5 gate on original deploy script $(date +%T) ==="
-  md5sum "$DEPLOY"
-  if ! md5sum "$DEPLOY" | grep -q "^def3c64c"; then
-    echo "=== [ctl] ABORT: deploy_glm53_pp4.sh md5 mismatch (expect def3c64c*) — restore only ==="
-    CTLFAIL=1
-  fi
-fi
-
-if [ "$CTLFAIL" = "0" ]; then
-  echo "=== [ctl] removing v2 test container glm53-pp4 $(date +%T) ==="
-  for i in 1 2 3 4 5; do
-    docker rm -f glm53-pp4 >/dev/null 2>&1 && break
-    echo "[warn] docker rm retry $i, waiting 25s"; sleep 25
-  done
-  echo "=== [ctl] waiting VRAM drain <2000MiB $(date +%T) ==="
-  for i in $(seq 1 90); do
-    used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | sort -rn | head -1)
-    [ "$used" -lt 2000 ] && break
-    sleep 10
-  done
-  nvidia-smi --query-gpu=index,memory.used --format=csv,noheader
-
-  echo "=== [ctl] deploying ORIGINAL MRR48 config $(date +%T) ==="
-  bash "$DEPLOY" > /root/bench_logs/deploy_mrr48ctl.log 2>&1
-  rc=$?
-  echo "=== [ctl] deploy rc=$rc $(date +%T) ==="
-  if [ "$rc" != "0" ]; then
-    tail -30 /root/bench_logs/deploy_mrr48ctl.log; CTLFAIL=1
-  fi
-fi
-
-if [ "$CTLFAIL" = "0" ]; then
-  echo "=== [ctl] waiting for 'fired up' $(date +%T) ==="
-  ok=0
-  for i in $(seq 1 90); do
-    docker logs glm53-pp4 2>&1 | grep -q "The server is fired up" && { ok=1; break; }
-    sleep 10
-  done
-  if [ "$ok" != "1" ]; then
-    echo "=== [ctl] ABORT: no fired up — restore only ==="; CTLFAIL=1
-  else
-    echo "=== [ctl] config gate: expect max_running_requests=48, pool 1040384 $(date +%T) ==="
-    docker logs glm53-pp4 2>&1 | grep -E "max_total_num_tokens" | tail -4
-    if ! docker logs glm53-pp4 2>&1 | grep -q "max_running_requests=48"; then
-      echo "=== [ctl] CONFIG MISMATCH (MRR != 48) — skipping control point, restore only ==="
-      CTLFAIL=1
-    fi
-  fi
-fi
-
-if [ "$CTLFAIL" = "0" ]; then
-  STAMP=$(date +%Y%m%d_%H%M)
-  LOG=/root/bench_logs/b300eq_mrr48ctl_${STAMP}
-  mkdir -p "$LOG"
-  echo "=== [ctl] control point b41_1k_c32 (MRR48 fresh) $(date +%T) LOGDIR=$LOG ==="
-  # same protocol as run_retest_pp4mrr64_v2.sh run_point: prewarm -> flush -> bench -> extract
-  timeout 1800 python3 "$BENCH" --corpus "$CORPUS" --input-len 1024 --output-len 16 \
-    --shared-frac 0 --concurrency 1 --num-requests 1 --run-id 9699 \
-    --pool-override 4500000 --url $URL/generate --container glm53-pp4 \
-    > "$LOG/prewarm_1024.log" 2>&1 || true
-  curl -s -m 60 -X POST $URL/flush_cache >/dev/null; sleep 3
-  timeout 7200 python3 "$BENCH" --corpus "$CORPUS" --input-len 1024 --output-len 128 \
-    --shared-frac 0 --concurrency 32 --num-requests 64 --run-id 9681 \
-    --pool-override 4500000 --url $URL/generate --container glm53-pp4 \
-    --dump-records "$LOG/b41_1k_c32_records.jsonl" > "$LOG/b41_1k_c32.log" 2>&1
-  brc=$?
-  echo "=== [ctl] bench rc=$brc $(date +%T) ==="
-  if [ "$brc" = "0" ]; then
-    python3 "$EXTRACT" "$LOG/b41_1k_c32.log" b41_1k_c32 pp4mrr48ctl "$LOG/all_results.jsonl"
-    grep -E "hit_rate=" "$LOG/b41_1k_c32.log" | tail -1
-  else
-    echo "=== [ctl] control point FAILED rc=$brc ==="; CTLFAIL=1
-  fi
-  nvidia-smi --query-gpu=index,memory.used --format=csv,noheader > "$LOG/vram_after_point.txt"
-fi
-
-echo "=== [ctl] chaining in-service restoration (always) $(date +%T) ctlfail=$CTLFAIL ==="
-bash "$RESTORE" >> /root/bench_logs/restore_insvc_pp4mrr64.log 2>&1
-rrc=$?
-echo "=== [ctl] restore rc=$rrc $(date +%T) ==="
-tail -12 /root/bench_logs/restore_insvc_pp4mrr64.log
-echo "=== [ctl] DONE ctlfail=$CTLFAIL restore_rc=$rrc $(date) ==="

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/scripts/deploy_glm53_e7b_hicc.sh

@@ -1,69 +0,0 @@
-#!/bin/bash
-# deploy_glm53_e7b_hicc.sh — E7b high-concurrency variant for the 60.8 retest
-# Base: deploy_glm53_607_exp.sh (md5 21db641f1d997e26fcf1ddc97163b87e) with ONLY:
-#   1) MRR env param (default 64) replaces hardcoded --max-running-requests 16
-#   2) decode CUDA graph bucket extended 1..64 (was "1 2 3 4 6 8", max 8);
-#      prefill graph max stays 8 (single-variable change vs E7b: MRR + decode graph)
-# Everything else identical recipe: image nightly-dev-20260828-daf63171, EAGLE 4/1/5,
-# mem0.90, chunk8192, hicache3, ctx270336, parsers, CAR 1stage patch flow.
-# Env: MEMFRAC(0.90) STEPS(4) TOPK(1) DRAFT(5) CTXLEN(270336) CHUNK(8192)
-#      MAXPRE(16384) MRR(64) GMAXD(64) GBSD("1 2 3 4 6 8 12 16 24 32 48 64")
-#      RESTART(no|yes) EXTRA("") CAR_PATCH(1)
-set -e
-MEMFRAC=${MEMFRAC:-0.90}
-STEPS=${STEPS:-4}
-TOPK=${TOPK:-1}
-DRAFT=${DRAFT:-5}
-CTXLEN=${CTXLEN:-270336}
-CHUNK=${CHUNK:-8192}
-MAXPRE=${MAXPRE:-16384}
-MRR=${MRR:-64}
-GMAXD=${GMAXD:-64}
-GBSD=${GBSD:-"1 2 3 4 6 8 12 16 24 32 48 64"}
-EXTRA=${EXTRA:-}
-if [ "$RESTART" = "yes" ]; then RP="--restart unless-stopped"; else RP="--restart no"; fi
-
-echo "[deploy] removing old container (if any, exact-name match only)"
-for i in $(seq 1 45); do
-  CID=$(docker ps -a --filter name=^/glm53-nvfp4$ -q)
-  [ -z "$CID" ] && break
-  docker rm -f glm53-nvfp4 >/dev/null 2>&1 || true
-  sleep 2
-done
-
-echo "[deploy] waiting for VRAM drain"
-for i in $(seq 1 45); do
-  used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | awk '{s+=$1} END {print s}')
-  [ "$used" -lt 2000 ] && break
-  sleep 2
-done
-used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | awk '{s+=$1} END {print s}')
-echo "[deploy] VRAM now: ${used} MiB total"
-
-echo "[deploy] starting: TP8 EAGLE ${STEPS}/${TOPK}/${DRAFT} memfrac=${MEMFRAC} mrr=${MRR} decode-graph<=${GMAXD} chunk=${CHUNK} extra='${EXTRA}'"
-docker run -d --name glm53-nvfp4 --gpus all --shm-size 64g --ipc=host $RP \
-  -p 30000:30000 -v /data/hf_models:/data/hf_models \
-  lmsysorg/sglang:nightly-dev-20260828-daf63171 \
-  python3 -m sglang.launch_server \
-  --model-path /data/hf_models/GLM-5.3-NVFP4 --tp 8 \
-  --mem-fraction-static $MEMFRAC --max-running-requests $MRR \
-  --chunked-prefill-size $CHUNK --max-prefill-tokens $MAXPRE \
-  --disable-shared-experts-fusion --moe-runner-backend flashinfer_cutlass --disable-flashinfer-autotune \
-  --speculative-algorithm EAGLE --speculative-num-steps $STEPS --speculative-eagle-topk $TOPK --speculative-num-draft-tokens $DRAFT \
-  --kv-cache-dtype fp8_e4m3 --enable-hierarchical-cache --hicache-ratio 3 \
-  --cuda-graph-max-bs-decode $GMAXD --cuda-graph-bs-decode $GBSD --cuda-graph-max-bs-prefill 8 \
-  --context-length $CTXLEN --reasoning-parser glm45 --tool-call-parser glm47 \
-  --host 0.0.0.0 --port 30000 $EXTRA
-
-echo "[deploy] container started; poll: docker logs -f glm53-nvfp4"
-
-# --- CAR 1stage patch injection (E7b winner) ---
-if [ "${CAR_PATCH:-1}" != "0" ] && [ -f /root/patches/custom_all_reduce.py ]; then
-  echo "[deploy] CAR_PATCH: injecting custom-AR 1stage patch"
-  docker stop -t 20 glm53-nvfp4 >/dev/null 2>&1 || true
-  DPATH=/sgl-workspace/sglang/python/sglang/srt/distributed/device_communicators
-  docker cp /root/patches/custom_all_reduce.py glm53-nvfp4:$DPATH/custom_all_reduce.py
-  docker cp /root/patches/custom_all_reduce_utils.py glm53-nvfp4:$DPATH/custom_all_reduce_utils.py
-  docker start glm53-nvfp4
-  echo "[deploy] CAR_PATCH injected, container restarted; poll health as usual"
-fi

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/scripts/deploy_glm53_pp4_mrr64.sh

@@ -1,57 +0,0 @@
-#!/bin/bash
-# ============================================================
-# GLM-5.3-NVFP4 D 生产口径部署（6000D 8卡，TP2 PP4）— MRR64 复测版
-# 2026-09-10
-#
-#   - 基线：deploy_glm53_pp4.sh（md5 def3c64c，B300 对标战役 TP2PP4 臂）
-#     唯一差异 = MRR 48 -> 64（用户决策：并发上限提到 64）
-#   - decode CUDA graph 保持栈默认覆盖 bs<=256（含 64，桶列表含 56/64），
-#     本臂从未有掉图问题；MRR48 才是 c64 点的活跃上限（48 活跃+16 排队）
-#   - 其余配方逐字不变：mem0.85、cps16384、radix 关、无投机、
-#     index_topk_freq=4 override、镜像 nightly-dev-20260828-daf63171
-#
-# 用法：nohup bash /root/deploy_glm53_pp4_mrr64.sh > <log> 2>&1 &
-# ============================================================
-set -uo pipefail
-
-CONTAINER="glm53-pp4"
-IMAGE="lmsysorg/sglang:nightly-dev-20260828-daf63171"
-MODEL="/data/hf_models/GLM-5.3-NVFP4"
-PORT=30000
-TP=2; PP=4; MEM=0.85; MRR=64; CPS=16384
-
-docker rm -f ${CONTAINER} 2>/dev/null || true
-sleep 2
-
-docker run -d --name ${CONTAINER} --gpus all --shm-size 64g --ipc=host \
-  --restart unless-stopped \
-  -p ${PORT}:${PORT} \
-  -v /data/hf_models:/data/hf_models \
-  ${IMAGE} \
-  python3 -m sglang.launch_server \
-    --model-path ${MODEL} \
-    --tp-size ${TP} --pp-size ${PP} \
-    --mem-fraction-static ${MEM} \
-    --max-running-requests ${MRR} \
-    --disable-radix-cache \
-    --disable-shared-experts-fusion \
-    --moe-runner-backend flashinfer_cutlass \
-    --disable-flashinfer-autotune \
-    --disable-custom-all-reduce \
-    --chunked-prefill-size ${CPS} \
-    --host 0.0.0.0 --port ${PORT} \
-    --json-model-override-args '{"index_topk_freq": 4}'
-
-echo "容器已启动，等待就绪..."
-for i in $(seq 1 60); do
-  code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:${PORT}/health 2>/dev/null)
-  if [ "$code" = "200" ]; then
-    echo "READY after ${i}0s"
-    docker ps --filter name=${CONTAINER} --format '{{.Names}} {{.Status}}'
-    # 确认 override 生效
-    echo "override args: $(docker inspect ${CONTAINER} --format '{{.Config.Cmd}}' | grep -o 'index_topk_freq[^,}]*')"
-    exit 0
-  fi
-  sleep 10
-done
-echo "TIMEOUT"; exit 1

experiments/pro6000/glm53_nvfp4_b300_equivalent_matrix/scripts/extract_summary.py

@@ -1,39 +0,0 @@
-#!/usr/bin/env python3
-"""extract_summary.py <bench_log> <tag> <arm> <results_jsonl>
-
-Parse the trailing "===== SUMMARY =====" JSON block from a bench_corpus_v2 log,
-append a tagged record to the campaign results JSONL, and print the headline
-metrics. Exit codes: 0 = ok and hit_rate<=0.01 (cold-point validity),
-3 = no SUMMARY block found, 4 = hit_rate above the cold-point threshold.
-"""
-import json
-import sys
-
-HIT_MAX = 0.01
-MARK = "===== SUMMARY ====="
-
-
-def main():
-    logp, tag, arm, outp = sys.argv[1:5]
-    with open(logp, encoding="utf-8", errors="replace") as f:
-        log = f.read()
-    i = log.rfind(MARK)
-    if i < 0:
-        print("NO_SUMMARY")
-        sys.exit(3)
-    s = json.loads(log[i + len(MARK):].strip())
-    with open(outp, "a", encoding="utf-8") as f:
-        f.write(json.dumps({"tag": tag, "arm": arm, "summary": s}) + "\n")
-    hit = (s.get("cache_hit_from_logs") or {}).get("hit_rate")
-    print(f"hit_rate={hit} out_tps={s.get('output_throughput_tok_s')} "
-          f"in_tps={s.get('input_throughput_tok_s')} "
-          f"ttft_p95={(s.get('ttft_s') or {}).get('p95')} "
-          f"tpot_p95={(s.get('tpot_s') or {}).get('p95')} "
-          f"ok={s.get('ok')}/{s.get('num_requests')} retractions={s.get('retractions_total')}")
-    if hit is None:
-        sys.exit(4)
-    sys.exit(0 if float(hit) <= HIT_MAX else 4)
-
-
-if __name__ == "__main__":
-    main()